Using Azure Pipelines at Dailymotion for the UWP Team, Running Unit Tests in Azure Pipelines with YAML [Part 2]

5 min readJul 15, 2019

This article will look at how the dailymotion UWP team used Azure Pipelines to run its unit test. We will go over how we setup our CI using YAML.

It will be divided into 4 parts:

Part 1 : Continuous integration, Creating automated Builds (x64, x86, Release, Debug, etc) using YAML

Part 2: Running Unit Tests in Azure Pipelines

Part 3: Running UI test with WinAppDrivers in Azure Pipelines

Part 4: Releasing test builds and publishing to the UWP Store

Running Unit Tests in Azure Pipelines

We will look into how we have setup our different unit tests projects.

Our Dailymotion.AllProjects.sln holds two unit test projects (and all the others but these are the only 2 that we are interested in for now):

DailymotionUnitTests is a Unit Test App project, This can be found under File=> New Project => Windows Universal => Unit Test App:

It allows us to test UWP specific scenarios, like navigation for example.

Next, we have DailymotionUnitTests.Core which is a MSTest Test Project this can be found under: File=> New Project => Test => MSTest Test Project

This is the bigger of our 2 test projects, as it can test all of the code that is not only UWP specific (it can test our shared code with our Xamarin projects for example).

Project Structure

Here is the structure of the dailymotion solution:

build/
├── pipelines/ 
    ├──azure-pipelines.ci.yml
    ├──azure-pipelines.release.yml
    └──templates/
       ├──build-app.yaml
       ├──build-single-architecture.yaml
       ├──run-ui-tests.yaml
       └──run-unit-tests.yaml
  
└──scripts/ 
     └──UpdateAppxManifestVersion.ps1src/
├── Dailymotion.AppOnly.sln
└── Dailyotion.*[Folders]tests/
├── DailymotionUnitTests/DailymotionUnitTests.csproj
├── DailymotionUnitTests.Core/DailymotionUnitTests.Core.csproj
└── DailymotionUITests/DailymotionUITest.csprojDailymotion.AllProjects.sln
README.md

YAML for run-unit-tests.yaml & ci pipeline

First we will look at our azure-pipelines.ci.yml file that will call the run-unit-tests.yaml (which is a template file):

# Universal Windows Platformtrigger:
- dev/*pr:
- dev/*name: 0.$(Date:yyMM).$(DayOfMonth)$(Rev:rr).0#because we dont have a lot of resources today only build in x64
jobs:
- template: ./templates/build-app.yaml
  parameters:
    platform: x64
    buildPlatform: Debug# Unit test for UWP project and Core libraries
- template: ./templates/run-unit-tests.yaml
  parameters:
    platform: x64
    buildPlatform: Debug

Every time a commit is done to one of our development branches we will build and unit test that code.

Next, we can look at our run-unit-tests.yaml file:

# This template contains jobs to run unit tests on the interactive test agents.parameters:
  platform: ''
  buildPlatform: ''jobs:
- job: UnitTests${{ parameters.platform }}
  displayName: UnitTests ${{ parameters.platform }}
  dependsOn: Build${{ parameters.platform }}
  pool:
    name: Hosted Windows 2019 with VS2019
  workspace:
    clean: outputs
  variables:
    skipComponentGovernanceDetection: true
  steps:
  - checkout: none- powershell: Write-Host "##vso[task.setvariable variable=agentInstanceId;isOutput=true]$($env:AgentName -replace '\D+' -as [int])"
    name: LogAgentStep
    displayName: Log this agent's instance for later cleanup
    env:
      AgentName: $(Agent.Name)
  
  - task: DownloadBuildArtifacts@0
    displayName: Download DailymotionUnitTests
    inputs:
      artifactName: drop
      itemPattern: drop/${{ parameters.buildPlatform }}/${{ parameters.platform }}/DailymotionUnitTests/AppPackages/DailymotionUnitTests_1.0.0.0_x64_Debug_Test/**- task: PowerShell@2
    displayName: Install Certificate
    inputs:
      filePath: $(Build.ArtifactStagingDirectory)\drop\${{ parameters.buildPlatform }}\${{ parameters.platform }}\DailymotionUnitTests\AppPackages\DailymotionUnitTests_1.0.0.0_x64_Debug_Test\Add-AppDevPackage.ps1
      arguments: -CertificatePath $(Build.ArtifactStagingDirectory)\drop\${{ parameters.buildPlatform }}\${{ parameters.platform }}\DailymotionUnitTests\AppPackages\DailymotionUnitTests_1.0.0.0_x64_Debug_Test\DailymotionUnitTests_1.0.0.0_x64_Debug.cer -Force- task: VSTest@2
    displayName: Run DailymotionUnitTests
    inputs:
      testAssemblyVer2: $(Build.ArtifactStagingDirectory)\drop\${{ parameters.buildPlatform }}\${{ parameters.platform }}\DailymotionUnitTests\AppPackages\DailymotionUnitTests_1.0.0.0_x64_Debug_Test\DailymotionUnitTests_1.0.0.0_x64_Debug.appx
      otherConsoleOptions: /Platform:${{ parameters.platform }}
      codeCoverageEnabled: true
      publishRunAttachments: true
  
  - task: PublishTestResults@2
    displayName: Publish tests DailymotionUnitTests
    inputs:
      testResultsFormat: 'VSTest'
      testResultsFiles: '**/TEST-*.trx'- task: DownloadBuildArtifacts@0
    displayName: Download DailymotionUnitTests.Core
    inputs:
      artifactName: drop
      itemPattern: drop/${{ parameters.buildPlatform }}/${{ parameters.platform }}/DailymotionUnitTests.Core/**- task: VSTest@2
    displayName: Run DailymotionUnitTests.Core
    inputs:
      testAssemblyVer2: $(Build.ArtifactStagingDirectory)\drop\${{ parameters.buildPlatform }}\${{ parameters.platform }}\DailymotionUnitTests.Core\DailymotionUnitTests.Core.dll
      otherConsoleOptions: /Platform:${{ parameters.platform }}
      codeCoverageEnabled: true
      publishRunAttachments: true- task: PublishTestResults@2
    displayName: Publish tests DailymotionUnitTests.Core
    inputs:
      testResultsFormat: 'VSTest'
      testResultsFiles: '**/TEST-*.trx'

The important parts are that because our project DailymotionUnitTests is a Unit Test App (UWP App) Project, we first need to install this application so that we can afterward run the tests.

Which is why we need to do these steps:

task: DownloadBuildArtifacts@0 : we first need to download the AppPackages
task: PowerShell@2 : Next we need to install the Application and its certificate
task: VSTest@2: lastly we can run the Unit Tests

Because DailymotionUnitTests.Core is a MSTest Test Project we also will need to run task:DownloadBuildArtifacts to download the bits that we need and run task:VSTest@2 to execute the tests. Running the MSTest project is much simpler as you can see.

Here are a example of what the logs will look like in the azure pipeline when your unit test have been correctly executed:

In this DailymotionUnitTests UWP test app we can see that we executed 12 tests and that they all passed. If the tests had failed you would have gotten an error in the web console.

In the Azure Pipelines dashboard you will also be able to see how long the Unit Test took to execute:

The biggest issue we faced was to give the correct Path for the MSTest project to execute correctly if you get warnings like this:

It means that your test were not executed, luckily Microsoft must have figured out that I had fat fingers and/or would miss type something and that is why being able to look into the Artifacts > drop and see what had been built will greatly help you find that missing project or dll.

In my case it was because I had rename the project and forgotten the change the path of where DailymotionUnitTests.Core was located.

You will find your test results here under Test Plan => Runs

It will give you access to this dashboard:

Which gives us a quick overview of what we have and under Test Results tab we can also have a detailed view of the different tests that were executed:

And that was a quick overview of how we were able to set up our Unit Tests when a commit is done on one of our development branch, in the next parts we will see: