Using Azure Pipelines at Dailymotion for the UWP Team, Continuous integration, creating automated build with YAML [Part 1]

This article will talk how the dailymotion UWP team used Azure Pipelines to create a CI/CD. We will go over how to setup this CI using YAML.

It will be divided into 4 parts:

Part 1 : Continuous integration, Creating automated Builds (x64, x86, Release, Debug, etc) using YAML

Part 2: Running Unit Tests in Azure Pipelines

Part 3: Running UI test with WinAppDrivers in Azure Pipelines

Part 4: Releasing test builds and publishing to the UWP Store

Init (some back story)

Before using Azure pipelines we would commit our code to our repo(s) and not really know (bad) if we had broken something. We had no checks to see what was happening and nothing told us if we had forgotten to commit a file. Also when creating test builds we would have to manually create these builds and manually install them on test machines…

When updating the dailymotion UWP application for the store, I would also have to manual create a build and then publish it to the store. Of course doing this manually it was prone to errors and very time consuming.

The other native teams (iOS and Andoird) are already using CI/CD for their own builds, so the UWP team decided to grow up and move forward. We decided to used Azure Pipelines to create our CI/CD pipeline. We did not want to have to host our own machine (not another thing to manage please!). The idea here was that each commit in the dev/* branchs would triggers the CI pipeline, and each pull request on master would create a new Release version of the app automatically. And lastly we wanted to have a daily x64 debug build that could be tested by our QA and deployed on our test machines.

In other words THE FUTURE!

Crazy right?

Dailymotion UWP Project Structure

We’ll need some script for the build in addition to the source files. Here’s the final structure of the dailymotion solution:

build/
├── pipelines/ 
    ├──azure-pipelines.ci.yml
    ├──azure-pipelines.release.yml
    └──templates/
       ├──build-app.yaml
       ├──build-single-architecture.yaml
       ├──run-ui-tests.yaml
       └──run-unit-tests.yaml
  
└──scripts/ 
     └──UpdateAppxManifestVersion.ps1
src/
├── Dailymotion.AppOnly.sln
└── Dailyotion.*[Folders]
tests/
├── DailymotionUnitTests/DailymotionUnitTests.csproj
├── DailymotionUnitTests.Core/DailymotionUnitTests.Core.csproj
└── DailymotionUITests/DailymotionUITest.csproj
Dailymotion.AllProjects.sln
README.md

FYI: The build folder structure was heavily inspired by the calculator app from Microsoft.

Creating our continuous integration

The first step was to make sure that the application could be built (we will see about testing later on).

Also, we did not need to test all of the different build architectures so we decided that we were only going to build the application in x64 and in debug mode. As of today we are still using the free version of azure pipelines (which only has one host agent) so to keep things simple we only build for one platform.

Our azure-pipelines.ci.yml looks as follows:

trigger:
- dev/*
pr:
- dev/*
name: 0.$(Date:yyMM).$(DayOfMonth)$(Rev:rr).0
#because we dont have a lot of resources today only build in x64
jobs:
- template: ./templates/build-app.yaml
  parameters:
    platform: x64
    buildPlatform: Debug

We are setting the name to a specific build version so that when testing we can see what build is being sent. We are using jobs and templates in Azure pipelines which will allow us to reuse parts of our script later on.

Our templates/build-app.yaml looks as follows:

# This template contains a job to build the app for a single architecture.
parameters:
  platform: ''
  buildPlatform: ''
jobs:
- job: Build${{ parameters.platform }}
  displayName: Build ${{ parameters.platform }}
  pool:
    vmImage: vs2017-win2016
  variables:
    BuildConfiguration: ${{ parameters.buildPlatform }}
    BuildPlatform: ${{ parameters.platform }}
  workspace:
    clean: outputs
  steps:
  - checkout: self
    clean: true
    
  - template: ./build-single-architecture.yaml

Our build-single-architecture.yaml looks as follows:

# This template contains steps to build the app for a single architecture.
# The job containing these steps must set the variables 'BuildConfiguration' and 'BuildPlatform'.
steps:
  - task: NuGetToolInstaller@1
- task: NuGetCommand@2
    displayName: NuGet restore Dailymotion.AllProjects.sln
    inputs:
      command: custom
      arguments: restore Dailymotion.AllProjects.sln -Verbosity Detailed -NonInteractive
#this will allow me to know which build is sent to testing
  - task: PowerShell@2
    displayName: Set version number in AppxManifest
    inputs:
      filePath: $(Build.SourcesDirectory)\build\scripts\UpdateAppxManifestVersion.ps1
      arguments: '-AppxManifest $(Build.SourcesDirectory)\src\Dailymotion\Package.appxmanifest -Version $(Build.BuildNumber)'
- task: VSBuild@1
    displayName: Build solution Dailymotion.AllProjects.sln
    inputs:
      solution: Dailymotion.AllProjects.sln
      vsVersion: 15.0
      msbuildArgs: /p:OutDir=$(Build.BinariesDirectory)\$(BuildConfiguration)\$(BuildPlatform)\ /p:GenerateProjectSpecificOutputFolder=true /p:AppVersion=$(Build.BuildNumber) /t:Publish /p:PublishDir=$(Build.BinariesDirectory)\$(BuildConfiguration)\$(BuildPlatform)\publish\
      platform: $(BuildPlatform)
      configuration: $(BuildConfiguration)
      clean: true
      maximumCpuCount: true
  
  - task: PublishBuildArtifacts@1
    displayName: Publish drop artifact
    inputs:
      artifactName: drop
      pathToPublish: $(Build.BinariesDirectory)
      parallel: true

Here is a quick overview of what happens with this script

- task: NuGetToolInstaller@1

This task initialize our nuget tool (which we will us after to restore the missing libraries)

- task: NuGetCommand@2

This task will restore all of the references and packages for our solution

- task: PowerShell@2

This allows us to set the version of the app to the name of the build (idea taken from the MS calculator)

- task: VSBuild@1

We build our whole solution, UWP, Unit Tests, UI Tests

- task: PublishBuildArtifacts@1

We publish the build to our artifacts drop folder so that we can use these bits later on.

Setting up Azure Pipelines

Now we need to setup these files in Azure Pipelines:

We will need to create a New build Pipeline:

under Pipeline => + New => New Build Pipelines => Your Code => Configure

You will end up with your new build pipeline:

Next we need to specify where our CI YAML files are located because of our file structure:

You can run/queue the build to see what happens, when everything goes as planed you should see this:

You can also see what files have been built by clicking on Artifacts you can even have access to the files built:

Build the YAML Release for Dailymotion

Now we are going to look into building a release version of the dailymotion app using Azure Pipelines.

This release file should only be executed when a pull request is done on the master.

For this we have created a second build pipeline called Build appbundle which builds our solution:

Here is the build/pipelines/azure-pipelines.release.yml file:

trigger:
- master
pr:
- master
pool:
  vmImage: 'VS2017-Win2016'
variables:
  solution: '**/src/Dailymotion.AppOnly.sln'
  buildPlatform: 'x86|x64|ARM'
  buildConfiguration: 'Release'
  appxPackageDir: '$(build.artifactStagingDirectory)\AppxPackages\\'
steps:
- task: NuGetToolInstaller@0
- task: NuGetCommand@2
  displayName: NuGet restore src/Dailymotion.AppOnly.sln
  inputs:
    command: custom
    arguments: restore src/Dailymotion.AppOnly.sln -Verbosity Detailed -NonInteractive
- task: VSBuild@1
  inputs:
    platform: 'x64'
    solution: '$(solution)'
    configuration: '$(buildConfiguration)'
    msbuildArgs: '/p:AppxBundlePlatforms="$(buildPlatform)" /p:AppxPackageDir="$(appxPackageDir)" /p:AppxBundle=Always /p:UapAppxPackageBuildMode=StoreUpload'
- task: CopyFiles@2
  displayName: 'Copy Files to: $(build.artifactstagingdirectory)'
  inputs:
    SourceFolder: '$(system.defaultworkingdirectory)'
    Contents: '**\bin\$(BuildConfiguration)\**'
    TargetFolder: '$(build.artifactstagingdirectory)'
- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: drop'
  inputs:
    PathtoPublish: '$(build.artifactstagingdirectory)'

As you can see it looks very similar to our other yaml files but this time we have buildPlatform set to ‘x86|x64|ARM’ and buildConfiguration to ‘Release’.

It takes 25 minutes to built for x64, x86 and ARM in Release whereas for the x64 in debug it only takes 3 minutes. You need to make sure you setup your builds correctly so that you don’t wast time waiting for your CI to built the app. However this could change when we plan on buying more host agents we could decide to build the app for every platform.

Again, if you click on artifacts we will have access to the Release application bundle.

And that is how we started to simplify our life using Azure Pipelines in the next parts we will see:

Part 2: Running Unit Tests in Azure Pipelines

Part 3: Running UI test with WinAppDrivers in Azure Pipelines

Part 4: Releasing test builds and publishing to the UWP Store

Happy coding