Blog - Artur Tagisow
28 Jan 2022

Azure DevOps rolling release private npm package

This tutorial describes how to - in Azure DevOps - create a private NPM package that is released whenever a PR is merged to its source code. You can find the results of following the tutorial in my public Azure DevOps project here.

We'll create:

  1. a dummy package we'll deploy
  2. an Azure Artifacts feed - where our package will be stored privately and available only to our co-workers - instead of the public npm package registry
  3. An Azure Pipeline that will release the package to our Azure Artifacts feed whenever a PR is merged to the package's repo

1. Initial cleaning of your existing package

  • Add a scope to your package name
    If your package's name is npmpackage, then maybe change it to @my-company-name/npmpackage. This helps with avoiding collisions with packages from the public NPM repository (npmjs.com) and visually separates your internal packages from public ones.
  • Make sure to remove private: true from package.json because a private package can't be published
    We'll have to run npm publish later and this command will fail if your package.json contains private: true. Don't worry - your package will be only accessible to whoever you give access to (eg. coworkers). private: true just means "never publish this package ever - not even to private NPM feeds", so we have to remove it.
  • Add a "main" property
    When someone writes import { hello } from "mypackage" this tells Node from which file to actually import hello from. The value of this property should contain the path to an unminified file that's the build output of your library. In my case (with Vue), I run npm run build (you can see the script below), and that makes a dist/npmpackage.umd.js appear. So I set the value of my "main" property to a path that points to that file.
  • (if you use TypeScript) Add a "types" property
    This is similar to the "main" property. It just points to the .d.ts file which contains the type definitions for the file from your "main" property

All of the above changes are shown here:

- "name": "npmpackage",
+ "name": "@my_super_programming_company/npmpackage",
  "version": "0.1.0",
- "private": true,
+ "main": "./dist/npmpackage.umd.js",
+ "types": "./dist/main.d.ts",
  "scripts": {
    "build": "vue-cli-service build --target lib src/main.ts && tsc",
    "serve": "concurrently npm:serve:*",
    "serve:vue": "vue-cli-service build --watch --target lib src/main.ts",
    "serve:ts": "tsc --watch",
    "lint": "vue-cli-service lint",
    "prepare": "husky install"
 },

2. Creating an Azure Artifacts feed

The main source of JavaScript packages is the NPM registry. The packages there are publicly available like this one: webpack on NPM.

We want to publish a package that's closed source and owned by our organization/employer. The solution is to use Azure Artifacts. It's similar to the NPM registry, but only you can decide who can see and download packages uploaded to it. The best thing is we can use regular npm commands with like npm install etc.

Below are step by step instructions for creating an Azure Artifacts feed (aka "your own private NPM registry"):

  1. Azure Artifacts has its own section in the sidebar on the Azure DevOps website. Click its icon:
    1-azure-artifacts-in-menu.jpg
  2. Next, click the "Create Feed" button:
    2-create-feed-button.jpg
  3. A side menu will apear on the right:
    3-create-feed-form-sidemenu.jpg
    1. Name - In the "Name" field at the top, type in a name for your feed (can be anything you want).
    2. Visbility - My "Visibility" is public because I'm in a public Azure DevOps project. Your Azure DevOps project on the other hand is probably private, so you'll have Visibility: Private in this section. Don't worry - your packages will be private
    3. Scope - this is kinda important. If your Azure DevOps organization has more than one project - do you think they'd also like to use the functionality that's in your NPM package?
      For example, if you're developing an internal component library for a bank, that bank probably has 3-4 Azure DevOps projects within the same organization. This bank would probably be happy if their apps all had the same look and feel of your component library across the board.

      • Creating an Organization-wide feed will allow other projects in the same organization to use your package. If you want that, pick "Organization" here
      • However if you're just trying to separate out logic/concerns into a different place for your project's internal use eg. you want to use this package in 2-3 repositories that are in the same Azure DevOps project, just pick "Project".

      I picked "Organization" because that case is more complex.

  4. You've finished creating an Azure Artifacts feed! You'll see a "Connect to the feed to get started" screen. We won't be connecting to the feed ourselves for now, because our Azure DevOps Pipelines and Release definition will do that for us. You'll need to connect to the feed later when I'll be showing you how to get your application to use/consume your new NPM package.
    4-feed-created-screen.jpg

In the next step we'll build our package, assign it a version and store its build result for a later release.

3. Creating an Azure Pipeline that will build our package and store its dist

This is the step where we'll have to set up the versioning scheme of our package.

3.1. Backstory

Before I set up private NPM packages, my team was using a Git submodule to inject our component library into our single page app.

A git submodule is basically a file that says:

"download the component library's repo at commit 15apsxaj31jklas, into the root (SPA) repo".

If a PR was merged in the component library repo, all you had to do was create a second PR in the SPA and change the git submodule to:

"download the component libary's repo at commit [commit hash after PR was merged in the component library]".

As you can already see, our component library had a rolling release release scheme.

There were:

  • no changelogs
  • no semver
  • no manual publishing of a stable version

It was all just new PR merged = new version published. While Git submodules let us have a rolling release scheme out of the box (just bump the .gitmodules hash), keeping this approach after switching to NPM packages requires some tinkering.

I could abandon the rolling release scheme and ask developers to just run npm publish in their local component library repo to publish the new version. But I felt that'd end in a disaster:

  • developers publishing their local development branches instead of the main branch
  • messed up version numbers that skip some versions (eg. 2.0.1 -> 2.3.1)
  • I felt I'd have lots of complaints about

    I just merged my PR to the component library why the HELL can't I use it yet? Why the extra publish step? Let's just go back to git submodules

You probably wouldn't want to disrupt your team's workflow like that. I decided to adapt the rolling releases to NPM packages. The result I ended up with has the developer concerned only with merging their PR - no need to touch the version numbers or publish anything manually at all.

3.2. Versioning scheme

The versioning scheme for the package is 1.0.[BuildID]

  • The 1.0 part can be anything you want. You can change it to eg. 2.0.[BuildID] when your package reaches an important milestone/makes breaking changes. You can even have your team maintain two versions at once on two separate Git branches - eg. the old, stable, bugfix-only 1.0, and the new 2.0.
  • In Azure DevOps's build pipeline system, [BuildID] is a variable that means:

    How many times did Continuous Integration run since the start of this project?

    It's guaranteed to auto-increment and never appear twice with the same value, which makes it work for rolling-release package versioning.

Keep in mind CI runs in pull requests also count towards the BuildID, so your package's versions will have gaps like 1.0.126 -> 1.0.132 even though between two given released versions only one PR was merged to the main branch.

3.3. Implementation

We'll:

  1. npm run build our package
  2. set the package's version number based on the unique Azure Devops Pipeline variable - BuildID
  3. save the build output and version for this pipeline run for later
    Remember - Azure Pipelines are not meant to publish anything, they should just spit out some .zip file with the build output. We'll release to our package registry later with Azure Releases.

Below I'm using a special azure-pipelines.yml file which lets you save your Azure DevOps pipeline as code instead of using the web UI.

3.3.1. Adding the pipeline's code

In our npm package repository from the previous section, let's paste the below code into the repository root (the same foler where the .git folder is), and name it azure-pipelines.yml. In the next step we will create our own Azure Pipeline that will auto-detect this file.

# We don't want to run certain tasks in PR code-checks 
# (eg. prevent publishing artifacts in unapproved pull requests)
# This is used in `condition: ` statements below
# The $[] wasn't necessary before, but now https://docs.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml#code-try-1 recommends it
variables:
  isNonPrBuild: $[eq(variables['Build.SourceBranch'], 'refs/heads/master')]

steps:
- task: Npm@1
  displayName: 'Install dependencies'
  inputs:
    command: 'custom'
    # i'm using --ignore-scripts to disable husky's pre-commit linting etc.. I'm also using npm ci instead of npm install because it's faster
    customCommand: 'ci --ignore-scripts' 
    verbose: false

# 3.1. "`npm run build` our package"
- task: Npm@1
  displayName: 'Build dummy package'
  inputs:
    command: 'custom'
    customCommand: 'run build'

# 3.2. "set the package's version number based on the unique Azure Devops "Build ID""
  # Bumps "version" property in package.json before publishing
  # --force to suppress "Git working directory not clean" error (Azure Pipelines makes .npmrc dirty)
  # --no-git-tag-version to prevent the error that asks you to set git user.name user.email - which doesn't make sense in CI because that'd be lost after the pipeline finishes
- script: npm version 1.0.$(Build.BuildID) --force --no-git-tag-version # you can change the `1.0` infront if your package reaches an important milestone
  displayName: Set npm package version
  condition:
    eq(variables.isNonPrBuild, true)

# part 1 - "3.3 save the output files for later (so that we can release them with DevOps Releases)"
- task: CopyFiles@2
  displayName: 'Copy lib files'
  inputs:
    TargetFolder: '$(Build.ArtifactStagingDirectory)/npm'  
    Contents: |
      **/**
      !node_modules/**
  condition:
    eq(variables.isNonPrBuild, true)

# part 2 - "3.3 save the output files for later on Azure servers"
- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact: drop'
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
  condition:
    eq(variables.isNonPrBuild, true)

After that, I write git add, git status, git commit and git push to add the new file to the repo:

[artur@t430 npm_package]$ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        new file:   azure-pipelines.yml

[artur@t430 npm_package]$ git commit -m "add azure-pipelines.yml"
> @my_super_programming_company/npmpackage@0.1.0 lint
> vue-cli-service lint
DONE  No lint errors found!
[master fed07c5] add azure-pipelines.yml
1 file changed, 49 insertions(+)
create mode 100644 azure-pipelines.yml

[artur@t430 npm_package]$ git push
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 1.68 KiB | 1.68 MiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Analyzing objects... (3/3) (19 ms)
remote: Storing packfile... done (81 ms)
remote: Storing index... done (68 ms)
To https://dev.azure.com/atagisow/test_project/_git/npm_package
  ae76cfb..fed07c5  master -> master

[artur@t430 npm_package]$

3.3.2. Create a new pipeline

5-create-pipeline.png 6-add-azure-devops-repo.png 7-select-repo.png As you can see our azure-pipelines.yml is already pre-loaded:
8-azurepipelines-yml-autodetected.png

After creating your first pipeline, it'll automatically run for the first time. When it finishes (and succeeds), you'll have the build output of your component library saved as a build artifact for later release thanks to the task: PublishBuildArtifacts@1. Now let's release that saved output.

4. Add release stage

The pipeline will build your package's output files whenever a pull request is merged to the master branch but the package won't be published anywhere yet. For that, we need to create release definition.

  1. Go to Pipelines>Releases and click the "New pipeline button"
    create-release-pipeline.png

  2. release-pipeline-empty-job.png

  3. azure-release-add-artifact.png
  4. The "Source (build pipeline)" field is the build output (aka build "artifact") we've created in the first section — it appeared after the build succeeded in the "Building for the first time" part.
    azure-release-add-artifact-dialog.png
  5. Now that we've picked from where should our release pipeline take the build output, we need to tell Azure DevOps what to do with it. Let's create a release pipeline stage:
    azure-release-add-stage.png

  6. azure-release-npm-task.png
  7. Step descriptions for below image:
    1. Select the task
    2. Select the "custom" command. There's also a "publish" command but it doesn't support auto-calculating which package repository to upload to based on the .npmrc file in the repo
    3. Click the triple dots on the right and find the folder where our package.json is
    4. Type in the command. The –ignore-scripts for future-proofing if you ever decide to use husky
    5. Select the feed we created at the beginning ("Registry I select here"). I have an .npmrc file that points to that feed so I'm using that.
    6. Save your task
      azure-release-npm-task-full.png
  8. Our pipeline is created. Now Azure DevOps is showing us a message that we didn't release anything yet. We'll do that in the next section
    azure-pipeline-created.png

5. Releasing for the first time

  1. You'll need to make the pipeline from step 3 run at least once, so that an artifact is created:
    5-pipeline-success.png
  2. After that, go to the Pipelines>Releases screen and hit the "Create a release" button.
    5-create-release.png
  3. Select an artifact, then release it:
    5-select-artifact.png
  4. After that release is successful, our package should appear in Azure Artifacts: 6-done-2.png