This is a two-way integration with Azure DevOps Pipelines that provides several different integrations. The first allows you to trigger a pipeline from xMatters. You can also have Azure DevOps trigger xMatters from a pipeline task or release gate and use xMatters steps to update the task or gate status.
- Azure DevOps
- Azure DevOps Services account
- Permission to setup Service Connections
- Permission to setup/modify pipelines and releases
- xMatters
- Account - if you don't have one get one for free
- Permissions to create/edit Workflows
- xMatters
- Workflows
- AzureDevOpsPipelines.zip - example workflow containing all the triggers and steps
- Triggers
- buildTask.js - source for trigger to recieve pipeline task notifications
- releaseGate.js - source for trigger to recieve release gate notifications
- Steps
- queueBuild.js - source for step that can trigger a pipeline
- buildTaskStarted.js - source for step to notify that the pipeline task has started
- buildTaskComplete.js - source for step to notify that the pipeline task has completed and its result
- updateReleaseGate.js - source for step to update the release gate status
- checkResponseCount.js - source for step that checks if the response count meets the configured response threshold for an xMatters event. Used in the release gate example
- Shared Libraries
- base64.js - source for library used by some of the steps to encode the callback authentication token
- Workflows
- Azure DevOps
- InvokeRESTAPI_taskPayload.json - body to use with the "Invoke REST API" pipeline task
- InvokeRESTAPI_gatePayload.json - body to use with the "Invoke REST API" gate task
- azure-pipelines.yml - an example pipeline yaml config that can be used to setup the demo
- xMatters Extension - this is an example Azure DevOps extension you can build and install in your organization. The components can be used in place of the Generic service connection and Invoke REST API task/gate in the instructions.
You can use the Azure DevOps - Queue Build step to start the configured pipeline. The example flow is triggered by an xMatters event being created, but the trigger could be anything you like. For example if you use Azure DevOps for operational management if xMatters received a notification of an application issue your flow could trigger an Azure DevOps pipeline that restarts a service.
Using the Azure DevOps Invoke REST API task to send a notification to the Azure DevOps - Build Task trigger we can start an xMatters flow. You can also configure the Azure DevOps Invoke REST API task to implement a callback and in this case you can use the Azure DevOps - Build Task Started and Azure DevOps - Build Task Completed steps to update the build task status.
Invoke REST API (ApiResponse) - If you configure the Azure DevOps Invoke REST API task's Completion event for ApiResponse it will trigger the xMatters flow, but it will continue on a successful response from the request. Use this when you want to trigger xMatters but do not care the outcome of the xMatters flow. In this case if you use the Build Task Started and Build Task Completed steps in your flow they will fail because the task will not exist for the callback.
Invoke REST API (Callback) - [Examle uses this] If you configure the Azure DevOps Invoke REST API task's Completion event for Callback it will trigger the xMatters flow and wait for a callback containing the task status. Use this when you want the task status to reflect the xMatters flow outcome.
Using the Azure DevOps Invoke REST API task as a release gate you can send a notification to the Azure DevOps - Release Gate. In conjunction with the Azure DevOps - Update Release Gate step you can then update from your flow if the release gate succeeded or failed.
- Azure DevOps release gates use a polling mechanism to check the status of the gates. With each poll the callback information changes, so the flow must handle this and be able to respond with the correct callback information.
- As Azure DevOps polls through the gates it is waiting for all to succeed. It will not continue on until all gates succeed, so if the workflow sends a fail to the gate the release pipeline will not move on but it will continue to run until the gate failure timeout is reached. This is true for all gates.
- In most cases when using the Invoke REST API task with a release gate you will want to configure its Completion event to Callback or the gate will pass on successful reponse from the trigger request to xMatters. It will not wait for the xMatters flow to update the gate status.
These instructions will use the Generic service connection and Invoke REST API task/gate. You can use this document to build and install an example Azure DevOps extension that will work with this example.
If you do not already have an xMatters user specifically for integrations it is recommended to create one. This account should only be used for integrations and not need web login permissions.
If the account you choose to use for this integration is removed or made inactive then the integration will break.
-
Under this integration user create a new API key to use with Azure DevOps. Instructions
✏️ Make sure to note the key and secret for later when we configure Azure DevOps.
-
Import the example workflow. Instructions
-
Modify the Build Task and Release Gate form in the imported workflow and give the integration user sender permissions on the form. Instructions
-
Modify the Queue Build form and select the Enable in Web UI and Enable in Mobile App options. Also update the sender permissions for this form, but give an actual user permissions. You will later login to xMatters with this account to trigger the flow and the user needs sender permissions on the form. Instructions
-
Now open the flow Build Task and you will see a trigger at the top of the canvas called "Build Task - Azure DevOps - Build Task". Hover over the trigger step and select edit.
-
On the Settings tab copy the trigger's URL and paste it in your notes for later.
✏️ Make sure to save in your notes and label something like Build Trigger Endpoint for later when we configure Azure DevOps.
-
Repeat steps 5 and 6 for the Release Gate - Azure DevOps - Release Gate trigger in the Release Gate flow.
✏️ Make sure to save the trigger URL in your notes and label something like Release Trigger Endpoint for later when we configure Azure DevOps.
To use the Azure DevOps - Queue Build step we need to setup credentials for xMatters to use to connect to Azure DevOps.
We will be using a Personal Access Token and as good practice you probably want to use a specific Azure DevOps account for integrations.
If you use an actual person's account and it is deactivated or deleted then the integration will break.
-
Login to Azure DevOps as the user you want to create the Personal Access Token under
If you already have a Personal Access Token setup for xMatters (You are using the Azure DevOps Boards integration) you can use the same PTA, but you may need to skip to step 5 and update the permissions.
-
In the top right select the icon with a person with a gear on it. This will open user settings menu where you will select Personal access tokens
-
Select + New Token to create a new Personal Access Token
-
Set the following values
- Name - Give the token a descriptive name so that you know what it is for like "xMatters"
- Organization - set the Azure DevOps organization you want this token to have access too
- The user you are creating the token under must have access to the organization for them to show
- Expiration - select Custom Defined and set it for the farthest date possible (1 year)
-
For this integration the token should only need Read & execute access to Build, but as of writing this I have only tested with Full Access. If you use this token to perform other actions from xMatters it may need more permissions.
-
Now copy the token that is generated.
✏️ Make sure you have a copy because once you close the window you can not see it again and you will need later to finish xMatters integration setup
Azure DevOps will need a Service Connection for the pipeline task and release gate to connect to xMatters.
If you install the example extension you can use the xMatters connection type.
-
Go to Project Settings in the Azure DevOps project you want to setup the xMatters integration and open Service Connections and select New service connection.
-
For connection type select Generic and select Next
-
Set the connection settings as follows and click Save.
- Server URL - your xMatters instance URL (https://example.xmatters.com)
- Username - the API key you created earlier in xMatters
- Password/Token Key - the API secret you created earlier in xMatters
- Service connection name - give the connection a name like "xMatters"
These instructions explain how to setup the Invoke REST API pipeline task to send a notification to the xMatters Azure DevOps - Build Task trigger. The instructions will be setting up a new pipeline using the classic method, but you can use the included yaml config if you like.
If you install the example extension you can use the "xMatters - Workflow HTTP Trigger" task instead of the Invoke REST API task. The yaml configuration is using the Inovke REST API task.
- For learning purposes you probably want to create a Git repo for this example that is accessible from your project. It does not need anything in it.
- I will not go into all aspects of configuring an Azure DevOps pipeline.
- I am assuming you are running the example Workflow in xMatters, so we will be setting up a pipeline that essential only interacts with xMatters for demonstration. You can add to your own pipeline if you wish, but make sure you understand how it works.
- You can use the provided yaml configuration, but be aware that it will be written to the Git repo you have the pipeline configured too.
-
In the project you are setting up the xMatters integration go to Pipelines -> Pipelines and select New pipeline
-
Select "Use the classic editor" at the bottom of the selections. (This is where you can select one of the other options and use the yaml config)
-
Select the repo and default branch you want to connect the pipeline too and select Continue. I am using GitHub, but it can be which ever you like.
-
Select Empty job at the top of the screen.
-
Select Agent job 1 and then click Remove to delete it.
-
Open the Pipeline menu and select Add an agentless job
-
Add a new "Invoke REST API" task to the Agentless job
-
Select the new task and setting the following parameters.
Be sure to modify the parameters in the body as specified. Especially the xmRecipients field or no one will receive the notification.
- Display name - set this to whatever you like
- Connection type - Generic
- Generic service connection - select the service connection you setup earlier to your xMatters instance
- Method - POST
- Headers - clear this field except for "Content-Type":"application/json"
- Body - paste the contents from InvokeRESTAPI_taskPayload.json
- Update the following fields in the body
- xmSubject - set this to what you want the xMatters notification subject to be
- xmMessage - set this to the message you want displayed in the xMatters notification
- xmRecipients - set this with a comma separated list of xMatters users and/or groups you want to receive the notification
- Update the following fields in the body
- URL suffix and parameters - enter the xMatters Azure DevOps - Build Task trigger URL path without the instance domain and starting slash
- Completion event - Callback (If you set this to ApiResponse the task will continue on a successful request to xMatters, but not wait on the flow to complete)
-
We do not want to queue the pipeline yet so select Save only.
These instructions explain how to setup the Invoke REST API task as a release gate to send a notification to the xMatters Azure DevOps - Release Gate trigger.
If you install the example extension you can use the "xMatters - Workflow HTTP Trigger" gate instead of the Invoke REST API gate.
-
In the project you are setting up the xMatters integration go to Pipelines -> Releases and select New -> New release pipeline
-
On the "Select a template" window click Empty job at the top.
-
Name the stage whatever you like and close the window.
-
Select Add an artifact
-
On the "Add an artifact" window enter the following values and click Add
- Source type - Build
- Project - the project you are integrating with xMatters. Most likely the one you are in now.
- Source - select the example pipeline you created earlier
- Default version - the default is fine (Latest)
- Source alias - default is fine
-
Select the Pre-deployment conditions
-
On the "Pre-deployment conditions" window set the trigger to After release and enable Gates
-
Set "The delay before evaluation" to what you like. I set to 1 minute so that it moves on faster for demo purposes.
-
Now click Add to add a gate and select Invoke REST API
-
Set the followin values for the gate
Be sure to modify the parameters in the body as specified. Especially the xmRecipients field or no one will receive the notification.
- Display name - whatever you like
- Connection type - Generic
- Generic service connection - select the service connection you setup to xMatters earlier
- Method - POST
- Headers - clear everything but "Content-Type":"application/json"
- Body - paste the content from InvokeRESTAPI_gatePayload.json
- Update the following fields in the body
- xmSubject - set this to what you want the xMatters notification subject to be
- xmMessage - set this to the message you want displayed in the xMatters notification
- xmRecipients - set this with a comma separated list of xMatters users and/or groups you want to receive the notification
- Update the following fields in the body
- URL suffix and parameters - enter the xMatters Azure DevOps - Release Gate trigger URL path without the instance domain and starting slash
- Completion event - Callback
-
Scroll down to Evaluation options. These options can be set to whatever you need, but I shortened for the demo.
- The time between re-evaluation of gate - I set to the minimum of 5 mins for the demo
- Minimum duration - I left at the default of 0 (all gates pass in same cycle). You can change if you like.
- The timeout after which gates fail - the release waits for all gates to pass. This timeout is how long it will wait before failing the release. For demo purposes I set to 1 hour, but it does not matter.
We will now configure the Endpoints in xMatters so that flow steps can connect to Azure DevOps.
The build queue and pipeline task steps will use a different endpoint than the update release gate step. This is because the release gate update accesses an endpoint at different domain than the others.
The pipeline task and update release steps both use the callback token supplied in the Azure DevOps request. The pipeline task can share the same endpoint as the build queue step because they use the same base URL. In the case of the pipeline task the basic auth credentials are ignored by Azure DevOps and it uses the token.
-
Go back to the example workflow in xMatters and select the FLOWS tab. In the top right open the Components menu and select Endpoints
-
Select the Azure DevOps endpoint.
-
We will need to set or verify these basic settings
- Name - I would use "Azure DevOps" because the steps are already set to use this
- Base URL - https://dev.azure.com
- Trust self-signed certificates - disabled
-
For the endpoint Authentication we need to set the following
- Endpoint Type - Basic
- Username - this will be the Azure DevOps username of the account you created the Personal Access Token
- Password - enter the Personal Access Token
- Preemptive - enabled
-
Click the Save button at the bottom
-
No changes should be required for the Azure DevOps - Release Gate. If you want to verify the settings they should be as follows.
- Name - leave as "Azure DevOps - Release Gate" because the steps are already configured for this
- Base URL - https://vsrm.dev.azure.com
- Trust self-signed certificates - disabled
- Endpoint Type - No Authentication (The update release gate step uses the callback auth token supplied in the request from Azure DevOps)
-
Save any further endpoint updates and close the endpoints window.
You now need to configure the Queue Build flow to point to the example Azure DevOps pipeline you created earlier.
-
Open the Queue Build flow
-
On the canvas hover over the Azure DevOps - Queue Build step and select edit.
-
Now set the values for the step as follows.
- organization - set to the Azure DevOps organization name you created your example pipeline in
- projectName - set to the Azure DevOps project name you created your example pipeline in
- pipelineName - set to the name you gave your example pipeline
-
Now click Done, Save the flow, and you are done with configuration.
In this test we will follow the path that gets all passing results so that you can see all the integrations work.
-
Login to xMatters as one of the users you gave send permission to the Queue Build form
-
Go to Messaging and select Queue Build under the AZURE DEVOPS PIPELINES
-
No need to add recipients here. Just click the Send Message button.
-
Ignore the warning we do not actually want to send a notification for this initial event. This is triggering the Queue Build flow to start which will trigger the example pipeline you created earlier to start.
-
You should see your example pipeline in Azure DevOps start and if you drill into the details you will see it is waiting on the InvokeRESTAPI task. It is waiting on a reply from xMatters.
-
After a short time one of the notification targets you configured in the pipeline task payload field xmRecipients should recieve a notification similar to below (This is viewed from with xMatters app)
-
There will be two options, but select Continue Build for this test. If you select Stop Build it will fail the build.
-
After a short time you should see your example pipeline complete successfully.
-
Now the example release should be triggered and start. You should see it waiting on the release gate to pass.
-
After a short time one of the notification targets you configured in the release gate payload field xmRecipients should recieve a notification similar to below (This is viewed from with xMatters app)
-
There will be two options, select Approve. If you select Decline the gate will never pass and you will have to wait the timeout period for the release to fail.
-
You should now see your release complete. Timing can vary especially if you changed the gate evaluation times.
The example is using an approval type workflows to demonstrate triggering flows, sending notifications, user responses, and updating Azure DevOps. The xMatters flows can include other steps as well. This includes things like creating a ServiceNow ticket, Jira Issue, update a Slack/MS Teams channel, update status page, triggering other automation, of course sending notifications, etc.
Invoke REST API (ApiResponse) - If you configure the Azure DevOps Invoke REST API task's Completion event for ApiResponse it will trigger the xMatters flow, but it will continue on a successful response from the request. Use this when you want to trigger xMatters but do not care the outcome of the xMatters flow. In this case if you use the Build Task Started and Build Task Completed steps in your flow they will fail because the task will not exist for the callback.
Invoke REST API (Callback) - [Examples use this setting] If you configure the Azure DevOps Invoke REST API task's Completion event for Callback it will trigger the xMatters flow and wait for a callback containing the task status. Use this when you want the task status to reflect the xMatters flow outcome.
When customizing for your use cases keep in mind that the Azure DevOps release gates use a polling mechanism. Keep the following points in mind when building or modifying a flow that is triggered by a release gate. The demo workflow tries to manage all these.
- The callback information, except Timeline ID, changes with each poll request and any updates must use this new information. The xMatters flow must take this into account.
- If the flow replies back to the gate poll with a failure the release gate polling will continue. That callback information will no longer be valid, so inorder to later reply with a success the flow must wait for another poll from Azure DevOps. The demo workflow never replies on a failure, because by default no reply is considered a failure by Azure DevOps.
- xMatters
- You can view the log in the Workflow Activity panel to help troubleshoot. Instructions
- Verify that you entered the correct organization, project, and pipeline names
- Verify that you have entered the correct credentials when setting up the Azure DevOps endpoint
- Azure DevOps
- Verify the user and Persnal Access Token have the required permissions
- Azure DevOps
- Drill into the the build logs and check if there are any errors
- Verify that you entered the correct trigger endpoint URL. Do not include the xMatters instance domain and first slash, because this is supplied by the Service Connection.
- Verify the Service Connection is configured with the correct credentials
- xMatters
- Verify the xMatters account used for the Azure DevOps Service Connection has send permissions on the Build Task flow
- xMatters
- You can view the log in the Workflow Activity panel to help troubleshoot. Instructions
- Verify the build task update steps are configured for the Azure DevOps endpoint. Not the Azure DevOps - Release Gate endpoint.
- Verify that the Azure DevOps endpoint is configured correctly. This endpoint will be configured for Basic Auth, but the build task update steps use the auth token that Azure DevOps sends in the request to trigger the flow.
- Make sure the correct outputs of the trigger step are mapped to the inputs of the update steps.
- Azure DevOps
- Make sure you configured the Invoke REST API task to use Callback as the Completion event. If you leave it as ApiResponse it will succeed on a successful response to the initial request to xMatters, which if everything is configured correctly should be every time. Basically it will ignore the workflow outcome.
- Azure DevOps
- Drill into the the logs and check if there are any errors
- Verify that you entered the correct trigger endpoint URL. Do not include the xMatters instance domain and first slash, because this is supplied by the Service Connection.
- Verify the Service Connection is configured with the correct credentials
- xMatters
- Verify the xMatters account used for the Azure DevOps Service Connection has send permissions on the Release Gate flow
Release gates work differently than pipeline tasks. Azure DevOps release polls the release gate endpoints waiting for a success status. So keep in mind.
- With each poll the callback information to update the gate changes except for the timeline ID, so this new information must be used to update the gate.
- If the xMatters flow replies with a failure the Azure DevOps release will continue to run and poll the gates, because it waits for all successes or it's configured timeout.
- If the xMatters flow replies with a failure the callback information will no longer be valid. If the flow then later needs to update the gate as success it will have to wait for a new poll with new callback information.
- xMatters
- You can view the log in the Workflow Activity panel to help troubleshoot. Instructions
- Verify the release gate update steps are configured for the Azure DevOps - Release Gate endpoint. Not the Azure DevOps.
- Verify that the Azure DevOps - Release Gate endpoint is configured correctly. This endpoint will be configured for No Auth and the release gate update steps use the auth token that Azure DevOps sends in the poll request to trigger the flow.
- Make sure the correct outputs of the trigger step are mapped to the inputs of the update steps.
- Azure DevOps
- Make sure you configured the Invoke REST API task to use Callback as the Completion event. If you leave it as ApiResponse it will succeed on a successful response to the initial request to xMatters, which if everything is configured correctly should be every time. Basically it will ignore the workflow outcome.