Automate PagerDuty Maintenance Windows with Azure DevOps

Reliable notifications, scheduled escalations, and effective on-call management are essential for operations teams to quickly identify and address issues. PagerDuty is a platform that provides all of these capabilities, enabling teams to maintain high reliability and reduce operational noise.When working with any operations platform, reliability is key to avoiding unnecessary alerts or actions. For example, during a scheduled system update, creating a maintenance window in PagerDuty can prevent false alarms triggered by expected changes.

In this blog, we'll discuss how to create a PagerDuty maintenance window using an Azure DevOps pipeline. By adding this step to your deployment process, you can automatically start a maintenance window during system updates, improving alert accuracy and reducing manual overhead.

Prerequisites

• An Azure DevOps YAML pipeline with templates
• A PagerDuty account with a valid API token
• The PagerDuty token added as a variable (pagerDutyAPIKey) in the Azure DevOps Library
• PagerDuty service information added as an Azure DevOps Library variable(pagerdutybs)

Below is a sample pipeline step template that will be used to execute the PowerShell script for creating a PagerDuty maintenance window. Let's identify the parameter values that need to be provided as arguments to the PowerShell script, as shown below.

-token = We read the PagerDuty token value from the Azure DevOps variables, and provide it as parameter

-stageName: The pipeline stage name is provided as a parameter.

-accessToken = Pass the predefined Azure DevOps variable $(System.AccessToken) for authentication.

-pagerdutyservices = Retrieves PagerDuty service information from an Azure DevOps Library variable.

steps:

-download: none



-task: PowerShell@2

 displayName: 'Create Mainatenance Window'

 inputs:

    targetType: 'filePath'

    filePath: $(System.DefaultWorkingDirectory)/<path to script>

    arguments: '-token $(pagerDutyAPIKey) -stage ${{parameters.stageName}} -accessToken $(System.AccessToken) -pagerdutyservices $(pagerdutybs)'

We can add the PagerDuty service information to an Azure DevOps variable as shown below.

ex:  '[{"id": "POLUN4D", "type": "service_reference"}]'

Below is the script that is called by the pipeline step shown above.

• The pipeline checks the current stage name, and if the stage is not Production, it does not create a maintenance window.
• If the current stage is Production, then the steps to create the maintenance window are executed.
• Get the current system time as the start time.
• Get 5:30 AM as the end time. (In this example, we assume the maintenance window starts at 5:00 AM and ends at 5:30 AM.)
• Assign the value of the $pagerdutyservice variable to $businessService.
• Create the header section and body for the API call.
• Execute the API call to create the maintenance window.

param(

[string]$token,

[string]$stageName,

[string]$accessToken,

[string]$pagerdutyservice

)


if($stageName -eq "QA")

{

Write-Host "Skipping Pager duty Maintenace window creation for : $stageName"

exit 0

}

else{

Write-Host "Create Pager duty Maintenace window for : $stageName"

$starttime = (Get-Date -Format "yyyy-MM-ddTHH:mm:ssK")

$endTime = (Get-Date -Hour 5 -Minute 30 -Second 0 -Millisecond 0).ToString("yyyy-MM-ddTHH:mm:ssK")

$businessService = $pagerdutyservice | ConvertFrom-Json


$headers=@{}

$headers.Add("Accept","application/json")

$headers.Add("Content-Type","application/json")

$headers.Add("From", "")

$headers.Add("Authorization","Token token=$token")


$body = @{

    "maintenance_window" = @{

        "type"= "maintenance_window"

        "start_time"= $starttime

        "end_time"= $endtime

        "description"= "Project X maintenance Window - $stageName"

        "services" = @($businessService)

        }

        } | ConvertTo-Json -Depth 10

        Write-Host "Request body: $body"

        $response = Invoke-WebRequest -Uri 'https://api.pagerduty.com/maintenance_windows' -Method POST -Headers $headers -ContentType 'application/json' -Body $body -UseBasicParsing


  
Write-Host "Response: $($response.Content)"
}

After executing the script, you will find a maintenance window created with the description "Project X maintenance Window - $stageName" and the affected service set to POLUN4D on PagerDuty.