Task Retries in Azure DevOps Pipelines

Automated processes failing from the intermittent issues (due to connectivity failures etc.) is one of the main challenges you might face while automating build and releases using CI/CD tools. Azure DevOps has introduced new task retry feature as a solution to intermittent failures. You can retry only the failed task multiple times without requeue entire pipeline. This blog is explaining how to use retry in Azure DevOps pipelines.

Following is a PowerShell task in a pipeline which is generating random number and mark task as success if the random number is even, otherwise failing the task with an exception. we have to setup number of retried to enable the feature of retrying on a task failure.

Prevent Employees from using Company Email to Create Azure DevOps Organizations

Individuals who have company email which is added to an organization's Azure Active Directory can create their own personal Azure DevOps organizations using the company email. But, Azure DevOps has a feature to restrict creating Azure DevOps organizations using organization's email. As explained in the previous blog (Making Personal Access Token More Secure in Azure DevOps), Add "Azure DevOps administrator" user role to the Azure DevOps admin user to enable the feature. 

Go to Azure Active Directory Section of Organizations settings page of  Azure DevOps to enable the feature.

Making Personal Access Token More Secure in Azure DevOps

Personal Access Token (PAT) is used to authenticate Azure DevOps. But, once a new PAT created, it can be shared across multiple organization. It might not be the ideal solution when considering the security aspect. So, Azure DevOps has a new feature to restrict the PAT shared across multiple Azure DevOps organizations.

Pre requisites: Azure DevOps organization which connected to Azure AD

Go to Azure DevOps organization settings and select Azure Active Directory under general section. In there, you would be able to connect to Azure Active Directory. 

GitHub Action Pipeline with Terraform State File in Azure Blob

In previous blog, we discussed how to do infrastructure automation while maintaining the Terraform state file in Terraform cloud. But, teams can maintain terraform state file in Azure portal along with  other resources. You can learn how to maintain terraform state file in azure blob in this blog.

You can create a resource group, storage account and container in azure to keep the terraform state file. Then add storage container details to Terraform script as follows.

Note the backend section of the provider.tf below which specifies the Azure blob storage and container to maintain the Terraform state. It is required to create the blob storage container manually before trying out the pipeline.

GitHub Action Pipelines - Deploying Terraform

Terraform is an infrastructure automation tool which enable reliable and repeatable resource deployment in cloud environments. GitHub actions becoming a popular choice of CI/CD for most of the source code repos used in GitHub.

Let's look at how to setup a GitHub action pipeline to deploy Azure resource using Terraform script.

Pre-requisites:

• Terraform workspace
• Terraform API Token
• GitHub account
• Azure subscription

Let's add secret values which we going to use in action pipeline.

Setting up Ansible in Windows using WSL

Ansible is an open-source IT automation engine which is used to achieve several IT infrastructure automation needs such as configuration management, provisioning, deployment and many more. This blog explains how to get started with Ansible in Windows.

Pre-requisites:

• Windows 10 Machine

Let's get started.

First, we need to install Windows Subsystem for Linux (WSL) and Ubuntu in Windows machine as preparation prior to installing Ansible.

Login to Windows machine and open PowerShell in Administrative mode. Run following command to install WSL.

Building a .Net Core project with GitHub Actions

GitHub is one of the well-known version control tool which supports collaborative work among teams. Further, GitHub actions added as latest feature to GitHub to support automation of software delivery life cycle. This blog explains how to setup a build pipeline for .Net core project.

Pre- requisites:

• .Net Core Project pushed to GitHub Repository
• GitHub account

Let's get started.

Go to Actions tab in GitHub and select the existing workflow to setup a pipeline or setup work flow from the scratch. This blog explains how to setup a pipeline by utilizing .NET workflow. So, Select .NET workflow.

Configure Self-Hosted Runners in GitHub

GitHub action is set of tools which helps to automate software development lifecycle including continuous integration and deployment. As any of the automated workflow, GitHub use two type of  machines named as GitHub hosted runners which are hosted by GitHub and self-hosted runners which you can use your own machines to run GitHub actions. This blog explains how to setup a self-hosted runner in GitHub.

Why we need self-hosted runners?

• You can setup secured self-hosted agents when isolating your development environments to enhance the security by limiting the access from and to public internet.
• You can setup self-hosted agents when there are specific custom configurations required in runners

Let's get started.

Pre-requites: 

• Windows Server 2019 64-bit Virtual Machine
• GitHub account

Convert Azure DevOps YAML pipeline to GitHub Action Pipeline

GitHub is the widely used open-source code platform in modern tech world. Many companies started moving the repos and the pipelines to GitHub. If you are already using Azure DevOps YAML pipelines there is good documentation available here providing you with instructions to migrate your pipelines to GitHub Actions. But for you to get started quickly, the migrate Azure DevOps YAML pipelines to GitHub Actions YAML using a converter tool would come in handy. AS mentioned in the converter, the migration tool has approximately ~90% accuracy, depending on the YAML tasks.

Pre- requisites: Azure DevOps YAML pipeline

Resolving Windows Server 2019 Security Vulnerabilities Due to Out Dated TLS

TLS (Transport Layer Security) is used to sent data over internet securely as encrypted data to ensure hackers are unable to see what you transmitted. But, you need to select correct TLS version for your websites. If you use older versions of TLS, it is not secure to use it for sending data. Otherwise, your site would be exposed to cyber attacks such as POODLE,BEAST and many more. If you find your hosting server is vulnerable due to older TLS versions, you can disable it by modifying registry values. This blog explains how to disable TLS 1.0 via a PowerShell script.

This script would check for the available keys in  'HKLM:\SYSTEM\ControlSet001\Control\SecurityProviders\SCHANNEL\Protocols' registry path and add new keys as TLS 1.0\Client

Add another key as  TLS 1.0\Server

Resolving "Error parsing Resource ID {resourceid} ID was missing the `serverfarms` element"

If you are familiar with Terraform, you might have experience with terraform state import, which allows to import state of  the already provisioned resources and generate an infra script. This blog explains about an error which occurred when trying do terraform state import and the fix for it.

When trying to import the state of Azure app service plan as shown in the following command,  it gives a  error.

Terraform command :

Terraform import azurerm_app_service_plan.serviceplan /subscriptions/mysubscription /resourceGroups/myresource group/providers/Microsoft.Web/serverFarms/myplanname

Add Deployment Target Tags on Demand via Azure DevOps Pipeline

While you are using Azure DevOps pipelines to perform  build and deployment, you can select Azure DevOps agent or Deployment group as a build and deployment agent. If we consider about the Deployment group, there can be one or multiple deployments targets. While you are working with deployment group with multiple targets, there can be situations where you need to do a specific changes only for the selected deployment target. You achieve that via adding a new tag to deployment target. This blog explains how to add new tag to deployment target via a script.

prerequisite:

• Azure DevOps deployment group with a deployment target
• Azure DevOps personal access token (PAT)

Let's get started.

New YAML Template Editing Experience in Azure DevOps

Continuous integration and delivery is a mandatory part of today's software development process which can be implemented using several CI/CD tools. Azure DevOps is a tool which can be used to create CI/CD pipelines as classical pipeline or as  YAML pipeline. When you are using YAML pipelines, you might need to use several YAML templates to create a CI/CD pipeline. If you need to do any modifications to the templates, previously you had to select relevant template from the Repo and do the modifications. However, Azure DevOps has introduced new YAML template editor feature which can be used to edit YAML templates easily. 

This blog explains how to work with YAML template editor.

Let's get started.

Modifying Existing Task Groups with the Minimal Impact to Existing Pipelines

Azure DevOps task group is used to group and manage multiple pipeline tasks as a reusable component. While you are working in your projects, there can be requirements to edit existing task group. But, if you directly edit the task group, multiple pipelines get affected by the changes. So, This blog explains how to edit a task group with minimum affect to the existing pipelines.

Prerequisites: 

• Azure DevOps task group use in a pipeline

Let's get started.

Go to Azure DevOps task group section and select a task group. Do changes to the task group and save task group as draft.

Get Response Status Code with PowerShell Invoke-RestMethod

 As DevOps engineers, you get requirements to do scripts to perform several actions on application or get status of the application backend  using API calls. This post explains small issue that we experience while writing PowerShell scripts to trigger APIs.

Issue: Invoke-RestMethod cannot be used to get the return HTTP status from the API. API call failed with mentioning -StatusCodeVariable  is not valid. You can learn more about Invoke-RestMethod from here.

Run PowerShell script automatically after reboot

If you are working with scripting languages, you might have to do various scripts to fulfill requirements such as reboot the machine in the middle of installation until installation fully done. As scenario, You can think of Windows updates and the Reboot and check for further updates. This blog explains how to achieve it using schedule tasks.

Let's try to understand the script step by step.

• Following first few lines of the script is for make sure the script is run as administrator.

Secure Azure Virtual Machines Behind Azure Firewalls

Data Security is one of the most important concern of the IT industry. Hence, each individual of the project team is responsible for the security and encryption of the data. However, as OPS engineers, you have to introduce several data security mechanisms to your company. This blog post explains how to access the Azure Virtual Machine securely. 

Pre requisite: 

• Azure Virtual Machine
• Web app hosted in the IIS server inside the Azure Virtual machine
• Network configurations are done to make Web app accessible from the outside of the VM
• Azure firewall 

As the first step to secure the VM, we remove the public IP from the VM, go to public ip of the VM and click on Dissociate.

Adding AAD user to Azure SQL via Terraform

You might have heard about various authentication mechanisms are used for database access management securely. Azure SQL server also has its own authentication mechanism as single administer account which has control over the entire server. Other than this super admin, you can create second administrator account as Azure AD account. You can easily add AAD (Azure Active Directory) users as  admin to Azure SQL via the Azure portal. However, if you want to deploy Azure SQL server via IaC let's look at using Terraform to add an AAD user as admin to the Azure SQL in this blog.

We can create resource group, Azure Key Vault to store SQL server password, Azure SQL server, Azure AD group using Terraform script. Main.tf of the sample script as follows.

IaC with Terraform for Shared Azure Resources in Multiple Subscriptions

 While you are working with multiple azure subscriptions there might be situations where you need to share infrastructure in more than one subscription. If you try to automate such situation, you can achieve that using Terraform as explained in this post.

Pre-requisites: 

• User should have access to at least two azure subscriptions
• Create resource group in one subscription.

Manageable Infra scripts with Terraform for-each

While we are working on IT projects, you might get requirements such as provisioning hundreds of app services for a project. If you choose to do that task manually, that would get long time to complete the task. As a solution, we can use Infrastructure automation tools such as Azure CLI, ARM template, Terraform, etc. However, we need to write these Infa scripts well organized manner and manageable way. Otherwise, you will get into more trouble of maintaining long and repetitive scripts.

Terraform has several ways such as modules and loops to reduce number of lines of the script. In this blog, we will be discussing the for-each loop of the Terraform.

In the following sample we use main.tf and variable.tf files.  Three app service plans and three app services will be created using the script. Let's go thought the Main.tf file to understand how we define the for-each.