Configure YAML Pipeline Using YAML templates in Remote Repo in Azure DevOps

While we are configuring CI/CD pipelines for team projects there can be requirements to configure many CI/CD pipelines. If we could maintain set of CI/CD pipeline templates, it would be very helpful when configuring extensive number of pipelines. In this blog, we are going to discuss how to configure a pipeline when the YAML pipeline templates and YAML pipeline are in two different Azure DevOps repos. Further, you would be able to learn how to build a web app code base in third repo in same team pipeline.

Pre requisites: 

                * Repo containing Web App code 

                * Repo containing YAML CI/CD templates 

                * Repo to create YAML pipeline 

Create .NET 5 Early Access Web App in Azure Using CLI

 

.NET 5 is the latest framework introduced by Microsoft recently and Azure Apps Service support also released at the same time which is good news for .NET developers targeting the applications to run on Azure platform. However, there are few limitations in trying to automate the Infra deployment for .NET 5 early access Web App services in Azure. Famous Infra deployment technologies such as Terraform and Pulumi are yet to support .NET 5 app service automation. Further, Azure Cloud shell also has limited control on this latest feature as Azure CLI is not yet updated. As a solution, this blog explains how to deploy .NET early access infra using specific version of Azure CLI. 

As the first step, get verified the latest CLI version(2.15.1) installed in your working machine. You can use following command to get the current CLI version. 

Sharing Service Connection To Multiple Team Projects

 Azure DevOps has a feature where it can connect with external tools via service connection which allows easily integrate multiple tools required for your project development and deployment purposes. However, when you have multiple Azure DevOps team projects and dedicated service connection for each project, it is not easy to manage service connections. As a solution for this, Azure DevOps has introduced secured and shared service connection between projects. As example, if  the team has one azure subscription for development purposes, they can connect with azure using one shared service connection between Azure DevOps projects.

You can find this feature in Azure DevOps settings section. Go to Azure DevOps settings and select service connections under pipeline section. You can add new service connection or edit existing connection from this page. We use existing service connection to explain the feature in this blog post. Go to security section of the selected service connection.

Push Azure DevOps NuGet Packages to GitHub Packages via Azure DevOps Pipelines

 Azure DevOps has two mechanisms to keep deployable packages. One is publish to pipeline itself and other way is publish to Azure DevOps artifacts feed. We can publish deployable out come of the build pipeline to Azure DevOps feed as NuGet packages. However, integration capability of modern software allows us to share these packages between different tools. This post will explains how to push a  NuGet package in Azure DevOps feed to GitHub via Azure DevOps pipelines.

Prerequisites: 

1.                 NuGet package publish to  Azure DevOps feed
2.                 GitHub PAT

Let's get started.

• Create Azure DevOps deployment pipeline by selecting empty template.
• Add build pipeline as the artifact to the deployment pipeline which is necessary to set continuous deployment triggers.

Azure VM generalize image backup and restore

This blog explains how to get a generalize image backup of Azure VM and restore it. While Azure VM is up and running, VM incur cost due to processing power and disk usage. As a solution to this, if there is an Azure VM in an idle state we can make a VM image out of it including all the configurations of the VM and remove the VM. Hence, the cost can be reduced as the VM image is stored in the storage and it costs for the storage.

Let's get started

Trigger an Azure Build Pipeline Based on Changes in Multiple Azure Git Repos

While we are working in the projects, there can be common tools or utilities developed which are shared with the multiple projects. When developing such projects using Azure DevOps Repos as source control, we can keep common utilities in one repo and other projects in separate repos. However, while we developing utilities in separate repo, once any change done to the utilities, other projects which refer this utility should be get acknowledge and should be able to work with the latest utilities. Hence, once new change done to the utilities code, other dependent projects builds should get triggered automatically and build deployable package with latest utilities version.

Azure DevOps has a feature where you can trigger a build pipeline once a change is done to another repo other than the main code repo. That would help us to achieve the above-mentioned challenge. This post discusses how to trigger a build pipeline due to the code push done to the multiple repos.

Fix Service Connection Configuration Failure in Azure DevOps and Jira Integration

While working in the modern software development teams, you would have teams don't stick to one tool as they always try to use more tools if the tools can integrate with each other. Jira is one of the most famous team/project management tool and Azure DevOps is mostly famous as a CI/CD tool. If a team use both of the tools, it is better to have link between tasks in the Jira and deployments in the Azure DevOps team projects. This blog explains how to integrate Jira and Azure DevOps, further, discuss the issues faced when integrating the two tools and how to solve the issues.

• Let's go to Azure DevOps and check possibility to create a service connection to Jira. You would be able to find the Jira in the service connection type list, however, you would be able to see save button of the service connection creation process disables even after enter the values. It doesn't allow to create the service connection between two tools.

Link Jira Issues with Azure DevOps Deployment

In the previous post we have discussed how to create a service connection between Jira and Azure DevOps. Since, project management tasks and deployment done using two different tools, It is very useful to have a link between the tools to easily track the deployment progress of the each process explain in Jira issue. This post explains how to link Jira work items and Azure DevOps deployment pipeline.

Pre requites:

1.  Azure DevOps Repo with a code sample.
2.  Azure DevOps continuous integration and deployment pipelines to deploy code changes to target platform.

• Open Azure DevOps deployment pipeline in the edit mode.

• Go to integration section in options section of the Azure DevOps deployment pipeline. 

Getting Started with Azure Scale Set Agents

Azure DevOps pipelines use hosted agents or private agents to run the build steps. However, while using the private agents one of the main challenges faced by the users is cost of the agents. As a solution to this we can use Azure Scale set agents as describe in this post. Scale set will create agents on demand and destroy them after use.

• First create a VM with all the required software installed in it.

• Create a VM image out of the VM created . VM image creation is explained in previous post. Click here to see VM image creation post.

• Open Azure Cloud Shell and execute following commands.

• Run following command to select relevant subscription if you have multiple subscriptions.

Prioritizing Waiting Jobs in Azure Pipelines

While we are working with Azure DevOps pipelines for build and deployment purpose, we use several private and hosted agents. However, sometimes there are more CI and PR builds in the queues due to limited number of parallel job availability. While we do daily tasks, there can be situations where we need to release a bug fix quickly but pipelines have long queues. At that type of situation, team has to wait prior until all the builds on the queue get completed. This blog explains the feature available in Azure DevOps as a solution to this issue.

Go to the build logs of the build pipeline which need to be started next. In the logs you would be able to see the position of the current  build . As the following image the selected build is 4 th in the queue. We can click on the "Run next" to queue this build next as priority.

Fix "unknown error: cannot find Chrome binary" Failure in UI Test Execution Using Test Clients in Azure DevOps Pipelines.

 This blog explains one of the issues faced by Test Automation team when executing the UI test scripts using Chrome web driver. Lets have a look at, what is the process Test Automation team followed and the issue encountered. Importantly let's see how it can be resolved. 

In the UI test scripts developed with Spec flow, Selenium and C#, the purpose is to run automated UI test using a CD pipeline. Team has setup a Azure DevOps agent to run in interactive mode and installed the chrome web browser in the agent VM. Once, pipelines are  executing the UI test automation using the test agent it has reported the following error.

System.AggregateException : One or more errors occurred. (unknown error: cannot find Chrome binary) (unknown error: cannot find Chrome binary)

---- OpenQA.Selenium.WebDriverException : unknown error: cannot find Chrome binary

---- OpenQA.Selenium.WebDriverException : unknown error: cannot find Chrome binary

Resolving “Exist soft deleted vault with the same name” at the Time of Azure Keyvault Creation

Azure Keyvault is a well known component among which can be used to store secrets and certificates in a secure way.This post discusses the special feature called soft delete available with Azure Keyvault and how to purge soft deleted key vaults permanently. 

Soft delete is a feature of Azure key vault which allows users recover the deleted -vaults or vault objects such as keys,secrets and certificates for the specified retention period. Minimum retention is 7 days and maximum is 90 and can be only set once for a key vault..Assume a situation where you have enabled the soft delete feature of the key vault and you deleted the vault. After that when you try to create a vault with the same name it gives an error mentioning soft deleted vault is available with the same name. Hence, you cannot create a new vault with the same name.

ERROR: Exist soft deleted vault with the same name

Export Azure DevOps Pipeline Created with Task Groups from One Team Project to Another

While configuring build and deployment pipelines in Azure DevOps , we can create a new pipeline from the scratch or export existing pipeline from another Azure DevOps team project and update the values as relevant to the current project. You can export Azure DevOps build and release pipelines from one project to another project regardless these project in same organization or different organization using the mechanism explained here.

Using System OAuth Token in the Agent Phase of the Azure DevOps Build Pipeline to Access Azure DevOps REST API

Azure DevOps REST API endpoints are service endpoints that allows to create,update,retrieve and delete the Azure DevOps service's resources. While executing these REST API commands it is required to use Authentication mechanisms in order for the API it to perform authorized operations. This post explains how to execute API commands without providing authentication as a Personal Access Token (PAT) in the script execution in a build job.

There is a configuration in the Azure DevOps build pipeline which enable the scripts and other process launched by tasks to access the OAuth token through the SYSTEM.ACCESS.TOKEN variable. When access to the system access token is enabled it is possible to use $env:SYSTEM_ACCESSTOKEN environment variable in the task scripts, that you are executing in a build pipeline job.

Resolving Queue Creation Script Failures Due to Delay in Applying Default Action of a Azure Storage Account in IaC

This post discusses an issue faced, while trying to allow in the default action of a storage account,  running PowerShell based Azure CLI, Infrastructure as code.

Issue : Requirement is to create a new azure storage queue using the PowerShell script in an existing storage account which is attached to a vnet and have access restrictions applied. Hence, the script needs to remove the restrictions to storage by setting Defualt Action to allow and then  create new storage queue. Once the new queue added to the storage, restrict the storage account access again. However, the time delay to apply the removal of restriction sometimes create an a hoc issue as the next step to create the queue fails with, network access error.

Mac OS Installation Help - How did I resolved issues encountered

This post discusses the common issues that you may also face while setting up Mac OS in a Mac Book by wiping out data and trying to do a new installation.

Problem 01 : Ask for admin user credentials when try to restart on recovery mode.

Solution:  Restart the mac. when it shut off and powers back on, press and hold Command + R until you see the apple logo. Instead of going to recovery page directly, some mac versions ask for user credentials. If you doesn't know the credentials still you can do the installation. You can find erase mac option from the top menu of the page. Erase the mac and start the mac installation process.

Setup MLOPS workspace using Azure DevOps pipeline

Machine Learning is training machines using set of data to do some valuable predictions for the different business and social needs. So, data scientists put lot of effort to train more usable and valuable mathematical representation of real world process which is known as a model. To do the training process it is very important to have a good computational resources to train the model. Azure Machine Learning workspace is one of the well-known model training and building platform many data scientists use. If data scientist and DevOps engineers work together, they can automate the model training and deployment process using CI/CD pipelines. This post discusses, how to create a Machine Learning (ML) workspace in Azure by using a Azure DevOps build, which will allow you to use for training and testing a model.

Pre requisites: Azure subscription and an Azure DevOps Account

Read NuGet Artifacts of External Azure DevOps Organization Feed

Azure DevOps artifacts feeds can be used to store different kind of artifacts of projects of your organization created. Sometimes we may need to use these artifacts in outside Azure DevOps organization. This post explains how can share feeds between organizations.

Prerequisites: PAT from the external Azure DevOps organization with feed read permission
                        External organization has feeds with NuGet packages in it