Creating and Testing Ingest Pipelines with Grok for NGINX Logs in Elasticsearch

The Elastic Stack is widely used for log management, monitoring, analytics, and observability. It includes several tools such as Beats, which collect logs from various systems. However, not all data in these logs is necessary for indexing. To filter, transform, and enrich documents before they are indexed, we can use ingest pipelines. In this blog, we will discuss how to create a simple ingest pipeline and test it.

Automate PagerDuty Maintenance Windows with Azure DevOps

Reliable notifications, scheduled escalations, and effective on-call management are essential for operations teams to quickly identify and address issues. PagerDuty is a platform that provides all of these capabilities, enabling teams to maintain high reliability and reduce operational noise.When working with any operations platform, reliability is key to avoiding unnecessary alerts or actions. For example, during a scheduled system update, creating a maintenance window in PagerDuty can prevent false alarms triggered by expected changes.

In this blog, we'll discuss how to create a PagerDuty maintenance window using an Azure DevOps pipeline. By adding this step to your deployment process, you can automatically start a maintenance window during system updates, improving alert accuracy and reducing manual overhead.

Using StringList Parameters in Azure DevOps Pipelines

Azure DevOps YAML pipelines now support the stringList parameter type, which allows you to present multiple selectable options as checkboxes in the pipeline UI.

This blog explains a pipeline that uses a stringList parameter with three values: dev, qa, and prod. After adding the region parameter as a stringList, the pipeline looks similar to the image below.

Exploring the Full YAML Pipeline View in Azure DevOps

Azure DevOps YAML pipelines can be created using reusable YAML templates that help define parts of the pipeline as modules. By using templates, we can create shorter YAML pipelines and avoid duplication. However, debugging a YAML pipeline with templates is not easy, as we are unable to view the full, expanded pipeline easily. Azure DevOps now supports a new feature that allows us to see the full YAML pipeline of a pipeline run. In this blog post we are going to learn how to navigate to the full YAML pipeline view of the YAML pipeline that uses templates.

Pre-requisites:

• Azure DevOps YAML pipeline that uses templates

Resolve Azure DevOps Build Pipeline Warning, "The windows-2019 runner image is being deprecated, consider switching to windows-2022 (windows-latest) or windows-2025 instead"

The Windows-2019 runner image in Azure DevOps is being deprecated, and pipelines using the Windows-2019 hosted agent display a warning recommending a switch to a newer hosted agent. However, after changing the agent, build pipelines may break due to dependency and environment differences. This blog explains a NuGet restore failure that occurred after switching to Windows-2022 and provides a solution for it.

Warning: The windows-2019 runner image is being deprecated, consider switching to windows-2022 (windows-latest) or windows-2025 instead

Install Kibana on Azure Kubernetes Service (AKS)

Kibana is a data visualization tool used for log analysis and application monitoring use cases. It is a main component of the Elastic Stack. The installation of Elasticsearch on a Kubernetes cluster was explained in the previous blog post. In this blog, let's learn how to install Kibana on Kubernetes.

Pre-requites:

• Azure Kubernetes Cluster
• Full administrative access to the Kubernetes Cluster
• Workstation with kubectl installed
• Elastic Cloud on Kubernetes is installed

Install Elasticsearch Cluster on Azure Kubernetes Service

Elastic search is the first component of the ELK stack that gives powerful search and analytics capabilities to DevOps and development teams. We have discussed how to install Elastic Cloud for Kubernetes (ECK) in previous blog post. In this blog, let's learn how to install elastic stack on Kubernetes.

Pre-requites:

• Azure Kubernetes Cluster
• Full administrative access to the Kubernetes Cluster
• Workstation with kubectl installed
• Elastic Cloud on Kubernetes is installed

Open a shell or use the Azure Kubernetes Service 'Run Command' to run the script below and create an Elasticsearch node.

Resolve HTTP Error 500.19 - Internal Server Error

This blog explains a specific error encountered on a CORS enabled site hosted on an IIS server, the reason for the error and how to resolve it. 

Pre-requisites:

• CORS enabled site hosted on IIS server

Getting Started with Elastic Cloud on Kubernetes

In the modern DevOps world, ELK (Elasticsearch, Logstash, and Kibana) plays a crucial role by enabling the search, analysis, and visualization of large data sets. These capabilities are essential for troubleshooting issues, gaining insights, and ensuring the security of systems.

There are several deployment options for the ELK Stack, depending on a team's requirements and the desired level of infrastructure management. One of the deployment options is Elastic Cloud on Kubernetes (ECK), an official Kubernetes Operator provided by Elastic. Elastic Cloud on Kubernetes simplifies the deployment and management of Elastic Stack components within Kubernetes clusters.

This blog explains the process of installing Elastic Cloud on Kubernetes on an Azure Kubernetes Service (AKS) cluster using YAML templates.

Managing Planned Maintenance for Azure Container Apps

Azure Container App is a fully managed Azure service that receives regular updates automatically. These updates are applied with minimal impact on the container applications. However, Azure Container App users can now define a maintenance window, allowing them to schedule service maintenance at a convenient time. This ensures application availability during important periods without downtime due to service maintenance.

There are two types of service updates, critical and non-critical updates. Critical updates are applied immediately when required, and users cannot set a maintenance window for them. However, maintenance windows can be set for non-critical updates. This blog explains how to set a maintenance window for Azure Container App using a GitHub Action pipeline.

Conditional Variable Assignment in Azure DevOps Pipelines

Azure DevOps pipeline expression functions enable the creation of powerful Azure DevOps YAML pipelines to achieve various advanced automation needs for build and deployment purposes. This blog explains how to use the if condition with pipeline variable values.

In an Azure DevOps YAML pipeline, a variable if condition can be defined as follow

iif(condition, the_value_returns_when_condition_true, the_value_returns_when_condition_false) 

Enhance Your Azure Kubernetes Service Experience with GitHub Copilot Plugins

This blog explains how to use GitHub Copilot Chat to perform various Azure Kubernetes Service (AKS) tasks, such as creating an AKS cluster and generating kubectl commands to interact with the cluster.

Prerequisites:

• GitHub Copilot and GitHub Copilot for Azure installed
• Azure Kubernetes Service extension for Visual Studio Code installed

Docker Log Options for Better Troubleshooting

Logs are essential for troubleshooting any issues encountered during the software development and delivery process. Docker also provides several logging options to aid in the troubleshooting experience. This blog discusses various ways to review log files with Docker

Pre-requisites:

• Docker installed development machine
• Basic knowledge on Docker commands

Enabling Azure Container Storage Monitoring with Managed Prometheus

Azure Monitor Managed Service for Prometheus is a fully managed monitoring solution in Azure Monitor. It allows us to collect Prometheus metrics from Azure Kubernetes Services (AKS) clusters to monitor the workload.

This blog provides a guide on monitoring stateful workloads running on Azure Container Service within an Azure Kubernetes Service (AKS) cluster with managed Prometheus enabled.