Creating and Testing Ingest Pipelines with Grok for NGINX Logs in Elasticsearch

The Elastic Stack is widely used for log management, monitoring, analytics, and observability. It includes several tools such as Beats, which collect logs from various systems. However, not all data in these logs is necessary for indexing. To filter, transform, and enrich documents before they are indexed, we can use ingest pipelines. In this blog, we will discuss how to create a simple ingest pipeline and test it.

Automate PagerDuty Maintenance Windows with Azure DevOps

Reliable notifications, scheduled escalations, and effective on-call management are essential for operations teams to quickly identify and address issues. PagerDuty is a platform that provides all of these capabilities, enabling teams to maintain high reliability and reduce operational noise.When working with any operations platform, reliability is key to avoiding unnecessary alerts or actions. For example, during a scheduled system update, creating a maintenance window in PagerDuty can prevent false alarms triggered by expected changes.

In this blog, we'll discuss how to create a PagerDuty maintenance window using an Azure DevOps pipeline. By adding this step to your deployment process, you can automatically start a maintenance window during system updates, improving alert accuracy and reducing manual overhead.

Using StringList Parameters in Azure DevOps Pipelines

Azure DevOps YAML pipelines now support the stringList parameter type, which allows you to present multiple selectable options as checkboxes in the pipeline UI.

This blog explains a pipeline that uses a stringList parameter with three values: dev, qa, and prod. After adding the region parameter as a stringList, the pipeline looks similar to the image below.

Exploring the Full YAML Pipeline View in Azure DevOps

Azure DevOps YAML pipelines can be created using reusable YAML templates that help define parts of the pipeline as modules. By using templates, we can create shorter YAML pipelines and avoid duplication. However, debugging a YAML pipeline with templates is not easy, as we are unable to view the full, expanded pipeline easily. Azure DevOps now supports a new feature that allows us to see the full YAML pipeline of a pipeline run. In this blog post we are going to learn how to navigate to the full YAML pipeline view of the YAML pipeline that uses templates.

Pre-requisites:

• Azure DevOps YAML pipeline that uses templates

Resolve Azure DevOps Build Pipeline Warning, "The windows-2019 runner image is being deprecated, consider switching to windows-2022 (windows-latest) or windows-2025 instead"

The Windows-2019 runner image in Azure DevOps is being deprecated, and pipelines using the Windows-2019 hosted agent display a warning recommending a switch to a newer hosted agent. However, after changing the agent, build pipelines may break due to dependency and environment differences. This blog explains a NuGet restore failure that occurred after switching to Windows-2022 and provides a solution for it.

Warning: The windows-2019 runner image is being deprecated, consider switching to windows-2022 (windows-latest) or windows-2025 instead

Install Kibana on Azure Kubernetes Service (AKS)

Kibana is a data visualization tool used for log analysis and application monitoring use cases. It is a main component of the Elastic Stack. The installation of Elasticsearch on a Kubernetes cluster was explained in the previous blog post. In this blog, let's learn how to install Kibana on Kubernetes.

Pre-requites:

• Azure Kubernetes Cluster
• Full administrative access to the Kubernetes Cluster
• Workstation with kubectl installed
• Elastic Cloud on Kubernetes is installed

Install Elasticsearch Cluster on Azure Kubernetes Service

Elastic search is the first component of the ELK stack that gives powerful search and analytics capabilities to DevOps and development teams. We have discussed how to install Elastic Cloud for Kubernetes (ECK) in previous blog post. In this blog, let's learn how to install elastic stack on Kubernetes.

Pre-requites:

• Azure Kubernetes Cluster
• Full administrative access to the Kubernetes Cluster
• Workstation with kubectl installed
• Elastic Cloud on Kubernetes is installed

Open a shell or use the Azure Kubernetes Service 'Run Command' to run the script below and create an Elasticsearch node.

Resolve HTTP Error 500.19 - Internal Server Error

This blog explains a specific error encountered on a CORS enabled site hosted on an IIS server, the reason for the error and how to resolve it. 

Pre-requisites:

• CORS enabled site hosted on IIS server