Exposing Kafka from kind to a Local Windows Application

In our previous blog post, we discussed a workaround for the DNS error that occurs when an application running on a local Windows machine connects to Kafka. In this post, we will discuss the proper solution.

Our Kafka cluster runs inside Docker through a local kind Kubernetes cluster rather than directly on the host machine. Kafka advertises broker addresses using Kubernetes DNS names, which can be resolved from inside the cluster but not by an application running on Windows. To make Kafka reachable from the host, we must expose the required ports from the kind node container to the host machine.

Troubleshooting Kafka DNS Resolution from a Local .NET Application

This blog explains a DNS resolution error that can occur when testing an event-driven .NET application locally with Kafka running in a kind cluster. You’ll learn what causes the issue and how to apply a practical workaround to get your local development environment working quickly.

Pre-requisites:

• A Kafka cluster running on a local Kind cluster.

• Strimzi configured to manage the Kafka cluster.

• A .NET API capable of creating or publishing to Kafka.

• kubectl configured to access your Kind cluster.

• PowerShell running on Windows.

• The BootstrapServers value is intentionally set to, localhost:9092 in appsettings.json

Compare Commits Easily in Azure DevOps with the New UI Feature

Troubleshooting software bugs often involves comparing commits to identify the changes introduced between two points in the project's history. This is one of the most common techniques used by developers to investigate regressions, validate fixes, and understand the impact of code changes.

Azure DevOps now provides a built in UI to compare commits, eliminating the need to manually construct comparison URLs. In this blog post, we'll explore how to use this feature to compare commits and review the differences between them.

Prerequisites

• An Azure DevOps repository
• Multiple branches or commits available for comparison

Easily Navigate Large Azure DevOps Pipelines

When an Azure DevOps pipeline contains hundreds of stages, navigating the pipeline UI can become challenging for several reasons.

• * Finding a specific stage among hundreds can be difficult.
• * Extensive horizontal and vertical scrolling is often required.
• * Failed stages are harder to identify in a large pipeline.
• * Navigation more difficult for users.

To address these challenges, Azure DevOps has introduced a new feature that makes navigating large pipelines much easier.

Prerequisites:

An Azure DevOps pipeline with multiple stages.

Managing Kubernetes Clusters Efficiently with K9s Terminal UI

When working with multiple Kubernetes clusters, administrators and developers often use kubectl commands and kube config contexts to switch between clusters. While this is a powerful approach, it requires users to remember various commands and context names, which can become difficult as the number of clusters grows.

For those who prefer a more interactive way to manage Kubernetes environments, tools with user-friendly interfaces can simplify day to day operations. In this blog, we will learn about K9s, a terminal based user interface that provides an efficient way to interact with Kubernetes clusters. K9s enables users to navigate resources, monitor workloads, view logs, and manage clusters directly from the terminal.

Prerequisites: This blog is demonstrated on Windows, and Chocolatey should be installed on your machine.

Visualizing Kafka Clusters Using Kafka UI in Kind

In the previous blog, we discussed how to set up an Apache Kafka cluster on Kind (Kubernetes in Docker) and how to interact with it using Kafka tools. While command-line utilities are useful, managing and monitoring Kafka becomes much easier with a graphical interface. In this blog, we will explore Kafka UI, a web-based tool that provides a convenient way to view topics, inspect messages, monitor consumer groups, track offsets and lag, and manage Kafka clusters without relying solely on the command line.

Pre-requisites:

• Kafka running on Kind

Getting Started with Kafka on Kind Cluster

Apache Kafka is one of the most widely used event streaming platforms for building real time data pipelines and streaming applications. Before implementing Kafka in production environments, it is important to understand its core concepts, architecture, and operational model.

A practical way to learn Kafka is by running it locally in a lightweight Kubernetes environment. Kind (Kubernetes in Docker) provides a lightweight Kubernetes environment that can run entirely on a local machine. It is easy to set up and consumes minimal resources.

In this blog, we will be learning how to deploy and run Apache Kafka on a Kind cluster, allowing you to experiment with Kafka concepts and gain hands-on experience in a local development environment.

Prerequisites: Kind installed locally and a single-node Kind cluster.

Troubleshooting Kind Cluster Creation Issues

Kind (Kubernetes in Docker) is a tool used to run a lightweight Kubernetes cluster locally inside Docker containers.

It is a great platform for learning real Kubernetes behavior because clusters can be spun up easily using a single command. In this blog, we will discuss an error that occurred while creating a Kind cluster for the first time and how to resolve it.

Pre-requisites:

• Docker Desktop
• Kubectl installed
• Kind installed