Visualizing Kafka Clusters Using Kafka UI in Kind

In the previous blog, we discussed how to set up an Apache Kafka cluster on Kind (Kubernetes in Docker) and how to interact with it using Kafka tools. While command-line utilities are useful, managing and monitoring Kafka becomes much easier with a graphical interface. In this blog, we will explore Kafka UI, a web-based tool that provides a convenient way to view topics, inspect messages, monitor consumer groups, track offsets and lag, and manage Kafka clusters without relying solely on the command line.

Pre-requisites:

• Kafka running on Kind

1. Below is the manifest for creating Kafka UI. Run the following command to apply it.

kubectl apply -f <path to manifest>kafka-ui.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka-ui
  namespace: kafka
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka-ui
  template:
    metadata:
      labels:
        app: kafka-ui
    spec:
      containers:
        - name: kafka-ui
          image: provectuslabs/kafka-ui:latest
          ports:
            - containerPort: 8080
          env:
            - name: KAFKA_CLUSTERS_0_NAME
              value: local

            - name: KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS
              value: phkafkacluster-kafka-bootstrap:9092
---
apiVersion: v1
kind: Service
metadata:
  name: kafka-ui
  namespace: kafka
spec:
  selector:
    app: kafka-ui
  ports:
    - port: 8080
      targetPort: 8080

2. Run the following command to check if Kafka is running.

kubectl get pods -n kafka

3. Now let’s access the Kafka UI using port forwarding. Run the following command.

kubectl port-forward svc/kafka-ui -n kafka 8080:8080

4. Open a browser and go to http://localhost:8080

5. Now we should be able to access the Kafka dashboard as shown below. The dashboard displays the Kafka clusters.

6. Under the cluster, there are sections for Brokers, Topics, and Consumers