You can monitor the consumer lag for Kafka clients connecting to IBM Event Streams. This information is available through both the Event Streams UI and CLI.
Each partition will have a consumer within a consumer group with information relating to its consumption as follows:
- Client ID and Consumer ID: each partition will have exactly one consumer in the consumer group, identifiable by a Client ID and Consumer ID.
- Current offset: the last committed offset of the consumer.
- Log end offset: the highest offset of the partition.
- Offset lag: the difference between the current consumer offset and the highest offset, which shows how far behind the consumer is.
Using the Event Streams UI
To access the consumer groups side panel in the Event Streams UI, do the following:
- Log in to the Event Streams UI from a supported web browser (see how to determine the login URL for your Event Streams UI).
- Click Topics in the primary navigation.
- Locate your topic using the Name column and click the row for the topic.
- Click the Consumer groups tab.
- The Consumer groups dashboard will display all consumer groups for the selected topic. Click the consumer group of interest from the Consumer group ID column. The consumer group side panel should now be displayed on screen.
This side panel will display a table containing consumer group information for each partition of the topic.
Using the Event Streams CLI
To access information about a consumer group in the Event Streams CLI, do the following:
- Log in to your cluster as an administrator by using the IBM Cloud Pak CLI:
cloudctl login -a https://<cluster_address>:<cluster_router_https_port>
- Run the following command to initialize the Event Streams CLI on the cluster:
cloudctl es init
- To list all consumer groups on a cluster, run:
cloudctl es groups
- To list information about a consumer group, run:
cloudctl es group --group <consumer-group-id>
<consumer-group-id>is the name of the consumer group of interest.
The CLI will print a table containing consumer group information for each partition of the topic.
The following example shows the information the command returns for a consumer group called
$ cloudctl es group --group my-consumer > Details for consumer group my-consumer Group ID State my-consumer Stable Topic Partition Current offset End offset Offset lag Client Consumer Host my-topic 2 999 1001 2 some-client some-consumer some-host my-topic 0 1000 1001 1 some-client some-consumer some-host my-topic 1 1000 1000 0 some-client some-consumer some-host OK
To view other Kafka-related metrics, consider configuring a Kafka Exporter.
For information on how to monitor the general health of a particular topic, see monitoring topic health.