Event Endpoint Management continues to be enhanced

Event Endpoint Management continues to be available and enhanced as part of IBM's integration platforms.

End of Marketing announcement for IBM Event Automation

Event Streams and Event Processing are deprecated and no enhancements will be provided. Support and security fixes will be provided in line with published support lifecycle policies. Future event-driven solutions that make use of Apache Kafka and Flink will be delivered by IBM Confluent Platform.

For more information, see the support lifecycle transition and software ordering completion announcement.

About MirrorMaker 2.0

MirrorMaker 2.0 is a capability within Apache Kafka that is used for copying (mirroring) topics from one Kafka cluster to another. MirrorMaker 2.0 can be used for a wide range of use cases such as data distribution, to disaster recovery, and infrastructure maintenance.

Since MirrorMaker 2.0 is a part of Apache Kafka, it is also part of Event Streams, and is fully supported.

Note: Also see geo-replication in Event Streams, which provides a simplified way of configuring MirrorMaker 2.0 for a subset of use cases.

MirrorMaker 2.0 makes use of the Kafka Connect framework, and offers dynamic configuration changes, offset synchronization, bidirectional mirroring, and improved performance.

Offset synchronization

An important feature of MirrorMaker 2.0 is the offset synchronization.

When topics are mirrored to another cluster, the event offset for a given event in the origin topic will be different to the event in the destination topic. This can occur due to records that are deleted from the origin topic due to retention policies, or because of the control records that are created when using transactions. The issue is that if a consumer moves from the origin to the destination cluster without considering these offset differences, it will duplicate, or worse, loose events.

MirrorMaker 2.0 handles these offset differences by storing a mapping between the offsets in the origin and destination clusters. It then uses this mapping to correctly populate the consumer offsets topic on the destination cluster. When consumers start up against the destination cluster, they will begin reading from the correct point in the topic’s event stream.

MirrorMaker 2.0 also synchronizes topic partitioning and access control lists, to ensure the destination cluster is as similar as possible to the origin cluster.

Reference

The following blog posts explain how to configure MirrorMaker 2.0 for its most common use cases.

Data distribution:

Disaster recovery:

Infrastructure maintenance: