Overview

Simplify management of your Kafka topics

Event Endpoint Management simplifies the management of your Kafka topics by providing:

• Authentication of client applications.
• Quota enforcement to limit the number of events a client can publish or consume over a specified time period.
• Redaction of sensitive information.
• Schema-based event filtering and enforcement to ensure only messages that match the schema are delivered to clients or are allowed to be written to an event endpoint.
• A single enforcement point, the Event Gateway, to all the Kafka clusters that your clients use.
• A single management point, the Event Manager, where you configure the Kafka topics that are exposed to clients and the rules that apply to the topics.
• Ability to select how events from what topics, and from which Kafka clusters, are accessible to client applications.

Without Event Endpoint Management, your clients require a separate network endpoint for each Kafka cluster. If you want to implement authentication, quota enforcement, or redaction, then you must configure it individually on each Kafka cluster.

The following diagram shows the Kafka client and server interaction without Event Endpoint Management:

The following diagram shows the same Kafka clusters and clients with the Event Endpoint Management Event Gateway managing the communication between them:

An Event Endpoint Management deployment has the following components:

• One Event Manager instance. The Event Manager is where you define the Kafka clusters, topics, access controls, and other rules.
• One or more Event Gateway instances. The Event Gateways are located between the Kafka clusters and the clients, and apply the rules that you define in the Event Manager.

How Event Endpoint Management works

As part of your event-driven architecture solution, Event Endpoint Management provides the capability to describe and catalog your Kafka topics as event sources, and to grant access to application developers within the organization. Application developers can discover event endpoints and configure their applications to access them through the Event Gateway. With Event Endpoint Management, you can control access to any of your event endpoints, and also control what data can be produced to them or consumed from them.

Access to the event endpoints is managed by the Event Gateway. The Event Gateway handles the incoming requests from applications to produce (write) events to a topic or to consume from a topic’s stream of events. The Event Gateway is independent of your Kafka clusters, making access control to topics possible without requiring any changes to your Kafka cluster configuration.

The following diagram shows how Event Endpoint Management fits into the overall Event Automation architecture.

Event Endpoint Management can be deployed as a standalone installation as part of Event Automation, or it can be deployed as part of Cloud Pak For Integration. You can also integrate Event Endpoint Management with IBM API Connect by importing the AsyncAPI document that defines the event source. This integration provides the option to use events as part of your overall API management solution.

Operation flow

The following diagram shows the flow of operations in Event Endpoint Management; from a Kafka administrator adding a topic in Event Manager, to a client application accessing the published topic through the Event Gateway.

1. The Kafka administrator adds a topic to Event Endpoint Management. They can select a topic from an existing Kafka cluster or specify a new cluster. After a topic is added to Event Endpoint Management, it is known as an event source.

   The Kafka administrator creates options (with controls, if required) for an event source to define different ways of presenting the event source in the catalog.

2. The Kafka administrator publishes an option. The option is then available in the catalog for application developers to discover and use.
3. In the catalog, the application developer can browse the available entries and discover information about the kind of event data available, based on which they can decide which one to use in their applications.
4. The application developer chooses an appropriate event endpoint for their application to use. They subscribe to that event endpoint to provide their application with access to the event endpoint through the Event Gateway. The developers can manage their subscriptions through the Event Endpoint Management UI.
5. The application developer connects their application to the event endpoint, and this sets up their application with access to the events through the Event Gateway.
6. The application connects to the Event Gateway for access to the event source’s event stream.
7. The Event Gateway routes traffic securely to and from the Kafka cluster that holds the topic, providing the access to the application to interact with the event endpoint.