Read about the key concepts and terms used in Event Endpoint Management.
Topic
A Kafka topic, which contains a set of related events.
Event source
An event source is the term used to describe a Kafka topic when it has been added to Event Endpoint Management. A Kafka administrator can make event sources accessible by publishing an option to the catalog where it becomes an event endpoint. Application developers can discover and browse the available event endpoints and decide which ones they want access to. They can then configure their applications to either consume from the event endpoint or produce to it.
Option
A Kafka administrator can create an option for an event source that enables them to control how the topic’s stream of events are made available within the catalog. When an option is published to the catalog, it is called an event endpoint.
Controls
You can add controls to options so that you have greater management over how topic data is presented to applications.
You can add the following controls to an option on a consume-enabled event source:
- Approval: Use the approval control to force users to request access to your topic.
Redaction: Use a redaction control to hide sensitive data.- Schema filtering: Use the schema filtering control to manage how the data in your topic is presented to consumers and to ensure consistency.
You can add the following controls to an option on a produce-enabled event source:
- Approval: Use the approval control to force users to request access to your topic.
Schema enforcement: Use the schema enforcement control to enforce the shape of data that is allowed to be produced to this topic.
Event Endpoint
When a Kafka administrator publishes an option to the Event Endpoint Management catalog, the option exposes a topic as an event endpoint. This contains all the descriptive elements provided when describing a topic as an event source, and the management configuration specified in the published option. If other options have already been published relating to an event source, they are available under the event endpoint in the catalog.
For an application developer, the event endpoint is the object that they can discover in the catalog, and behaves like a Kafka topic when it is accessed through the Event Gateway.
Catalog
The catalog lists all available topics that represent event sources and, if applicable, their available options. Application developers in your organization can use the catalog to browse the available topics, and to view more information about each of them, including a description, tags, sample messages, schema details if used, and so on, enabling access to the stream of events represented by the topics using the published options.
In Event Endpoint Management 11.1.1 and later, Kafka administrators can use the catalog to check what options are published and made available to others in the organization.
Cluster
Kafka runs as a cluster of one or more servers (Kafka brokers). The load is balanced across the cluster by distributing it amongst the servers.
Subscription
Application developers configure their applications to subscribe to the stream of events, providing access to the message content from the event stream, and generating the required credentials for their application to consume from the topic.
In Event Endpoint Management 11.1.1 and later, Kafka administrators can limit subscriptions by adding the approval control to the options of a topic and they can manage subscriptions that are created for their topics in the Event Endpoint Management Topic Detail page.
Consume
An application developer can discover event endpoints that are published to the catalog. The application developer must subscribe to an option on a consume-enabled event source to get access for a client application to consume Kafka events from the related topic through the Event Gateway, by using standard Kafka client libraries.
Produce
An application developer can write events to an event endpoint that is published to the catalog. The application developer must subscribe to an option on a produce-enabled event source to get access for a client application to produce Kafka events to the related topic through the Event Gateway, by using standard Kafka client libraries.
Event Gateway
Access to the event sources are managed by the Event Gateway. The Event Gateway handles the incoming requests from applications to access a topic, routing traffic securely between the Kafka cluster and the application.
The Event Gateway is independent of your Kafka clusters, making access to topics possible without requiring any changes to your Kafka cluster configuration.
Gateway group
A gateway group is a group of gateway instances that can be logically grouped and scaled together. When deploying a gateway, you can define gateway groups by name, and any gateway instances with the same group name are considered a member of that group.
The following are examples of gateway groupings:
- Gateways that are deployed to manage Kafka traffic to designated Kafka clusters.
- Gateways that are colocated within the same geography.
- Gateways that represent different Kafka topics and streams at different stages of adoptable maturity, such as development, test, or production.
When gateways start, they join their defined gateway group and pull any configuration that is specified for that group. The configuration for a gateway group is determined when publishing an event source, where you can select which gateway group the event source must be accessible through. By publishing an event source to a gateway group, you can restrict which gateways have access to an event source without specifying the gateway instances directly. Gateway instances can be added or removed from a gateway group at any time, providing flexibility as the use of event sources matures.
Note: A gateway instance can only be a member of one gateway group.
Important: When removing a gateway deployment, work with subscribed users to ensure their clients are configured to work with another member of the same gateway group, so they can continue to consume your event source.
Message
The unit of data in Kafka. Each message is represented as a record, which comprises two parts: key and value. The key is commonly used for data about the message and the value is the body of the message. Message is also sometimes referred to as event data and record.
To learn more about key concepts, see the Apache Kafka documentation.