Event Endpoint Management continues to be enhanced

Event Endpoint Management continues to be available and enhanced as part of IBM's integration platforms.

End of Marketing announcement for IBM Event Automation

Event Streams and Event Processing are deprecated and no enhancements will be provided. Support and security fixes will be provided in line with published support lifecycle policies. Future event-driven solutions that make use of Apache Kafka and Apache Flink will be delivered by IBM Confluent.

For more information, see how you can plan your transition.

Before you begin

Complete the following before you start the migration. Review your Event Streams instance, clean up unneeded resources, and prepare client application configurations for switchover.

Prerequisites checklist

Complete the following prerequisites before you set up Confluent Platform:

Matching the Confluent Platform version

Event Streams 13.0 includes Apache Kafka 4.2.0. Deploy Confluent Platform 8.x or later by using Confluent for Kubernetes (CFK) 3.x.

Matching the target Confluent Platform Kafka version to your Event Streams Kafka version ensures that the migration changes only the platform, not the underlying Kafka protocol or broker behaviour.

For the full list of supported environment and component version combinations, see the Confluent documentation.

Note: Plan any Kafka version upgrades as a separate activity after the migration is complete.

Recording broker and topic configurations

Use the configuration details recorded during the planning phase to document all broker defaults and per-topic settings. Apply these values to Confluent Platform during setting up Confluent Platform and verify them during topic configuration migration.

Removing unused resources

Delete any topics and schemas that are no longer in use. Removing unused resources before the migration reduces replication time and simplifies post-migration validation.

Preparing client applications

Review and update client application configurations in advance to minimize changes during switchover. Depending on your deployment, this preparation might include updating serialization libraries, REST API endpoints, security protocols, and authentication credentials.

Schema Registry compatibility

Verify that your client applications use Schema Registry libraries and wire formats that are compatible with Confluent Schema Registry:

  • Confirm that each Avro client application uses a Confluent-compatible Schema Registry client, specifically Confluent serializers and deserializers pointed at the Apicurio /apis/ccompat/v7 endpoint.
  • Confirm that your Avro clients use payload mode, where the schema ID is embedded in the message body. If your clients use headers mode, where the schema ID is stored in a message header, they cannot be read by Confluent serializers without reproducing the messages. For more information, see component mapping.

Kafka Bridge and REST producer applications

Update applications that produce messages through the Event Streams Kafka Bridge or REST producer to use the Confluent REST Proxy API before you migrate.

The Confluent REST Proxy supports two API versions. The v2 API uses topic-name-based paths such as POST /topics/{topic} and is the closest equivalent to the Event Streams REST producer. The v3 API is the latest version and requires a cluster ID lookup before producing messages, by using paths such as POST /v3/clusters/{cluster_id}/topics/{topic}/records. For more information, see migrating Kafka Bridge and REST producer.

Migrating from SCRAM-SHA-512 authentication

Confluent for Kubernetes (CFK) does not natively support SCRAM-SHA-512. SCRAM-SHA-512 cannot be configured as a listener authentication.type in the Kafka custom resource. Additionally, SCRAM credentials are neither created nor managed by the operator. You must create and manage them manually in Kafka.

To continue authenticating to Kafka securely by using a username and password, use SASL/PLAIN authentication mechanism combined with TLS encryption in CFK.

To switch your external client applications from SCRAM-SHA-512 to SASL/PLAIN authentication, make the following changes in your Kafka configurations:

  • Update sasl.mechanism from SCRAM-SHA-512 to PLAIN.
  • Change the login module in sasl.jaas.config to org.apache.kafka.common.security.plain.PlainLoginModule from org.apache.kafka.common.security.scram.ScramLoginModule.

The security.protocol setting remains SASL_SSL.

Prepare these configuration changes in advance so that you can apply them to all the applications when you switch over.

You can continue to use the existing credentials for authentication by migrating the SCRAM credentials from Event Streams to CFK and storing them as SASL/PLAIN credentials. For more information, see migrating SCRAM-SHA-512 credentials to SASL/PLAIN.

Preparing Kafka Connect and connectors

If your Event Streams deployment uses Kafka Connect, build the CFK Connect plugin image with your connector plugin JARs before you start the migration. The image must be available before the CFK Connect cluster can be deployed. For step-by-step instructions, see building the plugin image.

Planning TLS certificate management

If you enable TLS for broker and inter-broker communication, Confluent Platform requires TLS certificates. Decide how you will issue and manage these certificates before you set up Kafka, because the approach affects how you configure Confluent for Kubernetes (CFK).

Unless you use CFK built-in auto-generated TLS certificates, you are responsible for managing the full certificate lifecycle, including CA creation, certificate issuance, rotation, and distribution to client applications. You can fulfill this responsibility with any tooling your organization already has in place, such as an internal PKI, an external certificate authority, or a certificate management tool such as cert-manager. On Red Hat OpenShift, the Red Hat cert-manager Operator automates issuance and renewal. For the fully automated option with no external PKI required, see auto-generated TLS certificates.

For step-by-step guidance on setting up CAs and secrets for CFK, see setting up certificates.

Next steps

After completing the preparation tasks, proceed to setting up certificates.