Event Endpoint Management continues to be enhanced

Event Endpoint Management continues to be available and enhanced as part of IBM's integration platforms.

End of Marketing announcement for IBM Event Automation

Event Streams and Event Processing are deprecated and no enhancements will be provided. Support and security fixes will be provided in line with published support lifecycle policies. Future event-driven solutions that make use of Apache Kafka and Flink will be delivered by IBM Confluent Platform.

For more information, see the support lifecycle transition and software ordering completion announcement.

What’s new

IBM’s conditions of support for Apache Flink in IBM Event Automation has been expanded. For more information, see the support policy statement.

Find out what is new in Event Processing version 1.5.x.

Release 1.5.4

Event source node: Select topic from Event Endpoint Management

In Event Processing 1.5.4, you can discover and select topics directly from Event Endpoint Management 11.8.0 and later, when you configure event source nodes. You can now connect more easily to virtual topics that are published and managed in Event Endpoint Management.

To use this feature, configure the Event Endpoint Management connection details in your Event Processing instance, then follow the steps to configure your event source node.

UI enhancements: Enhanced landing page with flow management

Event Processing 1.5.4 introduces an enhanced landing page with improved flow management capabilities:

  • Flow overview section: displays total number of flows and number of flows in each status.
  • Flow filtering: filter flows by status and running time to locate long-running flows.
  • Flow metrics: expand individual flows to view detailed metrics such as running time, restart count, consumer lag, checkpoint duration, and checkpoint count.

For more information, see managing flows.

Run flow from time offset

Event Processing 1.5.4 introduces the ability to start processing events from a time offset when running flows. You can replay events from a relative point in time, providing more flexibility in testing and debugging flows.

You can use the time offset when running flows to:

  • Test flows with historical data without processing all events from the beginning.
  • Debug issues that occurred at a specific time.
  • Replay events from a known point in time for analysis.

For more information, see running a flow.

Event activity metrics: Latest Flink watermark

Event Processing 1.5.4 introduces an additional metric for event source and event destination nodes in the Flow metrics > Event activity tab.

You can now monitor the most recent watermark for all the event source and event destination nodes in your flow. These metrics provide insights into the temporal behavior of your flows and can be used to identify timing-related issues.

For more information, see monitoring event activity.

IBM Operator for Apache Flink version 1.5.4 update includes Apache Flink version 1.20.4.

Flink 1.20 is supported only for deploying Flink jobs outside the Event Processing UI. The Event Processing UI is not supported when running a Flink instance with Flink 1.20.

Support for Kubernetes 1.34 and 1.35

Event Processing version 1.5.4 introduces support for Kubernetes platform versions 1.34 and 1.35 that support the Red Hat Universal Base Image (UBI) containers.

Documentation: Highlighting differences between versions

Any difference in features or behavior introduced by Event Processing 1.5.4 compared to 1.5.3 or earlier is highlighted in this documentation by using the following graphic: Event Processing 1.5.4 icon

Security and bug fixes

Event Processing release 1.5.4 contains security and bug fixes.

Release 1.5.3

Combine node: enhancements for mapping properties

Event Processing 1.5.3 introduces usability enhancements for mapping properties in the combine node. You can now search properties in the reference input stream, clear all mappings, expand or collapse property groups, and undo or redo changes when working with complex schemas that contain many properties or nested structures.

For more information, see combine node.

Monitoring event activity in running flows

Event Processing 1.5.3 introduces event activity metrics in the Flow metrics tab. You can now monitor detailed activity for event source and event destination nodes, including:

  • Number of events sent or received by event source and event destination nodes
  • Bytes read from event source nodes and bytes written to event destination nodes
  • Message lag for event source nodes

These metrics provide deeper insights into the data flow through your event processing pipelines. You can use these insights to identify bottlenecks and optimize performance.

For more information, see monitoring event activity.

Session cluster monitoring

Event Processing 1.5.3 introduces the Session cluster page for monitoring the health and capacity of the Flink session cluster that supports your authoring environment. This page provides real-time visibility into TaskManager resource utilization and cluster capacity:

  • TaskManager overview: view the number of free processing slots and total TaskManagers in your session cluster.
  • Resource metrics: monitor CPU load and memory usage for each TaskManager with visual indicators.
  • Slot allocation: see how processing slots are distributed and utilized across TaskManagers.
  • Log downloads: download logs from individual TaskManagers, all TaskManagers, JobManager, or all logs combined using the dropdown menu.

For more information, see monitoring session cluster.

IBM Operator for Apache Flink version 1.5.3 update includes Apache Flink version 2.2.1.

Flink 1.20.3 is also supported, but only for deploying Flink jobs outside the Event Processing UI. The Event Processing UI is not supported when running a Flink instance with Flink 1.20.3.

Documentation: Highlighting differences between versions

Any difference in features or behavior introduced by Event Processing 1.5.3 compared to 1.5.2 or earlier is highlighted in this documentation by using the following graphic: Event Processing 1.5.3 icon

Security and bug fixes

Event Processing release 1.5.3 contains security and bug fixes.

Release 1.5.2

User-defined functions (UDFs) in SQL processor nodes

Event Processing 1.5.2 introduces support for user-defined functions (UDFs) in SQL processor nodes. You can use UDFs to extend the built-in functionality of Flink by writing custom logic in Java.

All UDFs available in Flink are supported. For more information, see configuring UDFs.

Processor node: merge multiple input streams with the combine node

Event Processing version 1.5.2 introduces the Combine node to merge multiple event sources into a single unified event stream by mapping properties to a reference event source schema.

Unlike joins, the combine node does not correlate events across streams. Instead, it merges events from multiple sources with the same schema into a single stream, allowing downstream nodes to process them consistently.

For more information, see combine node.

Metrics for monitoring running flows

Event Processing 1.5.2 introduces the Flow metrics tab where you can monitor the health and performance of running flows. The tab is available next to the Output events tab, and provides real-time insights into the current state of your flow including uptime, restarts, and checkpoint metrics.

For more information, see monitoring flow metrics.

Documentation: Highlighting differences between versions

Any difference in features or behavior introduced by Event Processing 1.5.2 compared to 1.5.1 or earlier is highlighted in this documentation by using the following graphic: Event Processing 1.5.2 icon

Security and bug fixes

Event Processing release 1.5.2 contains security and bug fixes.

Release 1.5.1

Additional outputs for conditional branching

Event Processing version 1.5.1 introduces additional output support for the Filter and SQL Processor nodes, enabling conditional branching with multiple output paths.

  • Filter node: Configure an additional output to capture events that do not match the filter expression, enabling the use of both matched and unmatched events in downstream nodes for separate processing.
  • SQL Processor node: Define multiple VIEW statements to create side outputs that split the processing logic and send different results to separate downstream nodes.

This feature provides greater flexibility in designing event processing flows with conditional logic and multiple processing paths.

Automatic merge of public certificates to the custom truststore

In Event Processing 1.5.1 and later, when you provide a custom truststore containing self-signed or internal CA certificates, public CA certificates are automatically added from the Java truststore to your custom truststore. This means that public CA certificates are maintained through regular Java version updates, and you are not required to manually add and update them. The truststore merge feature is enabled by default.

For more information, see configuring SSL.

Annotate on canvas with comments

Event Processing 1.5.1 introduces the commenting feature on the flow canvas. With comments, you can document your work directly on the canvas and collaborate with team members.

For more information, see adding comments to your flow.

Documentation: Highlighting differences between versions

Any difference in features or behavior introduced by Event Processing 1.5.1 compared to 1.5.0 or earlier is highlighted in this documentation by using the following graphic: Event Processing 1.5.1 icon

Security and bug fixes

Event Processing release 1.5.1 contains security and bug fixes.

Release 1.5.0

Event Processing 1.5.0 and later supports Apache Flink 2.2.0 for both UI-based flows and for Flink jobs deployed outside the Event Processing UI.

Flink 1.20.3 is also supported, but only for deploying Flink jobs outside the Event Processing UI. This includes:

The Event Processing UI is not supported when running a Flink instance with Flink 1.20.3.

To use Flink 1.20.3 for deploying Flink jobs, you must configure the FlinkDeployment custom resource to prevent automatic version upgrades. For more information, see upgrading.

Support for Red Hat OpenShift Container Platform 4.21

Event Processing version 1.5.0 introduces support for Red Hat OpenShift Container Platform 4.21.

Removal of support for OpenShift Container Platform 4.15

Event Processing 1.5.0 and later do not support OpenShift Container Platform version 4.15. For more information about supported versions, see the support matrix.

Removal of support for Kubernetes 1.25 and 1.26

Event Processing 1.5.0 and later do not support Kubernetes versions 1.25 and 1.26. For more information about supported versions, see the support matrix.

Security and bug fixes

Event Processing release 1.5.0 and IBM Operator for Apache Flink version 1.5.0 contain security and bug fixes.