Upgrading

Upgrade your Event Streams installation as follows. The Event Streams operator handles the upgrade of your Event Streams instance.

Before you begin, review the upgrade procedure and determine the appropriate steps for your deployment based on your platform, current version, and target version.

Upgrade paths

You can upgrade Event Streams to the latest 12.3.x version directly from any 12.2.x version by using the latest 12.3.x operator. The upgrade procedure depends on whether you are upgrading to a major, minor, or patch level version, and what your catalog source is.

If you are upgrading from Event Streams version 12.1.x or earlier, you must first upgrade your installation to 12.2.x and then follow these instructions to upgrade from 12.2.x to 12.3.x.

• On OpenShift, you can upgrade to the latest version by using operator channel v12.3. Review the general upgrade prerequisites before following the instructions to upgrade on OpenShift.

  Note: If your operator upgrades are set to automatic, patch level upgrades are completed automatically. This means that the Event Streams operator is upgraded to the latest 12.3.x version when it is available in the catalog, and your Event Streams instance is then also automatically upgraded, unless you set a schedule for the upgrade by pausing the reconciliation.

• On other Kubernetes platforms, you must update the Helm repository for any level version update (any digit update: major, minor, or patch), and then upgrade by using the Helm chart. Review the general upgrade prerequisites before following the instructions to upgrade on other Kubernetes platforms.

Planning your upgrade

Deprecated fields in v1beta2 API

The v1beta2 API is deprecated in Event Streams 12.3.0 and later, and will be removed in a future release. Several fields within the v1beta2 API are also deprecated.

Note: Your existing Event Streams instances will continue to work with the v1beta2 API in 12.3.x versions without requiring any configuration changes.

The following table lists the deprecated fields:

Prerequisites

• The images for Event Streams release 12.3.x are available in the IBM Cloud Container Registry. Ensure you redirect your catalog source to use icr.io/cpopen as described in Implementing ImageContentSourcePolicy to redirect to the IBM Container Registry.

• Ensure that you have installed a supported container platform and system. For supported container platform versions and systems, see the support matrix.

• To upgrade successfully, your Event Streams instance must include a node pool with the controller role and persistent storage enabled. A minimum of three controller nodes is recommended to maintain quorum. If your instance is not configured with a controller node pool that uses persistent storage, the upgrade will fail, and data loss might occur.

  For example:

   apiVersion: eventstreams.ibm.com/v1beta2
   kind: EventStreams
   metadata:
     name: example-pre-upgrade
     namespace: myproject
   spec:
     # ...
     strimziOverrides:
     #...
      kafka:
      #...
      nodePools:
      - name: kafka
         replicas: 3
         storage:
            type: persistent-claim
            #...
         roles:
            - broker
      - name: controller
         replicas: 3
         storage:
            type: persistent-claim
            #...
         roles:
            - controller
  

• If you installed the Event Streams operator to manage instances of Event Streams in any namespace (one per namespace), then you might need to control when each of these instances is upgraded to the latest version. You can control the updates by pausing the reconciliation of the instance configuration as described in the following sections.

• If you are running Event Streams as part of IBM Cloud Pak for Integration, ensure you meet the following requirements:

  • Follow the upgrade steps for IBM Cloud Pak for Integration before upgrading Event Streams.
  • If you are planning to configure Event Streams with Keycloak, ensure you have the IBM Cloud Pak for Integration 2023.4.1 (operator version 7.2.0) or later installed, including the required dependencies.

• Ensure all applications connecting to your instance of Event Streams that use the schema registry are using Apicurio client libraries version 2.5.0 or later.

  Important: Support for Apicurio Registry Core REST API version 2 is deprecated and will be removed in a future release. To ensure continued compatibility, update all Apicurio client libraries to use API version 3. For more information, see prerequisites.

Note: There is no downtime during the Event Streams upgrade. The Kafka pods are rolled one at a time, so a Kafka instance will always be present to serve traffic. However, if the number of brokers you have matches the min.insync.replicas value set for any of your topics, then that topic will be unavailable to write to while the Kafka pods are rolling.

Scheduling the upgrade of an instance

In 11.1.x and later, the Event Streams operator handles the upgrade of your Event Streams instance automatically after the operator is upgraded. No additional step is required to change the instance (product) version.

If your operator manages more than one instance of Event Streams, you can control when each instance is upgraded by pausing the reconciliation of the configuration settings for each instance, running the upgrade, and then unpausing the reconciliation when ready to proceed with the upgrade for a selected instance.

Pausing reconciliation by using the CLI

1. Log in to your Kubernetes cluster as a cluster administrator by setting your kubectl context.

2. To apply the annotation first to the EventStreams and then to the Kafka custom resource, run the following command, where <type> is either EventStreams or Kafka:

   kubectl annotate <type> <instance-name> -n <instance-namespace> eventstreams.ibm.com/pause-reconciliation='true'
   

3. Follow the steps to upgrade on OpenShift.

Unpausing reconciliation by using the CLI

To unpause the reconciliation and continue with the upgrade of an Event Streams instance, run the following command to first remove the annotations from the Kafka custom resource, and then from the EventStreams custom resource, where <type> is either Kafka or EventStreams:

kubectl annotate <type> <instance-name> -n <instance-namespace> eventstreams.ibm.com/pause-reconciliation-

When the annotations are removed, the configuration of your instance is updated, and the upgrade to the latest version of Event Streams completes.

Pausing reconciliation by using the OpenShift web console

1. Log in to the OpenShift Container Platform web console using your login credentials.

2. Expand Operators in the navigation on the left, and click Installed Operators.

   

3. From the Project list, select the namespace (project) the instance is installed in.
4. Locate the operator that manages your Event Streams instance in the namespace. It is called Event Streams in the Name column. Click the Event Streams link in the row.
5. Select the instance you want to pause and click the YAML tab.

6. In the YAML for the custom resource, add eventstreams.ibm.com/pause-reconciliation: 'true' to the metadata.annotations field as follows:

   apiVersion: eventstreams.ibm.com/v1beta2
   kind: EventStreams
   metadata:
     name: <instance-name>
     namespace: <instance-namespace>
     annotations:
       eventstreams.ibm.com/pause-reconciliation: 'true'
   spec:
     # ...
   

7. This annotation also needs to be applied to the corresponding Kafka custom resource. Expand Home in the navigation on the left, click API Explorer, and type Kafka in the Filter by kind... field. Select Kafka.
8. From the Project list, select the namespace (project) the instance is installed in and click the Instances tab.
9. Select the instance with the name <instance-name> (the same as the Event Streams instance).

10. In the YAML for the custom resource, add eventstreams.ibm.com/pause-reconciliation: 'true' to the metadata.annotations field as follows:

    apiVersion: eventstreams.ibm.com/v1beta2
    kind: Kafka
    metadata:
    name: <instance-name>
    namespace: <instance-namespace>
    annotations:
       eventstreams.ibm.com/pause-reconciliation: 'true'
    

11. Follow the steps to upgrade on OpenShift.

Unpausing reconciliation by using the OpenShift web console

To unpause the reconciliation and continue with the upgrade of an Event Streams instance, first remove the annotations from the Kafka custom resource, and then from the EventStreams custom resource. When the annotations are removed, the configuration of your instance is updated, and the upgrade to the latest version of Event Streams completes.

Upgrading on the OpenShift Container Platform

Upgrade your Event Streams instance running on the OpenShift Container Platform by using the CLI or web console as follows.

Pre-upgrade checks and preparation

Complete the following steps to plan your upgrade on OpenShift.

• If your Kafka cluster uses custom listener certificates with SSL private keys, ensure that they are in PKCS#8 format (BEGIN PRIVATE KEY) before upgrading. Private keys in PKCS#1 format (BEGIN RSA PRIVATE KEY) are not compatible with Event Streams 12.3.0 and later.

  Run the following command to convert your private key to PKCS#8 format:

  openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in PKCS1.key -out PKCS8.key
  

  For more information, see troubleshooting.

• Determine which Operator Lifecycle Manager (OLM) channel is used by your existing Subscription. You can check the channel you are subscribed to in the web console (see Update channel section), or by using the CLI as follows (this is the subscription created during installation):

  1. Run the following command to check your subscription details:

     oc get subscription
     

  2. Check the CHANNEL column for the channel you are subscribed to, for example, v12.2 in the following snippet:

     NAME                        PACKAGE                     SOURCE                      CHANNEL
     ibm-eventstreams            ibm-eventstreams            ibm-eventstreams-catalog    v12.2
     

• If your existing Subscription does not use the v12.3 channel, your upgrade is a change in a major version. Complete the following steps to upgrade:
  1. Ensure the catalog source for new version is available.
  2. Change your Subscription to the v12.3 channel by using the CLI or the web console. The channel change will upgrade your operator, and then the operator will upgrade your Event Streams instance automatically.
• If your existing Subscription is already on the v12.3 channel, your upgrade is a change to the patch level (third digit) only. Make the catalog source for your new version available to upgrade to the latest level. If you installed by using the IBM Operator Catalog with the latest label, new versions are automatically available. The operator will upgrade your Event Streams instance automatically.

Making new catalog source available

Before you can upgrade to the latest version, the catalog source for the new version must be available on your cluster. Whether you have to take action depends on how you set up the catalog sources for your deployment.

• Latest versions: If your catalog source is the IBM Operator Catalog, latest versions are always available when published, and you do not have to make new catalog sources available.

• Specific versions: If you used the CASE bundle to install catalog source for a specific previous version, you must download and use a new CASE bundle for the version you want to upgrade to.
  • If you previously used the CASE bundle for an online install, apply the new catalog source to update the CatalogSource to the new version.
  • If you used the CASE bundle for an offline install that uses a private registry, follow the instructions in installing offline to remirror images and update the CatalogSource for the new version.
• In both cases, wait for the status.installedCSV field in the Subscription to update. It eventually reflects the latest version available in the new CatalogSource image for the currently selected channel in the Subscription:
  • In the OpenShift Container Platform web console, the current version of the operator is displayed under Installed Operators.
  • If you are using the CLI, check the status of the Subscription custom resource, the status.installedCSV field shows the current operator version.

Upgrading Subscription by using the CLI

If you are using the OpenShift command-line interface (CLI), the oc command, complete the steps in the following sections to upgrade your Event Streams installation.

1. Log in to your Red Hat OpenShift Container Platform as a cluster administrator by using the oc CLI (oc login).

2. Ensure the required Event Streams Operator Upgrade Channel is available:

   oc get packagemanifest ibm-eventstreams -o=jsonpath='{.status.channels[*].name}'
   

3. Change the subscription to move to the required update channel, where vX.Y is the required update channel (for example, v12.3):

   oc patch subscription -n <namespace> ibm-eventstreams --patch '{"spec":{"channel":"vX.Y"}}' --type=merge
   

Wait for the operator to reconcile the Event Streams instance and roll all the pods.

Note: During this reconciliation process, the Kafka controller pod might temporarily enter a CrashLoopBackOff state with the following error. This happens because the new operator has updated the ConfigMap, but the pod is still using the previous image. This is expected and resolves automatically within a few minutes after the new image is updated by the operator. No manual intervention is required, and the Kafka cluster remains operational throughout this period.

cat: /opt/kafka/custom-config/metadata.state: No such file or directory

Upgrading Subscription by using the web console

If you are using the web console, complete the steps in the following sections to upgrade your Event Streams installation.

1. Log in to the OpenShift Container Platform web console using your login credentials.

2. Expand Operators in the navigation on the left, and click Installed Operators.

   

3. From the Project list, select the namespace (project) the instance is installed in.
4. Locate the operator that manages your Event Streams instance in the namespace. It is called Event Streams in the Name column. Click the Event Streams link in the row.
5. Click the Subscription tab to display the Subscription details for the Event Streams operator.
6. Click the version number link in the Update channel section (for example, v12.2). The Change Subscription update channel dialog is displayed, showing the channels that are available to upgrade to.
7. Select v12.3 and click the Save button on the Change Subscription Update Channel dialog.

Wait for the operator to reconcile the Event Streams instance and roll all the pods.

Note: During this reconciliation process, the Kafka controller pod might temporarily enter a CrashLoopBackOff state with the following error. This happens because the new operator has updated the ConfigMap, but the pod is still using the previous image. This is expected and resolves automatically within a few minutes after the new image is updated by the operator. No manual intervention is required, and the Kafka cluster remains operational throughout this period.

cat: /opt/kafka/custom-config/metadata.state: No such file or directory

Note: The number of containers in each Kafka broker will reduce from 2 to 1 as the TLS-sidecar container will be removed from each broker during the upgrade process.

Upgrading on other Kubernetes platforms by using Helm

If you are running Event Streams on Kubernetes platforms that support the Red Hat Universal Base Images (UBI) containers, you can upgrade Event Streams by using the Helm chart.

Pre-upgrade checks and preparation on other Kubernetes platforms

Complete the following steps to plan your upgrade on other Kubernetes platforms.

• Determine the chart version for your existing deployment:

  1. Change to the namespace where your Event Streams instance is installed:

     kubectl config set-context --current --namespace=<namespace>
     

  2. Run the following command to check what version is installed:

     helm list
     

  3. Check the version installed in the CHART column, for example, <chart-name>-12.2.2 in the following snippet:

     NAME                      NAMESPACE  REVISION  UPDATED                                 STATUS   CHART                            APP VERSION
     ibm-eventstreams          es         1         2026-03-20 11:49:27.221411789 +0000 UTC deployed ibm-eventstreams-operator-12.2.2  12.2.2
     

• Check the latest chart version that you can upgrade to:

  1. Log in to your Kubernetes cluster as a cluster administrator by setting your kubectl context.

  2. Add the IBM Helm repository:

     helm repo add ibm-helm https://raw.githubusercontent.com/IBM/charts/master/repo/ibm-helm
     

  3. Update the Helm repository:

     helm repo update ibm-helm
     

  4. Check the version of the chart you will be upgrading to is the intended version:

     helm show chart ibm-helm/ibm-eventstreams-operator
     

     Check the version: value in the output, for example: version: 12.3.0

• If the chart version for your existing deployment is earlier than 12.2.x, you must first upgrade your installation to 12.2.x and then follow these instructions to upgrade to chart version 12.3.x.

• If your existing installation is in an offline environment, you must carry out the steps in the offline installation instructions to download the CASE bundle and mirror the images for the new version you want to upgrade to, before running any helm commands.

• Complete the steps in Helm upgrade to update your Custom Resource Definitions (CRDs) and operator charts to the latest version. The operator will then upgrade your Event Streams instance automatically.

Note: If you use the Helm diff plugin, you might encounter a failure during the upgrade. For more information, see troubleshooting.

Upgrading by using Helm

You can upgrade your Event Streams on other Kubernetes platforms by using Helm.

Depending on how you installed the operator, select one of the following upgrade approaches:

• If you installed the operator with all required resources together, follow the instructions in upgrading the operator with cluster-scoped resources.
• If you installed the cluster-scoped resources and operator separately, follow the instructions in upgrading the operator and cluster-scoped resources separately.

Upgrading the operator with cluster-scoped resources

If you installed the Event Streams operator and cluster-scoped resources together, upgrade to the latest version by running the following command:

helm upgrade \
<release-name> ibm-helm/ibm-eventstreams-operator \
-n <namespace> \
--set watchAnyNamespace=<true/false> \
--set previousVersion=<previous-version>

Where:

• <release-name> is the name you provide to identify your operator.
• <namespace> is the name of the namespace where you want to install the operator.
• watchAnyNamespace=<true/false> determines whether the operator manages instances of Event Streams in any namespace or only a single namespace (default is false if not specified). For more information, see choosing operator installation mode.

• <previous-version> is the version of the Helm chart being upgraded from. For example, if your Helm chart version is 12.2.2, set the field as: --set previousVersion=12.2.2. You can retrieve the version of your existing Helm chart by running the following command:

  helm list --filter <release-name> -n <namespace> -o json | jq '.[0].app_version'
  

Wait for the operator to reconcile the Event Streams instance and roll all the pods.

Upgrading the operator and cluster-scoped resources separately

If you installed the Event Streams operator and cluster-scoped resources separately, upgrade them in the following order:

1. Upgrade the cluster-scoped resources (CRDs and ClusterRoles):

   helm upgrade \
   <crd-release-name> ibm-helm/ibm-eventstreams-operator \
   -n <namespace> \
   --set namespaceScopedResources=false \
   --set watchAnyNamespace=<true/false> \
   --set previousVersion=<previous-version>
   

   Where:

   • <crd-release-name> is the name of the Helm release for the cluster-scoped resources (for example, es-crds).
   • <namespace> is the namespace where the cluster-scoped resources were installed.
   • namespaceScopedResources=false ensures that only cluster-scoped resources are upgraded.
   • watchAnyNamespace=<true/false> must match the value used during installation.
   • <previous-version> is the version of the Helm chart being upgraded from (for example, 12.2.2).

2. After the cluster-scoped resources are upgraded, upgrade the operator:

   helm upgrade \
   <operator-release-name> ibm-helm/ibm-eventstreams-operator \
   -n <namespace> \
   --set clusterScopedResources=false \
   --set watchAnyNamespace=<true/false> \
   --set previousVersion=<previous-version>
   

   Where:

   • <operator-release-name> is the name of the Helm release for the operator (for example, eventstreams).
   • <namespace> is the namespace where the operator is installed.
   • clusterScopedResources=false ensures that only namespace-scoped resources are upgraded.
   • watchAnyNamespace=<true/false> must match the value used during installation.
   • <previous-version> is the version of the Helm chart being upgraded from (for example, 12.2.2).

Wait for the operator to reconcile the Event Streams instance and roll all the pods.

Upgrade reconciliation

Wait for the operator to reconcile the Event Streams instance and roll all the pods.

Note: During this reconciliation process, the Kafka controller pod might temporarily enter a CrashLoopBackOff state with the following error. This happens because the new operator has updated the ConfigMap, but the pod is still using the previous image. This is expected and resolves automatically within a few minutes after the new image is updated by the operator. No manual intervention is required, and the Kafka cluster remains operational throughout this period.

cat: /opt/kafka/custom-config/metadata.state: No such file or directory

Verifying the upgrade

After the upgrade, verify the version and status of Event Streams by using the CLI or the UI.

Post-upgrade tasks

Remove the Kafka Proxy

In Event Streams 12.2.0 and later, the Kafka Proxy is deprecated. Your applications can continue to use Kafka Proxy services to access Kafka. Consider updating your applications to use the default Kafka services and then remove the Kafka Proxy services as follows.

To remove the Kafka Proxy, delete the spec.kafkaProxy: {} property from your EventStreams custom resource:

Note: When removing the Kafka Proxy, ensure that your producers and consumers do not depend on a Kafka Proxy and are connected directly to Kafka brokers.

apiVersion: eventstreams.ibm.com/v1beta2
kind: EventStreams
# ...
spec:
  kafkaProxy: {} # Delete this line
# ...

Migrate ImageContentSourcePolicy to ImageDigestMirrorSet

The ImageContentSourcePolicy API is deprecated in OpenShift 4.14 and later versions. If you have ImageContentSourcePolicy resources configured, you must migrate them to ImageDigestMirrorSet.

Note: This migration is only required if you are running in an offline environment and have existing ImageContentSourcePolicy resources from a previous installation.

Complete the following steps to migrate:

1. Log in to your Red Hat OpenShift Container Platform as a cluster administrator by using the oc CLI (oc login).

2. Get the name of the ImageContentSourcePolicy resources on your cluster:

   oc get ImageContentSourcePolicy
   

3. For Event Streams, migrate the ImageContentSourcePolicy to ImageDigestMirrorSet:

   a. Set an environment variable to the name of the Event Streams ImageContentSourcePolicy. For example, if the policy name is ibm-eventstreams:

   export ES_ICSP=ibm-eventstreams
   

   b. Save the ImageContentSourcePolicy as a YAML file:

   oc get ImageContentSourcePolicy ${ES_ICSP} -o yaml >> ${ES_ICSP}.yaml
   

   c. Convert the ImageContentSourcePolicy to ImageDigestMirrorSet:

   oc create -f $(oc adm migrate icsp ${ES_ICSP}.yaml | cut -f 4 -d ' ')
   

   Note: This command might trigger node upgrades. Wait for all the nodes to be in Ready state before you proceed to the next step.

   d. Delete the ImageContentSourcePolicy:

   oc delete ImageContentSourcePolicy ${ES_ICSP}
   

   Note: This command might trigger node upgrades. Wait for all the nodes to be in Ready state before you proceed to the next step.

4. Verify that the ImageDigestMirrorSet resources are created:

   oc get imagedigestmirrorset
   

   Important: After the ImageDigestMirrorSet resources are applied, you might see the node status as Ready, Scheduling, or Disabled. Wait until all the nodes show a Ready status.

5. Verify your cluster node status and wait for all nodes to be updated before proceeding:

   oc get MachineConfigPool -w
   

For more information, see the OpenShift documentation.