Upgrading and migrating

Upgrade your Event Streams installation as follows. The Event Streams operator handles the upgrade of your Event Streams instance.

Upgrade paths

Understand the upgrade paths available for Continuous Delivery (CD) releases and Extended Update Support (EUS) releases.

Upgrade paths for CD releases

You can upgrade Event Streams to the latest 11.1.6 version directly from any 11.1.x or 11.0.x version by using operator version 3.1.6. If you have an earlier Event Streams version than 11.0.x, you must first upgrade it to version 11.0.x before upgrading to 11.1.x.

Note: If your operator upgrades are set to automatic, minor version upgrades are completed automatically. This means that the Event Streams operator is upgraded to 3.1.6 when it is available in the catalog, and your Event Streams instance is then also automatically upgraded, unless you set a schedule for the upgrade by pausing the reconciliation.

Upgrade paths for EUS releases

No direct upgrade from EUS to the latest version is supported. To upgrade to latest version, you must upgrade to CD version 11.0.x, then upgrade to 11.1.x, and then proceed to upgrade your Event Streams version to the latest by following the instructions on this page starting with the prerequisites.

Prerequisites

Ensure you have followed the upgrade steps for IBM Cloud Pak for Integration before upgrading Event Streams.
The images for Event Streams release 11.1.x are available in the IBM Cloud Container Registry. Ensure you redirect your catalog source to use icr.io/cpopen as described in Implementing ImageContentSourcePolicy to redirect to the IBM Container Registry.
To upgrade successfully, your Event Streams instance must have more than one ZooKeeper node or have persistent storage enabled. If you upgrade an Event Streams instance with a single ZooKeeper node that has ephemeral storage, all messages and all topics will be lost and both ZooKeeper and Kafka pods will move to an error state. To avoid this issue, increase the number of ZooKeeper nodes before upgrading as follows:
```
 apiVersion: eventstreams.ibm.com/v1beta1
 kind: EventStreams
 metadata:
   name: example-pre-upgrade
   namespace: myproject
 spec:
   strimziOverrides:
     zookeeper:
       replicas: 3
```
If you installed the Event Streams operator to manage instances of Event Streams in any namespace (one per namespace), then you might need to control when each of these instances is upgraded to the latest version. You can control the updates by pausing the reconciliation of the instance configuration as described in the following sections.

Scheduling the upgrade of an instance

In 11.1.x and later, the Event Streams operator handles the upgrade of your Event Streams instance automatically after the operator is upgraded. No additional step is required to change the instance (product) version.

If your operator manages more than one instance of Event Streams, you can control when each instance is upgraded by pausing the reconciliation of the configuration settings for each instance, running the upgrade, and then unpausing the reconciliation when ready to proceed with the upgrade for a selected instance.

Pausing reconciliation by using the UI

Log in to the OpenShift Container Platform web console using your login credentials.
Expand Operators in the navigation on the left, and click Installed Operators.
From the Project list, select the namespace (project) the instance is installed in.
Locate the operator that manages your Event Streams instance in the namespace. It is called Event Streams in the Name column. Click the Event Streams link in the row.
Select the instance you want to pause and click the YAML tab.

In the YAML for the custom resource, add eventstreams.ibm.com/pause-reconciliation: 'true' to the metadata.annotations field as follows:

 apiVersion: eventstreams.ibm.com/v1beta2
 kind: EventStreams
 metadata:
   name: <instance-name>
   namespace: <instance-namespace>
   annotations:
     eventstreams.ibm.com/pause-reconciliation: 'true'

This annotation also needs to be applied to the corresponding Kafka custom resource. Expand Home in the navigation on the left, click API Explorer, and type Kafka in the Filter by kind... field. Select Kafka.
From the Project list, select the namespace (project) the instance is installed in and click the Instances tab.
Select the instance with the name <instance-name> (the same as the Event Streams instance).

In the YAML for the custom resource, add eventstreams.ibm.com/pause-reconciliation: 'true' to the metadata.annotations field as follows:

apiVersion: eventstreams.ibm.com/v1beta2
kind: Kafka
metadata:
  name: <instance-name>
  namespace: <instance-namespace>
  annotations:
    eventstreams.ibm.com/pause-reconciliation: 'true'

Follow the steps to upgrade by using the UI or the CLI.
To unpause the reconciliation and continue with the upgrade of an Event Streams instance, first remove the annotations from the Kafka custom resource, and then from the EventStreams custom resource. When the annotations are removed, the configuration of your instance is updated, and the upgrade to the latest version of Event Streams completes.

Pausing reconciliation by using the CLI

Log in to your Red Hat OpenShift Container Platform as a cluster administrator by using the oc CLI (oc login).
To apply the annotation first to the EventStreams and then to the Kafka custom resource, run the following command, where <type> is either EventStreams or Kafka:

oc annotate <type> <instance-name> -n <instance-namespace> eventstreams.ibm.com/pause-reconciliation='true'
Follow the steps to upgrade by using the UI or the CLI.
To unpause the reconciliation and continue with the upgrade of an Event Streams instance, run the following command to first remove the annotations from the Kafka custom resource, and then from the EventStreams custom resource, where <type> is either Kafka or EventStreams:

oc annotate <type> <instance-name> -n <instance-namespace> eventstreams.ibm.com/pause-reconciliation-

When the annotations are removed, the configuration of your instance is updated, and the upgrade to the latest version of Event Streams completes.

Upgrading by using the UI

If you are using the OpenShift Container Platform web console, complete the steps in the following sections to upgrade your Event Streams installation.

Log in to the OpenShift Container Platform web console using your login credentials.
Expand Operators in the navigation on the left, and click Installed Operators.
From the Project list, select the namespace (project) the instance is installed in.
Locate the operator that manages your Event Streams instance in the namespace. It is called Event Streams in the Name column. Click the Event Streams link in the row.
Click the Subscription tab to display the Subscription details for the Event Streams operator.
Click the version number link in the Update channel section (for example, v3.0). The Change Subscription update channel dialog is displayed, showing the channels that are available to upgrade to.
Select v3.1 and click the Save button on the Change Subscription Update Channel dialog.

Important: If you receive a warning message similar to the following in the status, remove it by changing the value of the spec.version field to latest in the YAML file:
```
Invalid value '11.0.4' set for spec.version. Your instance will be reconciled as 'latest', the only valid value for spec.version. Ensure you set spec.version to 'latest'.
```
All Event Streams pods that need to be updated as part of the upgrade will be gracefully rolled. Where required, ZooKeeper pods will roll one at a time, followed by Kafka brokers rolling one at a time.

Note: The number of containers in each Kafka broker will reduce from 2 to 1 as the TLS-sidecar container will be removed from each broker during the upgrade process.

Upgrading by using the CLI

If you are using the OpenShift command-line interface (CLI), the oc command, complete the steps in the following sections to upgrade your Event Streams installation.

Log in to your Red Hat OpenShift Container Platform as a cluster administrator by using the oc CLI (oc login).
Ensure the required Event Streams Operator Upgrade Channel is available:

oc get packagemanifest ibm-eventstreams -o=jsonpath='{.status.channels[*].name}'
Change the subscription to move to the required update channel, where vX.Y is the required update channel (for example, v3.0):

oc patch subscription -n <namespace> ibm-eventstreams --patch '{"spec":{"channel":"vX.Y"}}' --type=merge

Important: You might receive a warning message similar to the following in the status:
```
Invalid value '11.0.4' set for spec.version. Your instance will be reconciled as 'latest', the only valid value for spec.version. Ensure you set spec.version to 'latest'.
```
To remove the warning, change the spec.version to latest by entering the following command:

oc patch eventstreams -n <namespace> <name-of-the-es-instance> --patch '{"spec":{"version":"latest"}}' --type=merge

All Event Streams pods that need to be updated as part of the upgrade will be gracefully rolled. Where required, ZooKeeper pods will roll one at a time, followed by Kafka brokers rolling one at a time.

Verifying the upgrade

Wait for all Event Streams pods to complete the upgrade process. This is indicated by the Running state.
Log in to your Red Hat OpenShift Container Platform as a cluster administrator by using the oc CLI (oc login).
To retrieve a list of Event Streams instances, run the following command:

oc get eventstreams -n <namespace>
For the instance of Event Streams that you upgraded, check that the status returned by the following command is Ready.

oc get eventstreams -n <namespace> <name-of-the-es-instance> -o jsonpath="'{.status.phase}'"
To check the version of your Event Streams instance, run the following command:

oc get eventstreams -n <namespace> <name-of-the-es-instance> -o jsonpath="'{.status.versions.reconciled}'"

Post-upgrade tasks

Enable collection of producer metrics

In Event Streams version 11.0.0 and later, a Kafka Proxy handles gathering metrics from producing applications. The information is displayed in the Producers dashboard. The proxy is optional and is not enabled by default. To enable metrics gathering and have the information displayed in the dashboard, enable the Kafka Proxy.

Enable metrics for monitoring

To display metrics in the monitoring dashboards of the Event Streams UI, ensure you enable the OpenShift Container Platform monitoring stack.

Migrate to latest Apicurio Registry

Event Streams 11.1.6 icon Apicurio client libraries versions 2.3.1 and earlier use a date format that is not compatible with Apicurio Registry server versions 2.4.1 or later. Apicurio Registry is only deployed when you update the Event Streams custom resource to use the latest version of Apicurio Registry included with Event Streams.

Migrate your schema registry to use the latest Apicurio Registry as follows:

Ensure all applications connecting to your instance of Event Streams that use the schema registry are using Apicurio client libraries version 2.4.1 or later before migrating.
Log in to your Red Hat OpenShift Container Platform as a cluster administrator by using the oc CLI (oc login).
Add the eventstreams.ibm.com/apicurio-registry-version='>=2.4' annotation to your Event Streams custom resource with the following command:
```
oc annotate --namespace <namespace> EventStreams <instance-name> eventstreams.ibm.com/apicurio-registry-version='>=2.4'
```

The Event Streams operator will update your schema registry to use the latest version of Apicurio Registry included with Event Streams.