Upgrade your IBM Event Streams operator and operand instances as follows.
Upgrade paths
You must first upgrade the Event Streams operator, and then upgrade your Event Streams instance (operand version).
Upgrade paths for CD releases
The following upgrade paths are available for Continuous Delivery (CD) releases (2.x operators and 10.x operands and later, except for 2.2.x operators and 10.2.x operands):
- You can upgrade the Event Streams operator to the latest 3.0.5 version directly from versions 3.0.x, 2.5.x, and 2.4.x. If you have an earlier operator version than 2.4.0, you must first upgrade it to 2.4.0 before upgrading to 3.0.x.
- You can upgrade the Event Streams operand to the latest 11.0.4 version directly from versions 11.0.x, 10.5.x, and 10.4.x. If you have an earlier operand version than 10.4.0, you must first upgrade it to 10.4.0 before upgrading to 11.0.x.
Upgrade paths for EUS releases
You can also upgrade to the latest Event Streams CD release from the Event Streams Extended Update Support (EUS) release (2.2.x operators and 10.2.x operands) as follows:
- Upgrade your Event Streams EUS release by upgrading to the latest EUS operator (2.2.x). The latest operator revision for the EUS release ensures you have the latest updates and fixes applied, including updates to enable upgrading to the latest CD release.
- Upgrade your IBM Cloud Pak foundational services from EUS version 3.6.x to the latest CD version.
- After successfully upgrading to the latest CD version of foundational services, ensure you clean up the monitoring resources to avoid errors.
- Upgrade your Event Streams version to the latest CD release by following the instructions on this page starting with the prerequisites (operator version 3.0.5 and operand version 11.0.4).
Prerequisites
-
Ensure you have followed the upgrade steps for IBM Cloud Pak for Integration before upgrading Event Streams.
Note: If you upgrade your OpenShift version before upgrading your Event Streams operator version, the Event Streams operand might display a
Failed
status temporarily. This will be resolved when you upgrade your operator and then the operand version, the next steps in the upgrade process. -
The images for Event Streams release 11.0.x are available in the IBM Cloud Container Registry. Ensure you redirect your catalog source to use
icr.io/cpopen
as described in Implementing ImageContentSourcePolicy to redirect to the IBM Container Registry. -
To upgrade successfully, your Event Streams instance must have more than one ZooKeeper node or have persistent storage enabled. If you upgrade an Event Streams instance with a single ZooKeeper node that has ephemeral storage, all messages and all topics will be lost and both ZooKeeper and Kafka pods will move to an error state. To avoid this issue, increase the number of ZooKeeper nodes before upgrading as follows:
apiVersion: eventstreams.ibm.com/v1beta1 kind: EventStreams metadata: name: example-pre-upgrade namespace: myproject spec: strimziOverrides: zookeeper: replicas: 3
-
If you have an Event Streams 11.0.3 or earlier installation, and you previously added your own Kafka or Zookeeper metrics rules, then ensure you record these elsewhere to be added to the
metrics-config
ConfigMap after upgrading.
Important: If you have configured custom certificates on any of the listeners in your Event Streams custom resource, there will be an outage when you upgrade the operand to any 11.0.x version until you reconfigure your listeners with your custom certificates. This is because the upgrade involves a migration of the listener format, which removes any reference to the custom certificates. To ensure you are prepared to quickly deal with this outage after a successful upgrade, review how you can reconfigure your custom certificates.
Upgrading by using the UI
The upgrade process requires the upgrade of the Event Streams operator, and then the upgrade of your Event Streams instances. If you are using the OpenShift Container Platform web console, complete the steps in the following sections to upgrade your Event Streams installation.
Upgrade the Event Streams operator
- Log in to the OpenShift Container Platform web console using your login credentials.
- Expand Operators in the navigation on the left, and click Installed Operators.
- From the Project list, select the namespace (project) the instance is installed in.
- Locate the operator that manages your Event Streams instance in the namespace. It is called IBM Event Streams in the Name column. Click the IBM Event Streams link in the row.
- Click the Subscription tab to display the Subscription details for the Event Streams operator.
- Click the version number link in the Update channel section (for example, v2.5). The Change Subscription update channel dialog is displayed, showing the channels that are available to upgrade to.
- Select v3.0 and click the Save button on the Change Subscription Update Channel dialog.
All Event Streams pods that need to be updated as part of the upgrade will be gracefully rolled. Where required, ZooKeeper pods will roll one at a time, followed by Kafka brokers rolling one at a time.
Important: The Entity operator might display a CrashLoopBackOff
error. In addition, some Event Streams pods might display errors temporarily when the operator version has been updated. You can ignore these errors and continue with upgrading the Event Streams operand version, which will resolve these errors.
Note: The number of containers in each Kafka broker will reduce from 2 to 1 as the TLS-sidecar container will be removed from each broker during the upgrade process.
Upgrade the Event Streams operand (instance)
- Click Installed Operators from the navigation on the left to view the list of installed operators, including the upgraded IBM Event Streams operator.
- Select the IBM Event Streams operator from the list of Installed Operators.
- Click the Event Streams tab. This lists the Event Streams operands.
- Find your instance in the Name column and click the link for the instance.
- Click the YAML tab. The Event Streams instance custom resource is shown.
- In the YAML, change the
spec.version
field to the required version, for example, 11.0.4. - Click the Save button.
All Event Streams pods will gracefully roll again.
Alternatively, you can also upgrade your Event Streams instance in the IBM Cloud Pak for Integration Platform UI as follows:
- Log in to the IBM Cloud Pak for Integration Platform UI.
- Click the Navigation Menu in the top left.
- Expand Administration and click Integration instances.
If an update is available for a runtime, the
Information icon displays next to the runtime’s current Version number.
- Click the
More options in the row for the Event Streams instance, and then click Change version.
- Select 11.0.4 from the Select a new channel or version list.
- Click Change version to save your selections and start the upgrade.
In the runtimes table, the Status column for the runtime displays the
Upgrading
message. The upgrade is complete when the Status isReady
and the Version displays the new version number.
Upgrading by using the CLI
The upgrade process requires the upgrade of the Event Streams operator, and then the upgrade of your Event Streams instances. If you are using the OpenShift command-line interface (CLI), the oc command, complete the steps in the following sections to upgrade your Event Streams installation.
Upgrade the Event Streams operator
- Log in to your Red Hat OpenShift Container Platform as a cluster administrator by using the
oc
CLI (oc login
). -
Ensure the required IBM Event Streams Operator Upgrade Channel is available:
oc get packagemanifest ibm-eventstreams -o=jsonpath='{.status.channels[*].name}'
-
Change the subscription to move to the required update channel, where
vX.Y
is the required update channel (for example,v3.0
):oc patch subscription -n <namespace> ibm-eventstreams --patch '{"spec":{"channel":"vX.Y"}}' --type=merge
All Event Streams pods that need to be updated as part of the upgrade will be gracefully rolled. Where required, ZooKeeper pods will roll one at a time, followed by Kafka brokers rolling one at a time.
Important: The Entity operator might display a CrashLoopBackOff
error. In addition, some Event Streams pods might display errors temporarily when the operator version has been updated. You can ignore these errors and continue with upgrading the Event Streams operand version, which will resolve these errors.
Note: The number of containers in each Kafka broker will reduce from 2 to 1 as the TLS-sidecar container will be removed from each broker during the upgrade process.
Upgrade the Event Streams operand (instance)
Upgrade the Event Streams instance to move to the required version, where X.Y.Z
is the required version, for example, 11.0.4.
oc patch eventstreams -n <namespace> <name-of-the-es-instance> --patch '{"spec":{"version":"X.Y.Z"}}' --type=merge
All Event Streams pods will gracefully roll again.
Verifying the upgrade
- Wait for all Event Streams pods to complete the upgrade process. This is indicated by the
Running
state. - Log in to your Red Hat OpenShift Container Platform as a cluster administrator by using the
oc
CLI (oc login
). - To retrieve a list of Event Streams instances, run the following command:
oc get eventstreams -n <namespace>
- For the instance of Event Streams that you upgraded, check that the status returned by the following command is
Ready
.
oc get eventstreams -n <namespace> <name-of-the-es-instance> -o jsonpath="'{.status.phase}'"
Post-upgrade tasks
Reconfigure listeners with custom certificates
To enable your clients to connect with the upgraded Event Streams that uses custom certificates, add the certificate configuration again in the new format as follows:
- name: external
type: route
port: 9092
authentication:
type: scram-sha-512
tls: true
configuration:
brokerCertChainAndKey:
certificate: my-listener-certificate.crt
key: my-listener-key.key
secretName: my-secret
Enable collection of producer metrics
In IBM Event Streams version 11.0.0 and later, a Kafka Proxy handles gathering metrics from producing applications. The information is displayed in the Producers dashboard. The proxy is optional and is not enabled by default. To enable metrics gathering and have the information displayed in the dashboard, enable the Kafka Proxy.
Update metrics rules
If you previously added your own Kafka or Zookeeper metrics rules, then ensure you add these rules again to the metrics-config
ConfigMap. The rules in the following ConfigMap are the default rules. Add your own custom rules to data.kafka-metrics-config.yaml.rules:
or data.zookeeper-metrics-config.yaml.rules:
after the default rules.
If you do not want to export metrics then delete all of the rules in the metrics-config
ConfigMap.
kind: ConfigMap
apiVersion: v1
metadata:
name: metrics-config
data:
kafka-metrics-config.yaml: |
lowercaseOutputName: true
rules:
- attrNameSnakeCase: false
name: kafka_controller_$1_$2_$3
pattern: kafka.controller<type=(\w+), name=(\w+)><>(Count|Value|Mean)
- attrNameSnakeCase: false
name: kafka_server_BrokerTopicMetrics_$1_$2
pattern: kafka.server<type=BrokerTopicMetrics, name=(BytesInPerSec|BytesOutPerSec)><>(Count)
- attrNameSnakeCase: false
name: kafka_server_BrokerTopicMetrics_$1__alltopics_$2
pattern: kafka.server<type=BrokerTopicMetrics, name=(BytesInPerSec|BytesOutPerSec)><>(OneMinuteRate)
- attrNameSnakeCase: false
name: kafka_server_ReplicaManager_$1_$2
pattern: kafka.server<type=ReplicaManager, name=(\w+)><>(Value)
zookeeper-metrics-config.yaml: |
lowercaseOutputName: true
rules: []
Deprecation warnings for metrics
The metrics
configuration option for the EventStreams
custom resource has been deprecated. If you receive deprecation warnings about spec.kafka.metrics:
{}
or spec.zookeeper.metrics:{}
after upgrading, then remove the metrics: {}
line from the EventStreams
custom resource.
Enable metrics for monitoring
To display metrics in the monitoring dashboards of the Event Streams UI, ensure you enable the OpenShift Container Platform monitoring stack.