Installing on other Kubernetes platforms

The following sections provide instructions about installing Event Streams on Kubernetes platforms that support the Red Hat Universal Base Images (UBI) containers.

Overview

Event Streams is an operator-based release and uses custom resources to define your Event Streams configurations. The Event Streams operator uses the custom resources to deploy and manage the entire lifecycle of your Event Streams instances. Custom resources are presented as YAML configuration documents that define instances of the EventStreams custom resource type.

When deploying in an air-gapped (also referred to as offline or disconnected) environment, follow the instructions in the offline installation to install Event Streams on OpenShift and other Kubernetes platforms by using the CASE package.

Installing Event Streams has two phases:

  1. Use Helm to install the Event Streams operator: this will deploy the operator that will install and manage your Event Streams instances.
  2. Install one or more instances of Event Streams by using the operator.

Before you begin

  • Ensure you have set up your environment according to the prerequisites.
  • Ensure you have planned for your installation, such as preparing for persistent storage, considering security options, and considering adding resilience through multiple availability zones.
  • Obtain the connection details for your Kubernetes cluster from your administrator.

Create a namespace

Create a namespace into which the Event Streams instance will be installed. For more information about namespaces, see the Kubernetes documentation.

Ensure you use a namespace that is dedicated to a single instance of Event Streams. This is required because Event Streams uses network security policies to restrict network connections between its internal components. A single namespace per instance also allows for finer control of user accesses.

Important: Do not use any of the initial or system namespaces to install an instance of Event Streams (default, kube-node-lease, kube-public, and kube-system).

Add the Helm repository

Before you can install the Event Streams operator and use it to create instances of Event Streams, add the IBM Helm repository to your local repository list. This will provide access to the Event Streams Helm chart package that will install the operator on your cluster.

To add the IBM Helm repository to the local repository list, run the following command:

helm repo add ibm-helm https://raw.githubusercontent.com/IBM/charts/master/repo/ibm-helm

Install the Event Streams operator

Ensure you have considered the Event Streams operator requirements, including resource requirements and the required cluster-scoped permissions.

Choosing operator installation mode

Before installing the Event Streams operator, decide if you want the operator to:

  • Manage instances of Event Streams in any namespace.

    To use this option, set watchAnyNamespace: true when installing the operator. The operator will be deployed into the specified namespace, and will be able to manage instances of Event Streams in any namespace.

  • Only manage instances of Event Streams in a single namespace.

    This is the default option: if watchAnyNamespace is not set, then it defaults false. The operator will be deployed into the specified namespace, and will only be able to manage instances of Event Streams in that namespace.

    Important: When multiple Event Streams operators are installed on the same cluster, all the operators share the same custom resource definitions (CRDs). Do not install a previous version of the Event Streams operator if a later version is already installed on the same cluster.

Installing the operator

To install the operator, run the following command:

helm install \
   <release-name> ibm-helm/ibm-eventstreams-operator \
   -n <namespace> \
   --set watchAnyNamespace=<true/false>

Where:

  • <release-name> is the name you provide to identify your operator.
  • <namespace> is the name of the namespace where you want to install the operator.

  • watchAnyNamespace=<true/false> determines whether the operator manages instances of Event Streams in any namespace or only a single namespace (default is false if not specified).

    Set to true for the operator to manage instances in any namespace, or do not specify if you want the operator to only manage instances in a single namespace.

For example, to install the operator on a cluster where it will manage all instances of Event Streams, run the command as follows:

helm install eventstreams ibm-helm/ibm-eventstreams-operator -n "my-namespace" --set watchAnyNamespace=true

For example, to install the operator that will manage Event Streams instances in only the my-eventstreams namespace, run the command as follows:

helm install eventstreams ibm-helm/ibm-eventstreams-operator -n "my-eventstreams"

Note: If you are installing any subsequent operators in the same cluster, ensure you run the helm install command with the --set createGlobalResources=false option (as these resources have already been installed).

Checking the operator status

To check the status of the installed operator, run the following command:

kubectl get deploy eventstreams-cluster-operator -n <namespace>

Where:

  • <namespace> is the name of the namespace where the operator is installed.

A successful installation will return a result similar to the following with 1/1 in the READY column:

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
eventstreams-cluster-operator   1/1     1            1           7d4h

Technology Preview feature: KRaft

Technology Preview features are available to evaluate potential upcoming features. Such features are intended for testing purposes only and not for production use. IBM does not support these features, but might help with any issues raised against them. IBM welcomes feedback on Technology Preview features to improve them. As the features are still under development, functions and interfaces can change, and it might not be possible to upgrade when updated versions become available.

IBM offers no guarantee that Technology Preview features will be part of upcoming releases and as such become fully supported.

Event Streams version 11.1.5 and later includes Apache Kafka Raft (KRaft) as a Technology Preview feature. KRaft replaces ZooKeeper for managing metadata, moving the overall handling of metadata into Kafka itself.

When the UseKRaft and KafkaNodePools feature gates are enabled, the Kafka cluster is deployed without ZooKeeper. The spec.strimziOverrides.zookeeper properties in the EventStreams custom resource will be ignored, but still need to be present. The UseKRaft and KafkaNodePools feature gates provide an API that configure Kafka cluster nodes and their roles. The API is still in development and is expected to change before the KRaft mode is production-ready.

Limitations

The KRaft mode in Event Streams has the following limitations:

  • Moving existing Kafka clusters deployed with ZooKeeper to use KRaft, or the other way around, is not supported.
  • Upgrading your Apache Kafka or Event Streams operator version, or reverting either one to an earlier version is not supported. To do so, you delete the cluster, upgrade the operator, and deploy a new Kafka cluster.
  • The Topic Operator is not supported. The spec.entityOperator.topicOperator property must be removed from the Kafka custom resource.
  • JBOD storage is not supported. You can use type: jbod for storage, but the JBOD array can contain only one disk.
  • Geo-replication is not supported.

Enabling KRaft

To enable KRaft on non-OpenShift Kubernetes platforms using a Helm Chart, you can change the configuration values supplied in the values.yaml by adding extra arguments to helm install:

helm install .... --set featureGates.useKRaft=true --set featureGates.kafkaNodePools=true ...

Install an Event Streams instance

Instances of Event Streams can be created after the Event Streams operator is installed. If the operator was installed to manage a specific namespace, then it can only be used to manage instances of Event Streams in that namespace. If the operator was installed to manage all namespaces, then it can be used to manage instances of Event Streams in any namespace, including those created after the operator was deployed.

When installing an instance of Event Streams, ensure you are using a namespace that an operator is managing.

Creating an image pull secret

Before installing an Event Streams instance, create an image pull secret called ibm-entitlement-key in the namespace where you want to create an instance of Event Streams. The secret enables container images to be pulled from the registry.

  1. Obtain an entitlement key from the IBM Container software library.
  2. Click Entitlement keys in the navigation on the left, and click Add new key, or if you have an existing active key available, click Copy to copy the entitlement key to the clipboard.

  3. Create the secret in the namespace that will be used to deploy an instance of Event Streams as follows.

    Name the secret ibm-entitlement-key, use cp as the username, your entitlement key as the password, and cp.icr.io as the docker server:

    kubectl create secret docker-registry ibm-entitlement-key --docker-username=cp --docker-password="<your-entitlement-key>" --docker-server="cp.icr.io" -n "<image-namespace>"

Note: If you do not create the required secret, pods will fail to start with ImagePullBackOff errors. In this case, ensure the secret is created and allow the pod to restart.

Installing an instance by using the CLI

To install an instance of Event Streams from the command line, you must first prepare an EventStreams custom resource configuration in a YAML file.

A number of sample configuration files are included in the Helm chart package to base your deployment on. The sample configurations range from smaller deployments for non-production development or general experimentation to large scale clusters ready to handle a production workload.

The sample configurations are also available in GitHub, where you can select the GitHub tag for your Event Streams version to access the correct samples, and then go to /cr-examples/eventstreams/kubernetes to access the samples for the Kubernetes platforms that support the Red Hat Universal Base Images (UBI) containers.

More information about these samples is available in the planning section. You can base your deployment on the sample that most closely reflects your requirements and apply customizations on top as required.

Important: Ensure that the spec.license.accept field in the custom resource YAML is set to true, and that the correct values are selected for the spec.license.license and spec.license.use fields before deploying the Event Streams instance. These values are used for metering purposes and could result in inaccurate charging and auditing if set incorrectly. For more information about the available options, see the licensing reference.

Note: If experimenting with Event Streams for the first time, the Lightweight without security sample is the smallest and simplest example that can be used to create an experimental deployment. For the smallest production setup, use the Minimal production sample configuration.

To deploy an Event Streams instance, run the following commands:

  1. Apply the configured EventStreams custom resource in the selected namespace:

    kubectl apply -f <custom-resource-file-path> -n "<image-namespace>"

    For example:

    kubectl apply -f development.yaml -n "my-eventstreams"
  2. Wait for the installation to complete.
  3. Verify your installation and consider other post-installation tasks.