Ensure your environment meets the following prerequisites before installing Event Streams.
Container environment
Event Streams 2019.1.1 is supported on the following platforms and systems:
| Container platform | Systems |
|---|---|
| Red Hat OpenShift Container Platform 3.9 and 3.10 with IBM cloud foundational services 3.1.2* | - Linux® 64-bit (x86_64) systems - Microsoft Azure (IaaS) - Amazon Web Services (IaaS) |
| IBM Cloud Private 3.1.1, 3.1.2, and 3.2.0.1907 (or later fix pack) | - Linux® 64-bit (x86_64) systems - Linux on IBM® Z systems |
| IBM Cloud Private 3.1.2 | - Microsoft Azure (IaaS) - Amazon Web Services (IaaS) |
*Provided by IBM Cloud Private
Event Streams 2019.1.1 has Helm chart version 1.2.0 and includes Kafka version 2.1.1. For an overview of supported component and platform versions, see the support matrix.
Ensure you have the following set up for your environment:
- If you are installing Event Streams on the OpenShift Container Platform, ensure you have OpenShift installed, and also ensure you install and integrate IBM Cloud Private version 3.1.2.
Note: Event Streams Community Edition is not supported on Red Hat OpenShift Container Platform. - Install and configure IBM Cloud Private.
Important: In high throughput environments, ensure you configure an external load balancer for your IBM Cloud Private cluster to take full advantage of Event Streams scaling, and avoid potential bottlenecks.
Note: Event Streams includes entitlement to IBM Cloud Private Foundation which you can download from IBM Passport Advantage. - If you are installing Event Streams on an IBM Cloud Private cluster deployed on Amazon Web Services (AWS), ensure your proxy address uses lowercase characters.
- Install the Kubernetes command line tool, and configure access to your cluster.
- If you are installing Event Streams on the OpenShift Container Platform, ensure you also install the OpenShift Container Platform CLI.
- Install the IBM Cloud Private Command Line Interface (CLI).
- Install the Helm CLI required for your version of IBM Cloud Private, and add the IBM Cloud Private internal Helm repository called
local-chartsto the Helm CLI as an external repository. - For message indexing capabilities (enabled by default), ensure you set the
vm.max_map_countproperty to at least262144on all IBM Cloud Private nodes in your cluster (not only the master node). Run the following commands on each node:
sudo sysctl -w vm.max_map_count=262144
echo "vm.max_map_count=262144" | tee -a /etc/sysctl.conf
Important: This property might have already been updated by other workloads to be higher than the minimum required.
Hardware requirements
The Helm chart for Event Streams specifies default values for the CPU and memory usage of the Apache Kafka brokers and Apache ZooKeeper servers.
See the following table for memory requirements of each Helm chart component. Ensure you have sufficient physical memory to service these requirements.
Kubernetes manages the allocation of containers within your cluster. This allows resources to be available for other Event Streams components which might be required to reside on the same node.
Ensure you have one IBM Cloud Private worker node per Kafka broker, and a minimum of 3 worker nodes available for use by Event Streams.
Helm resource requirements
The following table lists the resource requirements of the Event Streams Helm chart. For details about the requirements for each pod and their containers, see the tables in the following sections.
| Pod | Number of replicas | Total CPU per pod | Total memory per pod (Gi) |
|---|---|---|---|
| Kafka | 3* | 1* | 3.5* |
| ZooKeeper | 3 | 0.1* | 1* |
| Geo-replicator | 0* | 1 per replica | 1 per replica |
| Administration UI | 1 | 1 | 1 |
| Administration server | 1 | 4.5 | 2.5 |
| REST producer server | 1 | 4 | 2 |
| REST proxy | 1 | unlimited | unlimited |
| Collector | 1 | 1 | 1 |
| Network proxy | 2 | unlimited | unlimited |
| Access controller | 1 | 0.1 | 0.25 |
| Index manager | 1 | unlimited | unlimited |
| Elasticsearch | 2 | unlimited | 4 |
Important: You can configure the settings marked with an asterisk (*).
Note: Before installing Event Streams (not Community Edition), consider the number of Kafka replicas and geo-replicator nodes you plan to use. Each Kafka replica and geo-replicator node is a separate chargeable unit.
The CPU and memory limits for some components are not limited by the chart, so they inherit the resource limits for the namespace that the chart is being installed into. If there are no resource limits set for the namespace, the containers run with unbounded CPU and memory limits.
Note: Geo-replication is only available in the paid-for version of Event Streams (not available in the Community Edition).
Kafka pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| Kafka | 1* | 2* |
| Metrics reporter | unlimited | 1.5* |
| Metrics proxy | unlimited | unlimited |
| Healthcheck | unlimited | unlimited |
ZooKeeper pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| ZooKeeper | 0.1* | 1* |
Geo-replicator pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| Replicator | 1 | 1 |
| Metrics reporter | unlimited | unlimited |
Administration UI pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| UI | 1 | 1 |
| Redis | unlimited | unlimited |
| Proxy | unlimited | unlimited |
Administration server pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| Rest | 4 | 2 |
| Codegen | 0.5 | 0.5 |
REST producer server pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| Rest-producer | 4 | 2 |
REST proxy pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| Rest-proxy | unlimited | unlimited |
Collector pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| Collector | unlimited | unlimited |
Network proxy pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| Proxy | unlimited | unlimited |
Access controller pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| Access controller | 0.1 | 0.25 |
| Redis | unlimited | unlimited |
Index manager pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| Index manager | unlimited | unlimited |
Elasticsearch pod
| Container | CPU per container | Memory per container (Gi) |
|---|---|---|
| Elastic | unlimited | 4 |
PodSecurityPolicy requirements
To install the Event Streams chart, you must have the ibm-restricted-psp PodSecurityPolicy selected for the target namespace.
You can define the PodSecurityPolicy when creating the namespace for your installation.
Event Streams applies network policies to control the traffic within the namespace where it is deployed, limiting the traffic to that required by Event Streams. For more information about the network policies and the traffic they permit, see network policies.
For more information about PodSecurityPolicy definitions, see the IBM Cloud Private documentation.
Note: The PodSecurityPolicy requirements do not apply to the Red Hat OpenShift Container Platform.
Red Hat OpenShift SecurityContextConstraints Requirements
If you are installing on the OpenShift Container Platform, the Event Streams chart requires a custom SecurityContextConstraints to be bound to the target namespace prior to installation. The custom SecurityContextConstraints controls the permissions and capabilities required to deploy this chart.
You can enable this custom SecurityContextConstraints resource using the supplied pre-installation setup script.
Network requirements
Event Streams is supported for use with IPv4 networks only.
File systems for storage
If you want to set up persistent storage, you must have physical volumes available, backed by one of the following file systems:
- NFS version 4
- GlusterFS version 3.10.1
Event Streams user interface
The Event Streams user interface (UI) is supported on the following web browsers:
- Google Chrome version 65 or later
- Mozilla Firefox version 59 or later
- Safari version 11.1 or later
Event Streams CLI
The Event Streams command line interface (CLI) is supported on the following systems:
- Windows 10 or later
- Linux® Ubuntu 16.04 or later
- macOS 10.13 (High Sierra) or later
Clients
The Apache Kafka Java client included with Event Streams is supported for use with the following Java versions:
- IBM Java 8
- Oracle Java 8
You can also use other Kafka version 2.0 or later clients when connecting to Event Streams. If you encounter client-side issues, IBM can assist you to resolve those issues (see our support policy).
Event Streams is designed for use with clients based on the librdkafka implementation of the Apache Kafka protocol.