Installing the Event Gateway as a Kubernetes Deployment

Before you begin

Review the planning topic to decide how you want to deploy your Event Gateway.

Licensing

If your Event Manager is not using a usage-based license, then first install the IBM Licensing Service in the same environment as the Event Gateway.

Required information

Decide on hostnames for your Event Gateway, identify the DNS domain of your cluster, and confirm the network path to your Event Gateway endpoint.

  • DNS domain: Determine the DNS domain of your cluster’s ingress controller. On OpenShift Container Platform, the default domain can be found with this command:

    oc get ingresses.config/cluster -o jsonpath={.spec.domain}

  • Hostname: To determine the number of hostnames you require for each gateway and gateway group, see the guidance about gateway deployment planning.

    Tip: To help identify your Event Gateway, set the hostname as <gateway group>-<gateway-id>-<kafka broker>, where <kafka broker> is an integer to represent each Kafka broker across the Kafka clusters that your Event Gateway is to manage traffic for.

    Alternatively, you can use a wildcard for the hostname, for example: *-gwy1-grp1.example.com:9092.

  • Fully Qualified Domain Name (FQDN) Determine the (FQDN) of your Event Gateway. The FQDN is formed from <gateway hostname>.<dns ingress domain>.

    For example, if your Kafka cluster has 3 brokers, your gateway group name is grp1, your gateway ID is gwy1, and your DNS domain is mydomain.com then the FQDNs for your Event Gateway are:

    • grp1-gwy1-1.mydomain.com
    • grp1-gwy1-2.mydomain.com
    • grp1-gwy1-3.mydomain.com

  • External access: How is your gateway to be exposed externally? The URL and configuration of the TLS certificate of your gateway endpoint depend on how the gateway endpoint is exposed. On OpenShift Container Platform, you must configure Routes for external access.

Creating TLS certificates for the Event Gateway

The best way to create the TLS certificates to secure your Event Gateway endpoint is to use cert-manager. Cert-manager creates TLS certificates based on a YAML definition, stores them in Kubernetes secrets, and automatically renews them before they expire. Cert-manager supports integration with external public signers, and can also create self-signed certificates.

On Kubernetes, install the cert-manager community operator. Cert-manager is included on Red Hat OpenShift: Red Hat OpenShift cert-manager.

Creating a self-signed certificate

The steps in this section show how to create self-signed certificates that you can use to secure your gateway endpoint.

If you want to create externally signed certificates with cert-manager, then see the cert-manager documentation.

  1. Create a ClusterIssuer object. A ClusterIssuer object can issue certificates in all namespaces. If you want to confine the issuer to a single namespace, then create an Issuer object instead.

    a. Create a YAML file that is called ClusterIssuer.yaml and paste in the following contents:

    apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-issuer
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: eem-selfsigned-ca
  namespace: cert-manager
spec:
  isCA: true 
  commonName: eem-selfsigned-ca
  secretName: root-secret
  privateKey:
    algorithm: RSA
  issuerRef:
    name: selfsigned-issuer
    kind: ClusterIssuer
    group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: eem-ca-issuer
spec:
  ca:
    secretName: root-secret

    b. Apply the YAML file in your Kubernetes environment:

    kubectl -n cert-manager apply -f ClusterIssuer.yaml

  2. Use the ClusterIssuer to create your gateway certificate.

    a. Create a file that is called GatewayCertificate.yaml and paste in the following contents:

    apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: eem-gateway-cert-request
spec:
  secretName: <gateway group>-<gateway ID>-certs
  privateKey:
    algorithm: RSA
  duration: 2160h # 90d
  renewBefore: 360h # 15d
  isCA: false
  usages:
    - server auth
  subject:
    organizations:
      - cert-manager
  commonName: <common name>
  dnsNames:
    - <hostname1.ingressdomain>
    - <hostname2.ingressdomain>
    - <hostname3.ingressdomain> 
  issuerRef:
    name: eem-ca-issuer
    kind: ClusterIssuer

    Set the following properties:

    • spec.secretName: Replace <gateway group> and <gateway ID> with the names that you intend to use for your gateway group and gateway ID.
    • spec.commonName: Set a human-readable name for your certificate.
    • spec.dnsNames: Set to a list of the FQDNs for your Event Gateway.

    b. Apply the YAML file in your Kubernetes environment:

    kubectl -n <gateway namespace> apply -f GatewayCertificate.yaml

    c. Verify that your gateway secret is created:

    kubectl -n <gateway namespace> get secrets

    The secret contains the gateway certificate and key in the properties spec.tls.crt and spec.tls.key.

Generating your Event Gateway Kubernetes Deployment YAML

  1. In the Event Endpoint Management UI navigation pane, click Administration > Event Gateways.
  2. Click Add gateway.
  3. Select the Kubernetes Deployment tile, then click Next.

  4. Provide the configuration properties for your gateway.

    You must provide these properties to be able to generate the YAML:

    • Gateway group: Create or specify an existing gateway group for your new gateway.
    • Gateway ID: Provide an ID for your new gateway that is unique within the gateway group.
    • Replicas: The number of Kubernetes replicas of the gateway pod to create.

    The remaining properties are set later in Configuring your Event Gateway Kubernetes Deployment YAML:

    • Server URL is a comma-separated list of the FQDNs and ports of the Event Gateway. See Configuring your Event Gateway Kubernetes Deployment YAML.
    • Gateway private key, Gateway certificate, CA certificate are the TLS certificates that are used to secure your Event Gateway endpoint. It is recommended to not upload certificates here, and to use cert-manager to create and manage your certificates, but if you do not want to use cert-manager, then you can upload the certificate PEM files here.
  5. Copy the generated YAML to two separate files:
    • <gateway name>-gateway_k8s_original.yaml
    • <gateway name>-gateway_k8s.yaml

    Important: Keep <gateway name>-gateway_k8s_original.yaml in a safe location and do not edit it. To remove write permissions to avoid accidental updates to this file, you can run chmod a-w <gateway name>-gateway_k8s_original.yaml.

Configuring your Event Gateway Kubernetes Deployment YAML

Open the <gateway name>-gateway_k8s.yaml file that you generated and replace all <placeholder> properties:

  • In the Deployment section, set spec.template.spec.containers.env.ACCEPT_LICENSE to "true".

  • In the ConfigMap section, set kafka.listener.listener.group.group.addresses to the FQDNs and ports of your gateway. For example:

    kafka.listener.listener.group.group.addresses=grp1-gwy1-1.ingress.domain:443,grp1-gwy1-2.ingress.domain:443,grp1-gwy1-3.ingress.domain:443
  • If you do not want the gateway CA certificate to be downloadable by users from the Event Endpoint Management UI, then in the ConfigMap section, delete the kafka.listener.listener.group.group.trust.pem property.
  • If you did not provide TLS certificates and a key in step 4, then delete the <group name>-<gateway ID>-certs secret from the YAML file and verify that the name of the secret you created matches the value in spec.template.spec.containers[egw].volumes[certs].secret.secretName.
  • The generated YAML includes a Kubernetes service definition for your gateway. A Kubernetes service is required for client access to your gateway. If the generated service configuration does not suit your requirements then you can either update it in the generated YAML, or delete the definition from the YAML and create the Kubernetes service later.

Install your Kubernetes Deployment Event Gateway

  1. Apply the <gateway name>-gateway_k8s.yaml file in your Kubernetes environment by using the kubectl command. For example:

    kubectl -n <gateway namespace> apply -f <gateway name>-gateway_k8s.yaml
  2. To monitor the deployment of the Event Gateway in the Event Endpoint Management UI navigation pane, click Administration > Event Gateways. When the gateway is registered, the status reports Running.
  3. Enable client network access to your gateway.

Enabling client network access

A Kubernetes Service must be created on all types of Kubernetes cluster, including OpenShift Container Platform. The service must allow access to the gateway for all types of traffic.

If the Event Gateway is to be exposed outside the host cluster, then you must also create the appropriate ingresses or OpenShift Container Platform Routes.

Ensure that TLS pass-through is enabled in all network infrastructure and software that processes communication between your Kafka client and your Event Gateway (such as load balancers).

Creating the Kubernetes Service

The Kubernetes service must map the internal ports that you defined in your gateway endpoint for the generated items to match. The service can be created only after the Event Gateway Kubernetes Deployment is created.

A service definition is created for you in the gateway YAML. If this definition does not suit your requirements, then you can generate a service definition YAML based on your Event Gateway deployment and update as required with the following command:

kubectl expose deployment/<gateway deployment name> --target-port=<port number> --port=<port number> -o yaml --dry-run=client

For example:

# generated with oc expose deployment/grp1-gwy1 --target-port=8443 --port=8443 -o yaml --dry-run=client
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: grp1-gwy1
    gatewayGroup: grp1
    gatewayId: gw1
  name: grp1-gwy1
spec:
  ports:
  # should match what was configured on the gateway in EEM Manager
  - port: 8443
    protocol: TCP
    targetPort: 8443
  selector:
    app: grp1-gwy1
    gatewayGroup: grp1
    gatewayId: gw1

For more information about creating Kubernetes services, see How to expose a TCP application with a Kubernetes Service.

Creating a Red Hat OpenShift Route

On Red Hat OpenShift, create the same number of routes as the total number of brokers across all connected Kafka clusters. For example, if you have 3 clusters each with 3 brokers, then you must specify at least 9 routes to correspond to the addresses that you specified in KAFKA_ADVERTISED_LISTENER.

Set spec.to.name to the Kubernetes Service that you created in Creating the Kubernetes Service, and set spec.tls.termination to passthrough.

In the following example, 3 separate routes are defined that all point to the same Kubernetes Service:

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: gw1-route1
spec:
  # host must match the dnsNames entry in the gateway certificate
  host: grp1-gwy1-1.mydomain.com
  port:
    targetPort: 8443
  tls:
    insecureEdgeTerminationPolicy: None
    termination: passthrough
  to:
    kind: Service
    name: grp1-gw1
    weight: 100
  wildcardPolicy: None
---
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: gw1-route2
spec:
  # host must match the dnsNames entry in the gateway certificate
  host: grp1-gwy1-2.mydomain.com
  port:
    targetPort: 8443
  tls:
    insecureEdgeTerminationPolicy: None
    termination: passthrough
  to:
    kind: Service
    name: grp1-gw1
    weight: 100
  wildcardPolicy: None
---
apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: gw1-route3
spec:
  # host must match the dnsNames entry in the gateway certificate
  host: grp1-gwy1-3.mydomain.com
  port:
    targetPort: 8443
  tls:
    insecureEdgeTerminationPolicy: None
    termination: passthrough
  to:
    kind: Service
    name: grp1-gw1
    weight: 100
  wildcardPolicy: None

Creating a Kubernetes Ingress

Follow the Kubernetes documentation on creating ingresses.