Configuring TLS

Event Endpoint Management uses TLS to secure all network communication between the Event Endpoint Management components and Kafka clients and clusters.

All Event Manager endpoints are secured by the <instance name>-ibm-eem-manager certificate. UI and admin API users see this certificate when they connect to the Event Manager. If you want to replace <instance name>-ibm-eem-manager with your own certificate, then you must supply this certificate during installation.

The Event Gateways secure their endpoint with the <gateway group>-<gateway name>-ibm-egw-cert certificate. Kafka client applications see this certificate when they connect to the Event Gateway. You configure this certificate when you install the Event Gateway.

The certificates used to secure mTLS controls are configured in the Event Endpoint Management UI. For more information about configuring mTLS control certificates, see Managing mTLS control certificates.

The certificates used to secure communication between the Event Endpoint Management and Kafka clusters are configured in the Event Endpoint Management UI. See adding the cluster.

Key points

  • Cert-manager is used to create and maintain the Event Manager and Event Gateway endpoint certificates. If you require the use of your own organization’s certificates, then you can override the default cert-manager certificates during installation.
  • Event Endpoint Management TLS certificates are stored in Kubernetes Secrets.
  • The Certificate objects that cert-manager creates contain pointers to a Kubernetes Secret (of the same name) that contains the actual X.509 certificate strings.
  • Cert-manager monitors and renews all the certificates that it creates.
  • You are responsible for monitoring and renewing any custom certificates that you provide.
  • Your certificates must be included in your back up. They are required for disaster recovery.

The following table lists the secrets that contain the TLS certificates that secure your Event Endpoint Management endpoints:

Secret name Description
<gateway group>-<gateway name>-ibm-egw-cert Server certificate that secures the Event Gateway endpoint (the certificate that Kafka clients see when they contact the gateway).
<gateway group>-<gateway name>-certs CA certificate for the Event Gateway endpoint. Signs the <gateway group>-<gateway name>-ibm-egw-cert.
<instance name>-ibm-eem-manager Server certificate of the Event Manager (the certificate that UI and admin API users see when they connect to the Event Manager).
<instance name>-ibm-eem-manager-ca CA certificate for <instance name>-ibm-eem-manager.

Default Event Manager TLS configuration

If you do not specify any custom certificates when you install your Event Manager instance, then the Event Endpoint Management operator uses cert-manager to generate a CA certificate with a self-signed issuer. The operator uses this self-signed CA certificate to generate the leaf certificate that secures your Event Manager endpoints. Cert-manager stores the CA certificate in a secret called <my-instance>-ibm-eem-manager-ca.

Cert-manager and the Event Endpoint Management operator create the following objects in the Event Manager namespace:

  • Issuers:

    • <my-instance>-ibm-eem-manager
    • <my-instance>-ibm-eem-manager-selfsigned

  • Certificates:

    • <my-instance>-ibm-eem-manager-ca
    • <my-instance>-ibm-eem-manager

  • Secrets:

    • <my-instance>-ibm-eem-manager-ca
    • <my-instance>-ibm-eem-manager

Certificate customization options

If you want to customize your Event Manager certificates, you have the following options:

  1. Customize only the UI endpoint certificate. This is the certificate that Event Endpoint Management UI user’s browser sees when it connects to the UI. If the user’s browser does not trust this certificate, then the browser presents a warning to the user.
  2. Customize the Event Manager CA certificate. The Event Endpoint Management operator uses the custom CA certificate to sign the endpoint server certificate that it creates.
  3. Customize the Event Manager endpoint server certificate.

These options are mutually exclusive.

Custom UI certificate

A separate custom certificate can be configured for the UI endpoint. This certificate is presented to the browser when a user accesses the Event Endpoint Management UI. To supply a custom certificate to the UI, follow these steps:

  1. Ensure that you have the certificate files tls.crt, ca.crt, and tls.key in PEM format, and that the Subject Alternative Name (SAN) in tls.crt includes your UI endpoint.

  2. Create a Kubernetes secret from your certificate files:

    kubectl -n <namespace> create secret generic custom-ui-cert-secret --from-file=ca.crt=ca.crt --from-file=tls.crt=tls.crt --from-file=tls.key=tls.key
  3. Set spec.manager.tls.ui.secretName to custom-ui-cert-secret.
  4. Set spec.manager.tls.trustedCertificates to include the custom-ui-cert-secret CA certificate.
  5. If you are updating an existing Event Endpoint Management instance, then confirm that the manager pod restarts.

Example custom UI certificate configuration:

apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  manager:
    tls:
      ui:
        secretName: custom-ui-cert-secret
      trustedCertificates:
        - secretName: custom-ui-cert-secret
          certificate: ca.crt
# ...

Note: On Red Hat OpenShift Container Platform, if you already have a Kubernetes secret that you want to use for your UI endpoint, and it does not use the property names ca.crt, tls.crt, and tls.key to store the certificate strings (for example: data.ca_cert.crt instead of data.ca.crt), then complete the following additional steps:

  1. Edit your Event Manager custom resource.

  2. Specify the property names that are used in your UI endpoint secret as shown:

    apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  manager:
    tls:
      ui:
        # ...
        caCertificate: "ca_cert.crt"
        serverCertificate: "server_cert.crt"
        key: "private_key.key"
# ...

Custom CA certificate for Event Manager

You can provide your own custom CA certificate to the Event Manager instance. Cert-manager then uses the custom CA certificate to generate the server certificate that secures your Event Manager endpoints.

To configure Event Manager to use your custom CA certificate, complete the following steps:

  1. Ensure that you have the ca.crt and tls.key files for your CA certificate in PEM format.

  2. Create a Kubernetes secret called custom-ca-secret that contains your CA certificate files:

    kubectl -n <namespace> create secret generic custom-ca-secret --from-file=ca.crt=ca.crt --from-file=tls.crt=ca.crt --from-file=tls.key=tls.key
  3. Edit your Event Manager custom resource and set spec.manager.tls.caSecretName to custom-ca-secret.

  4. If you are updating an existing Event Endpoint Management instance, then follow these extra steps:

    a. Regenerate the endpoint server certificate so that it is re-created as signed-by custom-ca-secret:

    kubectl -n <namespace> delete secret <instance name>-ibm-eem-manager

    b. Restart the Event Manager pod:

    kubectl -n <namespace> delete <manager pod name>

    c. If your Event Manager instance has any registered Event Gateways, then update the Event Gateway with the new Event Manager CA certificate. The Event Manager CA certificate is stored in the gateway’s BACKEND_CA_CERTIFICATES environment variable. Where BACKEND_CA_CERTIFICATES is set depends on your gateway deployment type:

    • Operator-managed gateway: spec.template.pod.spec.containers.env[BACKEND_CA_CERTIFICATES].
    • Kubernetes Deployment gateway: spec.template.spec.containers.env[BACKEND_CA_CERTIFICATES].
    • Docker gateway: -e BACKEND_CA_CERTIFICATES.

Custom server certificates for Event Manager

If you want to supply your own server certificate for all your Event Manager endpoints, then you can create a Kubernetes secret that contains your TLS certificate strings and configure your Event Manager to use this secret.

Subject Alternative Name (SAN) requirements

Subject Alternative Names (SANs) are extensions to X.509 digital certificates that allow multiple domain names, IP addresses, or other identifiers to be associated with a single TLS certificate.

The certificate that secures your Event Manager must include your endpoints as SAN entries.

On OpenShift Container Platform, your SANs must include the following entries:

<instance name>-ibm-eem-manager
<instance name>-ibm-eem-manager.<namespace>
<instance name>-ibm-eem-manager.<namespace>.svc
<instance name>-ibm-eem-manager.<namespace>.svc.cluster.local
<instance name>-ibm-eem-apic-<namespace>.<cluster api>
<instance name>-ibm-eem-gateway-<namespace>.<cluster api>
<instance name>-ibm-eem-manager-<namespace>.<cluster api>
<instance name>-ibm-eem-admin-<namespace>.<cluster api>
eem.<instance name>-ibm-eem-server-<namespace>.<cluster api>

Where:

  • <instance name> is the name of the Event Manager instance.
  • <namespace> is the namespace of your Event Manager instance.
  • <cluster api> is derived from the URL of your OpenShift Container Platform cluster. If the URL is https://console-openshift-console.apps.clusterapi.com/ then <cluster api> is apps.clusterapi.com.

On other Kubernetes platforms, your SANs must include the service endpoints that you define during installation.

Additional points to consider:

  • Include any additional routes or ingresses to your Event Manager that you create.
  • If you plan to integrate with IBM API Connect, then include the apic endpoint hostnames.
  • Wildcard SAN entries are supported.
  • If you use LOCAL authentication, then include a SAN entry for the local cluster network.

Example SAN entries for OpenShift Container Platform

When you use LOCAL authentication with the Event Manager, the required SAN entries are as follows:

spec:
  dnsNames:
    - '*.<NAMESPACE>.svc.cluster.local'
    - '*.<CLUSTER_API>'
    - eem.*.<CLUSTER_API>'

When you use OIDC authentication with the Event Manager, the required SAN entries are as follows:

spec:
  dnsNames:
    - '*.<CLUSTER_API>'
    - eem.*.<CLUSTER_API>'

The required SAN entries for the Event Gateway are as follows:

spec:
  dnsNames:
    - '*.<CLUSTER_API>'

Configuring a custom Event Manager certificate

To configure a custom Event Manager server certificate, follow these steps:

  1. Ensure that you have the ca.crt, tls.crt, and tls.key files for the certificate in PEM format. Your tls.crt must comply with the SAN requirements.

  2. Create a Kubernetes secret that contains your TLS certificate strings by running the following command:

    kubectl -n <namespace> create secret generic custom-cert-secret --from-file=ca.crt=ca.crt --from-file=tls.crt=tls.crt --from-file=tls.key=tls.key

  3. Edit your Event Manager custom resource and set spec.manager.tls.secretName to custom-cert-secret. If you are updating an existing deployment, then the Event Manager pod restarts.

  4. If your Event Manager instance has any registered Event Gateways, then update the Event Gateway with the new Event Manager CA certificate. The Event Manager CA certificate is stored in the gateway’s BACKEND_CA_CERTIFICATES environment variable. Where BACKEND_CA_CERTIFICATES is set depends on your gateway deployment type:

    • Operator-managed gateway: spec.template.pod.spec.containers.env[BACKEND_CA_CERTIFICATES].
    • Kubernetes Deployment gateway: spec.template.spec.containers.env[BACKEND_CA_CERTIFICATES].
    • Docker gateway: -e BACKEND_CA_CERTIFICATES.

Renewing certificates

The cert-manager automatically renews the certificates that it manages when they are approaching their expiration. Any custom certificates that you create you must monitor and renew manually.

Example OpenSSL commands to create TLS certificate files

The following examples show how to use OpenSSL to generate TLS certificates that you can use in demonstration and test deployments of Event Endpoint Management.

Creating a CA certificate and key

You can create a CA certificate and key with the following OpenSSL commands:

openssl genrsa -out tls.key 2048
openssl req -new -x509 -key tls.key -days 730 -out ca.crt

Creating a server certificate

The following example uses OpenSSL and gettext to generate a CA and server certificate for Event Manager on Red Hat OpenShift with LOCAL user authentication.

  1. Set the following environment variables:

    EMAIL=<email-address>
MANAGER_NAME=<name-of-the-event-manager-instance>
CLUSTER_API=<cluster-api>
NAMESPACE=<event-endpoint-management-installation-namespace>

    Where:

    • MANAGER_NAME is the name of the Event Manager instance.
    • CLUSTER_API is the cluster URL that can be obtained from the cluster. If the URL is https://console-openshift-console.apps.clusterapi.com/ then the CLUSTER_API must be set to apps.clusterapi.com.

  2. Create a file called csr_ca.txt and paste in the following contents:

    [req]
prompt = no
default_bits = 4096
default_md = sha256
distinguished_name = dn
x509_extensions = usr_cert

[dn]
C=US
ST=New York
L=New York
O=MyOrg
OU=MyOU
emailAddress=me@working.me
CN = server.example.com

[usr_cert]
basicConstraints=CA:TRUE
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer

  3. Create a file called my-eem-manager_answer.txt and paste in the following contents:

    [req]
default_bits = 4096
prompt = no
default_md = sha256
x509_extensions = req_ext
req_extensions = req_ext
distinguished_name = dn

[dn]
C=US
ST=New York
L=New York
O=MyOrg
OU=MyOrgUnit
emailAddress=${EMAIL}
CN = ${MANAGER_NAME}-ibm-eem-manager

[req_ext]
subjectAltName = @alt_names

[alt_names]
DNS.1 = ${MANAGER_NAME}-ibm-eem-manager
DNS.2 = ${MANAGER_NAME}-ibm-eem-manager.${NAMESPACE}
DNS.3 = ${MANAGER_NAME}-ibm-eem-manager.${NAMESPACE}.svc
DNS.4 = ${MANAGER_NAME}-ibm-eem-manager.${NAMESPACE}.svc.cluster.local
DNS.5 = ${MANAGER_NAME}-ibm-eem-apic-${NAMESPACE}.${CLUSTER_API}
DNS.6 = ${MANAGER_NAME}-ibm-eem-gateway-${NAMESPACE}.${CLUSTER_API}
DNS.7 = ${MANAGER_NAME}-ibm-eem-manager-${NAMESPACE}.${CLUSTER_API}
DNS.8 = ${MANAGER_NAME}-ibm-eem-admin-${NAMESPACE}.${CLUSTER_API}
DNS.9 = eem.${MANAGER_NAME}-ibm-eem-server-${NAMESPACE}.${CLUSTER_API}

  4. Generate the required certificates by running the following commands:

    a. Generate ca.key:

    openssl genrsa -out ca.key 4096

    b. Generate ca.crt:

    openssl req -new -x509 -key ca.key -days 730 -out ca.crt -config <( envsubst < csr_ca.txt )

    Note: The envsubst utility is available on Linux and can be installed by default as part of the gettext package.

    c. Generate manager.key:

    openssl genrsa -out ${MANAGER_NAME}.key 4096

    d. Generate manager.csr:

    openssl req -new -key ${MANAGER_NAME}.key -out ${MANAGER_NAME}.csr -config <(envsubst < my-eem-manager_answer.txt )

  5. Sign the csr to create the manager certificate:

    openssl x509 -req -in ${MANAGER_NAME}.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out ${MANAGER_NAME}.crt -days 730 -extensions 'req_ext' -extfile <(envsubst < my-eem-manager_answer.txt)

  6. Verify the manager certificate:

    openssl verify -CAfile ca.crt ${MANAGER_NAME}.crt