Migrating mTLS users

Migrate mutual TLS (mTLS) authenticated Kafka users from Event Streams to Confluent for Kubernetes (CFK) by completing the following steps:

• Review certificate model differences: Understand how Event Streams and CFK handle mTLS certificates differently.
• Complete prerequisites: Confirm that the setup phase is complete and that the CFK external listener is configured for mTLS.
• Set up certificate authorities: Set up the CAs used to sign user certificates.
• Create user certificates: Issue one client certificate per mTLS user being migrated.
• Connect client applications to CFK: Update each client application’s truststore and keystore to use the new certificates.
• Verify the migration: Confirm that client applications are authenticating correctly.

Certificate model differences

Before you begin, review how Event Streams and CFK handle mTLS certificates differently.

Event Streams certificate model

Event Streams manages two distinct certificate authorities:

The following table describes the certificate authority types in Event Streams:

Each KafkaUser with authentication.type: tls receives a dedicated client certificate signed by the Clients CA, stored in a secret named after the user (for example, admin), containing user.crt, user.key, and user.p12.

CFK certificate model

The following table describes the unified certificate model in CFK:

Note: Unlike Event Streams, where each broker has an individually named certificate, CFK uses one certificate per listener. That certificate uses wildcard SANs (for example, *.<namespace>.svc.cluster.local) to cover all broker pods, so no per-pod certificate is needed.

Prerequisites

Ensure that the following prerequisites are met before you begin:

• Complete the preparation and setup phases.
• Configure the CFK external listener for mTLS, with broker certificates issued by your chosen PKI tooling (see setting up Confluent Platform).

• Obtain a list of all mTLS KafkaUser resources to migrate. Run the following command to list all mTLS users on your Event Streams cluster:

  kubectl get kafkausers.eventstreams.ibm.com -n <eventstreams-namespace> -o json \
    | jq -r '.items[] | select(.spec.authentication.type == "tls") | .metadata.name'
  

  Where:

  • <eventstreams-namespace> is the namespace where Event Streams is installed.

Principal mapping

Configure principal mapping correctly to ensure that Kafka principals and ACLs are preserved after migration.

In Event Streams, the KafkaUser resource name becomes the Kafka principal (for example, a KafkaUser named admin has principal User:CN=admin). Event Streams sets the certificate CN to match the user name.

Note: In Event Streams, mTLS user principals include the CN= prefix in the format User:CN=<kafka-user-name>. ACLs that reference this format must be transformed before Confluent Platform can evaluate them correctly. For more information, see the principal transform step in migrating users.

CFK applies a principalMappingRules regular expression to the client certificate distinguished name. The rule is as follows:

RULE:.*CN[\s]?=[\s]?([a-zA-Z0-9.-]*)?.*/$1/

This extracts the CN value and uses it as the Kafka principal, producing User:admin rather than User:CN=admin.

You must set the commonName in each new client certificate to exactly match the original Event Streams KafkaUser name (for example, admin), regardless of which PKI tooling you use to issue it. This preserves the Kafka principal (User:admin) so that any existing ACLs remain valid after migration.

Migrating mTLS users to CFK

Complete the following steps to migrate each mTLS user from Event Streams to CFK:

Step 1: Setting up certificate authorities

Before you create user certificates, set up the certificate authorities you use to sign them. This migration uses two CAs: a cluster CA for broker certificates and a client CA for client certificates. For instructions about setting up your CAs with your chosen PKI tooling, see setting up certificates.

These steps assume two CAs are in place in your Confluent Platform namespace: a Cluster CA for signing broker certificates and a Client CA for signing client (user) certificates and the external listener certificate. The names cluster-ca-issuer and client-ca-issuer refer to those CAs, regardless of the tooling used to set them up.

If your Event Streams deployment used the brokerCertChainAndKey listener certificate override, build the client-tls secret from your existing PKI broker certificate before creating user certificates. For instructions, see migrating listener certificate override.

Step 2: Creating user certificates

Issue one client certificate per mTLS user being migrated by using your chosen PKI tooling. Each certificate must satisfy the following requirements:

• Common name (CN) must exactly match the original Event Streams KafkaUser name (for example, admin). This is what the principalMappingRules regular expression extracts to derive the Kafka principal. If it does not match, the user ACLs do not apply.
• Key usage must include client auth.
• Signing CA must be your Client CA. The broker trusts the Client CA on the external listener, so certificates signed by any other CA are rejected.

Place the resulting certificate and key, along with the Client CA certificate, in a Kubernetes secret with the following fields:

• tls.crt: the client certificate
• tls.key: the client private key
• ca.crt: the Client CA certificate (the CA that signed tls.crt)

The ca.crt field is required because step 3 extracts it from this secret to use as the truststore when connecting to CFK. If you are not using cert-manager, you must include the ca.crt field explicitly when creating the secret. cert-manager populates the ca.crt field automatically, but other PKI tooling does not.

Note the secret name so that you can reference it when configuring the client application. Repeat for each mTLS user being migrated.

Step 3: Connecting client applications to CFK

After you create the user certificates in step 2, update the truststore (the CA used to verify the broker) and keystore (the client certificate presented to the broker) for each client application. In Event Streams, the truststore contained the Cluster CA and the keystore contained the Event Streams-issued user.crt and user.key. In CFK, the truststore must contain the Client CA and the keystore must contain the new client certificate and key issued in step 2.

Complete the following steps to configure each client application to connect to CFK:

1. Extract the certificate and key from the Kubernetes secret created in step 2. The following example assumes PEM format (tls.crt, tls.key, ca.crt). Adjust the field names if you used a different format. Run the following commands:

   kubectl get secret <user-cert-secret> -n <cp-namespace> \
     -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt
   
   kubectl get secret <user-cert-secret> -n <cp-namespace> \
     -o jsonpath='{.data.tls\.crt}' | base64 -d > client.crt
   
   kubectl get secret <user-cert-secret> -n <cp-namespace> \
     -o jsonpath='{.data.tls\.key}' | base64 -d > client.key
   
   # Convert to PKCS8 (required for Java clients)
   openssl pkcs8 -topk8 -nocrypt -in client.key -out client.key.pem
   
   # Combine cert and key into a single PEM (required when using ssl.keystore.location with ssl.keystore.type=PEM)
   cat client.crt client.key.pem > client-combined.pem
   

   Where:

   • <user-cert-secret> is the name of the secret you set in the Certificate resource for this user.
   • <cp-namespace> is the Kubernetes namespace where you deploy Confluent Platform.

2. Configure the Kafka client with the following properties to connect to CFK by using mTLS:

   security.protocol=SSL
   ssl.protocol=TLSv1.2
   
   ssl.truststore.type=PEM
   ssl.truststore.location=/path/to/ca.crt
   
   ssl.keystore.type=PEM
   ssl.keystore.location=/path/to/client-combined.pem
   

After you complete these steps, verify the migration to confirm that the application connects successfully and that the correct Kafka principal is mapped.

Note: Keep both connections running for a period to confirm the new connection is stable before switching over fully. To roll back, revert the client configuration to the original bootstrap address and the Event Streams-issued certificate.

Verifying the migration

Use the following checks to confirm that the migration is complete and that client applications authenticate correctly.

Verifying the broker certificate chain

Run the following command to confirm that the broker presents a certificate signed by the expected CA:

openssl s_client -connect <bootstrap-address>:<port> \
  -showcerts </dev/null 2>/dev/null \
  | openssl x509 -noout -issuer -subject

Where:

• <bootstrap-address> is the bootstrap address of your Confluent cluster.
• <port> is the port number (for example, 9092).

Verifying the client certificate common name

Run the following command to confirm that the common name in the certificate matches the original KafkaUser name:

openssl x509 -noout -subject -in client.crt
# Expected output contains: CN=admin
# The CN value must match the original KafkaUser name. 

Verifying the broker trusts the client CA

Run the following command to confirm that the broker accepts the client certificate during the TLS handshake:

openssl s_client \
  -connect <bootstrap-address>:<port> \
  -cert client.crt \
  -key client.key \
  -CAfile ca.crt

A successful TLS handshake confirms that the broker trusts the client CA.

Verifying the Kafka principal mapping

Run the following command to check the broker logs for the authenticated principal after a connection attempt:

kubectl logs -n <cp-namespace> -l app=<kafka-cr-name> --tail=50 \
  | grep "Principal\|authentication\|ANONYMOUS"

Where:

• <cp-namespace> is the Kubernetes namespace where you deploy Confluent Platform.
• <kafka-cr-name> is the name of your Kafka custom resource (for example, kafka).