Error while rotating encryption key

Symptoms

You rotated the encryption key and the Event Endpoint Management Manager pod fails because the key was formatted incorrectly. You delete the Event Endpoint Management Manager pod to resolve the issue. However, when you restart the pod, it goes into a CrashLoopBackOff state, and the following error is displayed in the logs:

ERROR Events Gateway - [lambda$start$15:270] An error occurred starting the gateway verticle Bootstrap.
io.vertx.core.impl.NoStackTraceThrowable: Too many data encryption keys found, unable to start storage provider

Causes

While rotating the encryption key, the manager creates a backup of the current encryption key before the rotation process begins. This backup is not cleared correctly from storage, so the manager has 2 keys instead of 1.

Resolving the problem

To resolve the problem, delete the backup key from the underlying file system of the manager pod. The file for the backup key ends with -dek.enc.bak and is stored in the root directory of the Event Endpoint Management Manager file system.