Backing up and restoring

To back up and restore your Event Manager and Event Gateway instances, you can use a backup tool such as Velero. In addition, consider using a storage class that supports the Container Storage Interface (CSI) snapshotting (for example, the Ceph File System).

If you are running on Red Hat OpenShift Container Platform, the OADP operator uses Velero and simplifies the installation of the backup software on your cluster, and the management of your backups and restorations.

On other Kubernetes platforms, Velero provides a Helm chart that you can use to install the software or a command-line tool.

Follow these instructions to back up and restore your Event Endpoint Management and Event Gateway instances.

Before you begin

To back up and restore the Event Manager and Event Gateway instances, you must back up the following resources:

The main encryption key that is stored in a Kubernetes secret.
The data that is added to the instance. This is stored on Kubernetes Persistent Volumes (PVs) and within Kubernetes Persistent Volume Claims (PVCs).
Secrets containing certificates that are associated with the related instances.
The Event Manager and Event Gateway custom resource configurations.

This means that your backup storage location and configuration must be able to store both Kubernetes objects and volumes. For more information about the solutions available to back up PVs and PVCs, see the OpenShift Container Platform or Velero documentation. For example, you can use a remote object store such as AWS S3 and a CSI compliant storage class to create the PVC for your instance.

In your CSI-supported storage provider (such as Ceph), ensure you have the VolumeSnapshotClass configured.

If you are on the OpenShift Container Platform and use the OADP operator to back up your instances, you must specify backup and snapshot configurations in the DataProtectionApplication custom resource. For more information about installing the OADP operator, see the installing OADP documentation. You can also find more links about configuring the DataProtectionApplication custom resource for different backup locations in the installing OADP documentation. An example DataProtectionApplication configured against a quickstart minio instance would look as follows:

apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: velero-minio
  namespace: openshift-adp
spec:
  backupLocations:
    - velero:
        config:
          profile: default
          region: minio
          s3ForcePathStyle: 'true'
          s3Url: 'http://minio.velero.svc:9000'
        credential:
          key: minio
          name: minio-credentials
        default: true
        objectStorage:
          bucket: velero
          prefix: velero
        provider: aws
  configuration:
    nodeAgent:
      uploaderType: restic
    velero:
      defaultPlugins:
        - openshift
        - aws
        - kubevirt
        - csi

Optionally, to ensure that the correct items are backed up and restored properly, you can install the Velero client.

Backing up

Note the different labels that are used to identify resources to back up depending on your version of Event Endpoint Management.

For Event Endpoint Management 11.2.1 and later, the label is:
```
backup.events.ibm.com/component: eventendpointmanagement
```
For Event Endpoint Management 11.2.0, the label is:
```
events.ibm.com/backup: required
```

The code samples that are provided in the following steps are designed to work with any version.

When you have configured the Velero instance, complete the following steps to create a backup of your Event Endpoint Management and Event Gateway instances.

Add the appropriate back up label, for your version of Event Endpoint Management, to the Event Manager and Event Gateway custom resource. The Event Endpoint Management operator adds the label to the other Kubernetes resources required for back up and restore.

If the label is added to an existing instance of Event Manager or Event Gateway, the labels are treated as metadata only and are not added to other Kubernetes resources. To ensure that the label is added to other Kubernetes resources, roll the operator pod after the label change, or change any parameter in the .spec section of the custom resource when you change the label. To validate this process has worked correctly, use an example resource such as the Kubernetes secret <eem_instance_name>-ibm-eem-mek-bak, with the backup label present on this secret.
Add the back up label to the PersistentVolumeClaim that is binding the Event Manager instance to the PersistentVolume.

Create and apply a Backup custom resource by using the following snippet template.

In spec.includedNamespaces, add all the namespaces where the Event Manager and Event Gateway instances are installed. If you are using Event Endpoint Management 11.2.0 or earlier, and you have multiple instances of Event Endpoint Management or other components of Event Automation in the same namespace, you might want to consider adding another label to the backup resources so that you can uniquely identify them.

 apiVersion: velero.io/v1
 kind: Backup
 metadata:
   name: <unique_name>
   namespace: openshift-adp
 spec:
   storageLocation: <backup_storage_location>
   includeClusterResources: true
   snapshotVolumes: true
   orLabelSelectors:
     - matchExpressions:
         - key: backup.events.ibm.com/component
           operator: In
           values:
             - catalogsource
             - operatorgroup
             - subscription
             - eventendpointmanagement
             - eventgateway
     - matchExpressions:
         - key: events.ibm.com/backup
           operator: In
           values:
             - required
   includedNamespaces:
     - <event-manager-instance-namespace>
     - <event-gateway-instance-namespace>

After the Backup custom resource is applied, check whether the custom resource is updated with status information similar to the following snippet:

status:
  progress:
    itemsBackedUp: 12
    totalItems: 12
  csiVolumeSnapshotsCompleted: 1
  csiVolumeSnapshotsAttempted: 1
  phase: Completed

Optional: Run the following command in the Velero CLI to ensure that the resources that are specified in the Backup custom resource are backed up properly:
```
velero backup describe <backup_name> --details -n openshift-adp
```

If you have problems creating a backup, see the troubleshooting information for OADP.

Restoring

Restoring your instance might be required for a number of reasons:

Loss of the entire Kubernetes cluster.
Loss of a namespace.
Loss of the Event Manager data or the encryption key.
To wind back the data to a previous state.
Loss of the Event Manager and Event Gateway instances.

Before you restore your instance, the Event Endpoint Management operator must be installed.

Note: You must restore your instance into a namespace with the same name that the backup was taken from. Ensure that you do not have an instance with the same name as the one you are restoring. You cannot restore Event Manager and Event Gateway instances in the namespace if another instance with the same name exists there. If you are trying to rewind your data, you must uninstall the Event Manager and Event Gateway instances that you are restoring.

To restore your Event Manager and Event Gateway instances, complete the following steps:

Create and apply a Restore custom resource by using the following snippet.

In spec.includedNamespaces, add all the namespaces where Event Manager and Event Gateway instances were previously installed, these namespaces should exist before the restore.

apiVersion: velero.io/v1
kind: Restore
metadata:
  name: <unique_name>
  namespace: openshift-adp
spec:
  backupName: <name_of_the_backup_to_restore>
  existingResourcePolicy: update
  restorePVs: true
  orLabelSelectors:
  - matchExpressions:
    - key: backup.events.ibm.com/component
      operator: In
      values:
      - catalogsource
      - operatorgroup
      - subscription
      - eventendpointmanagement
      - eventgateway
  - matchExpressions:
    - key: events.ibm.com/backup
      operator: In
      values:
        - required
  includedNamespaces:
    - <event-manager-instance-namespace>
    - <event-gateway-instance-namespace>

After the Restore custom resource is applied and in Completed state, check that both the Event Manager and Event Gateway instances are created, and are in Running state.
Secure your access again by following guidance in the managing access and managing roles topics.

You can access the Event Manager and Event Gateway instances again with the same configuration and data that you used when you backed up the data.

Handling multiple instances

In Event Endpoint Management 11.2.1 and later, you can customize the label backup.events.ibm.com/component: eventendpointmanagement to differentiate between different instances of Event Endpoint Management that might exist in the same namespace.

To customize the label, complete the following steps:

Change the eventendpointmanagement value to any custom value. For example, my-eem.
Ensure that the custom value is listed in the list of values in the backup and restore custom resources.