Operator-managed Event Gateway in Evicted state

Symptoms

An operator-managed Event Gateway or Kubernetes Deployment Event Gateway reports an Evicted state due to running out of space.

Causes

Kubernetes pods use ephemeral storage for scratch space, caching and for logs. The Event Gateway is spewing out logs that are not being cleaned up in time, which causes the Event Gateway to reach the Evicted state.

Resolving the problem

This issue can be resolved by adding more ephemeral storage for the Event Gateway so that additional storage is available to clean up logs. You can add more storage by adding the ephemeral-storage property to the EventGateway custom resource or Kubernetes Deployment. For example:

 Copy Copy# excerpt from a Event Gateway CRD 
template:
  pod:
    spec:
      containers:
          name: egw
          resources:
            limits:
              ephemeral-storage: 500Mi

This updates the pod ephemeral storage limits for the Event Gateway, which prevents the Event Gateway instance from overloading the storage and being Evicted.