Configuring

Setting environment variables

You can configure Event Endpoint Management or Event Gateway by setting environment variables. This is done by providing a template override(env) which specifies one or more name-value pairs.

The format for Event Endpoint Management instances is:

spec:
  manager:
    template:
      pod:
        spec:
          containers:
            - name: manager
              env:
                - name: <name>
                  value: <value>

The format for Event Gateway instances is:

spec:
  template:
    pod:
      spec:
        containers:
          - name: egw
            env:
              - name: <name>
                value: <value>

Where:

• <name> is the specification that you want to configure.
• <value> is the value to configure the specification.

For example, to enable trace logging in the Event Manager:

spec:
  manager:
    template:
      pod:
        spec:
          containers:
            - name: manager
              env:
                - name: TRACE_SPEC
                  value: "<package>:<trace level>"

Enabling persistent storage

To persist the data input into an Event Endpoint Management instance, configure persistent storage in your EventEndpointManagement configuration.

To enable persistent storage for EventEndpointManagement , set spec.manager.storage.type to persistent-claim, and then configure the storage in one of the following ways:

• dynamic provisioning
• providing persistent volume
• providing persistent volume and persistent volume claim.

Ensure that you have sufficient disk space for persistent storage.

Note: spec.manager.storage.type can also be set to ephemeral, although no persistence is provisioned with this configuration. This is not recommended for production usage because it results in lost data.

Dynamic provisioning

If there is a dynamic storage provisioner present on the system, you can use the dynamic storage provisioner to dynamically provision the persistence for Event Endpoint Management. To configure this, set spec.manager.storage.storageClassName to the name of the storage class provided by the provisioner.

apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  manager:
    storage:
      type: persistent-claim
      storageClassName: csi-cephfs
# ...

• Optionally, specify the storage size in storage.size (for example, the default value used would be "100Mi"). Ensure that the quantity suffix, such as Mi or Gi, is included.
• Optionally, specify the root storage path where data is stored in storage.root (for example, "/opt/storage").
• Optionally, specify the retention setting for the storage if the instance is deleted in storage.deleteClaim (for example, "true").

Providing persistent volumes

Before installing Event Endpoint Management, you can create a persistent volume for it to use as storage. To use a persistent volume that you created earlier, set the spec.manager.storage.selectors to match the labels on the persistent volume and set the spec.manager.storage.storageClassName to match the storageClassName on the persistent volume. The following example creates a persistent volume claim to bind to a persistent volume with the label precreated-persistence: my-pv and storageClassName: manual. Multiple labels can be added as selectors, and the persistent volume must have all labels present to match.

apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  manager:
    storage:
      type: persistent-claim
      selectors:
        precreated-persistence: my-pv
      storageClassName: manual
# ...

• Optionally, specify the storage size in storage.size (for example, the default value used would be "100Mi"). Ensure that the quantity suffix, such as Mi or Gi, is included.
• Optionally, specify the root storage path where data is stored in storage.root (for example, "/opt/storage").
• Optionally, specify the retention setting for the storage if the instance is deleted in storage.deleteClaim (for example, "true").

Providing persistent volume and persistent volume claim

A persistent volume and persistent volume claim can be pre-created for Event Endpoint Management to use as storage. To use this method, set spec.manager.storage.existingClaimName to match the name of the pre-created persistent volume claim.

apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  manager:
    storage:
      type: persistent-claim
      existingClaimName: my-existing-pvc
# ...

Configuring TLS

TLS can be configured for the EventEndpointManagement instance in one of the following ways:

• Operator configured CA certificate
• User provided CA certificate
• User provided certificates
• User provided UI certificates

After the TLS is configured for the EventEndpointManagement instance, the TLS for the EventGateway instance must be configured. TLS can be configured for the EventGateway instance in one of the following ways:

• Using CA certificate for EventGateway
• User provided certificate for EventGateway

Operator configured CA certificate

By default, the operator configures TLS if no value is provided for CA certificate when creating the instance. The operator uses the Cert Manager installed on the system to generate a CA certificate with a self-signed issuer. It then uses this self-signed CA certificate to sign the certificates used for secure communication by the Event Endpoint Management instance. Cert Manager puts the CA certificate into a secret named <my-instance>-ibm-eem-manager-ca. This secret can be used for configuring the EventGateway TLS communications.

Cert Manager and Event Endpoint Management will create the following objects:

• Cert Manager Issuers:

  • <my-instance>-ibm-eem-manager
  • <my-instance>-ibm-eem-manager-selfsigned

• Cert Manager Certificates:

  • <my-instance>-ibm-eem-manager-ca
  • <my-instance>-ibm-eem-manager

The following code snippet is an example of a configuration where all certificates are created by Cert Manager and Event Endpoint Management:

apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  manager:
# ...

User-provided CA certificate

You can provide a custom CA certificate to the Event Endpoint Management instance.

The operator uses the Cert Manager installed on the system to create the certificates used for secure communication by the Event Endpoint Management instance. The certificates are signed by using the provided CA certificate.

The CA secret that is created and referenced in the Cert Manager must contain the keys ca.crt, tls.crt, tls.key. The ca.crt key and the tls.crt key can have the same value.

See the following example to use the user provided certificate files (ca.crt, tls.crt, and tls.key):

1. Set a variable for the NAMESPACE by running the following command:

   export NAMESPACE=<instance namespace>
   

2. Create the CA secret by running the following command:

   kubectl create secret generic ca-secret-cert --from-file=ca.crt=ca.crt --from-file=tls.crt=tls.crt --from-file=tls.key=tls.key -n ${NAMESPACE}
   

3. To provide a custom CA certificate secret, set spec.manager.tls.caSecretName key to be the name of the CA certificate secret that contains the CA certificate.

The following code snippet is an example of a configuration that uses the CA certificate secret that is created in the previous steps:

apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  manager:
    tls:
      caSecretName: ca-secret-cert
# ...

User-provided certificates

You can use a custom certificate for secure communication by the Event Endpoint Management instance. You can use the OpenSSL tool to generate a CA and certificates that are required for an Event Endpoint Management instance.

Note: The envsubst utility is available on Linux and can be installed by default as part of the gettext package.

See the following example for setting up OpenSSL tool to generate a CA and certificates that are required for an Event Endpoint Management instance:

1. If you are using a MAC, the following packages are required and can be installed by using HomeBrew:

   • gettext
   • openssl@3

   brew install gettext openssl@3
   

   Then run alias openssl=$(brew --prefix)/opt/openssl@3/bin/openssl to use Openssl3.

2. Set the following variables on your workstation:

   EMAIL=<email-address>
   MANAGER_NAME=<name-of-the-event-endpoint-management-manager-instance>
   CLUSTER_API=<cluster-api>
   NAMESPACE=<event-endpoint-management-installation-namespace>
   

   Where:

   • MANAGER_NAME is the name of the Event Endpoint Management instance
   • CLUSTER_API is the cluster URL that can be obtained from the cluster. If the URL is https://console-openshift-console.apps.clusterapi.com/ then the CLUSTER_API must be set to apps.clusterapi.com.

3. Create a file called csr_ca.txt with the following data:

   [req]
   prompt = no
   default_bits = 4096
   default_md = sha256
   distinguished_name = dn
   x509_extensions = usr_cert
   
   [dn]
   C=US
   ST=New York
   L=New York
   O=MyOrg
   OU=MyOU
   emailAddress=me@working.me
   CN = server.example.com
   
   [usr_cert]
   basicConstraints=CA:TRUE
   subjectKeyIdentifier=hash
   authorityKeyIdentifier=keyid,issuer
   

4. Create a file called my-eem-manager_answer.txt with the following data:

   [req]
   default_bits = 4096
   prompt = no
   default_md = sha256
   x509_extensions = req_ext
   req_extensions = req_ext
   distinguished_name = dn
   
   [dn]
   C=US
   ST=New York
   L=New York
   O=MyOrg
   OU=MyOrgUnit
   emailAddress=${EMAIL}
   CN = ${MANAGER_NAME}-ibm-eem-manager
   
   [req_ext]
   subjectAltName = @alt_names
   
   [alt_names]
   DNS.1 = ${MANAGER_NAME}-ibm-eem-manager
   DNS.2 = ${MANAGER_NAME}-ibm-eem-manager.${NAMESPACE}
   DNS.3 = ${MANAGER_NAME}-ibm-eem-manager.${NAMESPACE}.svc
   DNS.4 = ${MANAGER_NAME}-ibm-eem-manager.${NAMESPACE}.svc.cluster.local
   DNS.5 = ${MANAGER_NAME}-ibm-eem-apic-${NAMESPACE}.${CLUSTER_API}
   DNS.6 = ${MANAGER_NAME}-ibm-eem-gateway-${NAMESPACE}.${CLUSTER_API}
   DNS.7 = ${MANAGER_NAME}-ibm-eem-manager-${NAMESPACE}.${CLUSTER_API}
   

   Important: If you are planning to do any of the following for your deployment, ensure you modify the [alt_names] section in the previous example to include the Event Manager ui, gateway, and, if required, the apic endpoint hostnames:

   • You are planning to specify hostnames in the eventendpointmanager custom resource under spec.manager.endpoints.
   • You are planning to create additional routes or ingress.
   • You are not running on OpenShift Container Platform.

5. Generate the required certificates by running the following commands:

   • ca.key:

     openssl genrsa -out ca.key 4096
     

   • ca.crt:

     openssl req -new -x509 -key ca.key -days 730 -out ca.crt -config <( envsubst <csr_ca.txt )
     

   • manager key:

     openssl genrsa -out ${MANAGER_NAME}.key 4096
     

   • manager csr:

     openssl req -new -key ${MANAGER_NAME}.key -out ${MANAGER_NAME}.csr -config <(envsubst < my-eem-manager_answer.txt )
     

6. Sign the csr to create the manager crt by running the following command:

   openssl x509 -req -in ${MANAGER_NAME}.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out ${MANAGER_NAME}.crt -days 730 -extensions 'req_ext' -extfile <(envsubst < my-eem-manager_answer.txt)
   

7. Verify the certificate by running the following command:

   openssl verify -CAfile ca.crt ${MANAGER_NAME}.crt
   

8. Create a secret on the cluster by running the following command:

   Note: The secret must be added to the namespace where the Event Endpoint Management instance is intended to be created in.

   kubectl create secret generic ${MANAGER_NAME}-cert --from-file=ca.crt=ca.crt --from-file=tls.crt=${MANAGER_NAME}.crt --from-file=tls.key=${MANAGER_NAME}.key -n ${NAMESPACE}
   

9. Create an Event Endpoint Management instance called ${MANAGER_NAME} in the same namespace where you generated the secret in step 8. Ensure the spec.manager.tls.secretName field is set to the name of the secret from step 8. For example:

   apiVersion: events.ibm.com/v1beta1
   kind: EventEndpointManagement
   metadata:
     name: my-eem
     namespace: eem
   spec:
     license:
       # ...
     manager:
       tls:
         secretName: my-eem-manager-cert
   # ...
   

User-provided UI certificates

A separate custom certificate can be used for the UI. This certificate is presented to the browser when the Event Endpoint Management user interface is navigated. To supply a custom certificate to the UI set spec.manager.tls.ui.secretName to be the name of the secret containing the certificate.

The following code snippet is an example of a configuration that uses a user-provided certificate:

apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  manager:
    tls:
      ui:
        secretName: myUiSecret
# ...

Optionally, if running on the Red Hat OpenShift Container Platform:

• Specify the key in the secret that is pointing to the CA certificate ui.caCertificate (default, ca.crt).
• Specify the key in the secret that is pointing to the server certificate ui.serverCertificate (default, tls.crt).
• Specify the key in the secret that is pointing to the private key ui.key (default, tls.key).

Using CA certificate for EventGateway

A CA certificate can be used to securely connect an EventGateway instance to an EventEndpointManagement instance. To use a CA certificate in the EventGateway configuration, set spec.tls.caSecretName to be the name of the secret that contains the CA certificate. The CA certificate that is provided is used to sign the leaf certificates that are used by the EventGateway instance for secure communication. The CA certificate that is provided for the EventGateway instance should be the same CA certificate that is provided when configuring the TLS for the EventEndpointManagement instance.

The following code snippet is an example of an EventGateway configuration that uses a user-provided CA certificate:

apiVersion: events.ibm.com/v1beta1
kind: EventGateway
# ...
spec:
  license:
    # ...
  tls:
    caSecretName: <my-instance>-ibm-eem-manager-ca
# ...

User-provided certificate for EventGateway

A custom certificate can be used for secure communication of the Event Gateway instance. This method does not use Cert Manager, so the certificates that are provided must be managed by the user.

The Event Gateway uses a client certificate to register itself with Event Manager. The Event Gateway client certificate contains the necessary authentication and authorization to pull information about topics and subscriptions from Event Manager. Ensure that this client certificate meets the following requirements:

• It must be issued by a CA that Event Manager trusts.
• It must have a subject alternative name (SAN) URI of the format: egw://<host>:<port>/<gwgroup>/<gwid>

Where:

• host is the host name that Kafka applications use to connect to the gateway.
• port is the port number that Kafka applications use to connect to the gateway.
• gwgroup is the name of the group that this gateway belongs to.
• gwid is the unique ID for this gateway.

The following snippet is an example of this SAN requirement:

  X509v3 extensions:
    X509v3 Subject Alternative Name: 
          URI:egw://eem-gateway-instance-ibm-egw-rt-mynamespace.apps.my.cluster.com:443/london/londongw1

You can use the OpenSSL tool to generate a CA and certificates that are required for an Event Gateway instance.

Note: The envsubst utility is available on Linux and can be installed by default as part of the gettext package.

See the following example for setting up OpenSSL tool to generate a CA and certificates that are required for an Event Gateway instance:

1. If you are using a MAC, the following packages are required and can be installed by using HomeBrew:

   • gettext
   • openssl@3

   brew install gettext openssl@3
   

   Then run alias openssl=$(brew --prefix)/opt/openssl@3/bin/openssl to use Openssl3.

2. Set the following variables on your workstation:

   EMAIL=<email-address>
   GATEWAY_NAME=<name-of-the-event-gateway-instance>
   GATEWAY_GROUP=<event-gateway-group>
   GATEWAY_ID=<event-gateway-id>
   CLUSTER_API=<cluster-api>
   NAMESPACE=<event-gateway-installation-namespace>
   

   Where:

   • GATEWAY_NAME is the name of the Event Gateway instance
   • GATEWAY_GROUP is the name of the gateway group associated with the Event Gateway instance
   • GATEWAY_ID is the name of the gateway id associated with the Event Gateway instance
   • CLUSTER_API is the cluster URL that can be obtained from the cluster. For example, if the cluster URL is https://console-openshift-console.apps.clusterapi.com/, then the CLUSTER_API must be set to apps.clusterapi.com.

3. Create a file called csr_ca.txt with the following data:

   [req]
   prompt = no
   default_bits = 4096
   default_md = sha256
   distinguished_name = dn
   x509_extensions = usr_cert
   [dn]
   C=US
   ST=New York
   L=New York
   O=MyOrg
   OU=MyOU
   emailAddress=me@working.me
   CN = server.example.com
   [usr_cert]
   basicConstraints=CA:TRUE
   subjectKeyIdentifier=hash
   authorityKeyIdentifier=keyid,issuer
   

4. Create a file called my-eem-gateway_answer.txt with the following data:

   [req]
   default_bits = 4096
   prompt = no
   default_md = sha256
   x509_extensions = req_ext
   req_extensions = req_ext
   distinguished_name = dn
   [dn]
   C=US
   ST=New York
   L=New York
   O=MyOrg
   OU=MyOrgUnit
   emailAddress=${EMAIL}
   CN = ${GATEWAY_NAME}-ibm-egw
   [req_ext]
   subjectAltName = @alt_names
   [alt_names]
   DNS.1 = ${GATEWAY_NAME}-ibm-egw-svc
   DNS.2 = ${GATEWAY_NAME}-ibm-egw-svc.${NAMESPACE}
   DNS.3 = ${GATEWAY_NAME}-ibm-egw-svc.${NAMESPACE}.svc
   DNS.4 = ${GATEWAY_NAME}-ibm-egw-svc.${NAMESPACE}.svc.cluster.local
   DNS.5 = ${GATEWAY_NAME}-ibm-egw-rt-${NAMESPACE}.${CLUSTER_API}
   URI.1 = egw://${GATEWAY_NAME}:443/${GATEWAY_GROUP}/${GATEWAY_ID}
   

   Important: If you are planning to do any of the following for your deployment, ensure you modify the [alt_names] section in the previous example to include the endpoint hostnames:

   • You are planning to specify hostnames in the EventGateway custom resource under spec.endpoints.
   • You are planning to create additional routes or ingress.
   • You are not running on OpenShift Container Platform.

5. Generate the required certificates by running the following commands:

   • ca.key:

     openssl genrsa -out ca.key 4096
     

   • ca.crt:

     openssl req -new -x509 -key ca.key -days 730 -out ca.crt -config <( envsubst <csr_ca.txt )
     

   • gateway key:

     openssl genrsa -out ${GATEWAY_NAME}.key 4096
     

   • gateway csr:

     openssl req -new -key ${GATEWAY_NAME}.key -out ${GATEWAY_NAME}.csr -config <(envsubst < my-eem-gateway_answer.txt )
     

6. Sign the csr to create the gateway crt by running the following command:

   openssl x509 -req -in ${GATEWAY_NAME}.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out ${GATEWAY_NAME}.crt -days 730 -extensions 'req_ext' -extfile <(envsubst < my-eem-gateway_answer.txt)
   

7. Verify the certificate by running the following command:

   openssl verify -CAfile ca.crt ${GATEWAY_NAME}.crt
   

8. Create a secret on the cluster by running the following command:

   Note: The secret must be added to the namespace where the Event Gateway instance is intended to be created in.

   kubectl create secret generic ${GATEWAY_NAME}-cert --from-file=ca.crt=ca.crt --from-file=tls.crt=${GATEWAY_NAME}.crt --from-file=tls.key=${GATEWAY_NAME}.key -n ${NAMESPACE}
   

9. Create an Event Gateway instance called ${GATEWAY_NAME} in the same namespace where you generated the secret in step 8 and set:

   • The spec.tls.secretName property to the name of the secret from step 8.
   • The spec.gatewayGroupName and spec.gatewayID to the value set on the URI SAN of the certificate in step 4.

   For example:

    apiVersion: events.ibm.com/v1beta1
    kind: EventGateway
    # ...
    spec:
      license:
        # ...
      gatewayGroupName: mygroup
      gatewayID: myid
      tls:
        secretName: mygw-cert
    # ...
   

   Optionally, if running on the Red Hat OpenShift Container Platform:

   • Specify the key in the secret that is pointing to the CA certificate tls.caCertificate (default, ca.crt).
   • Specify the key in the secret that is pointing to the server certificate tls.serverCertificate (default, tls.crt).
   • Specify the key in the secret that is pointing to the private key tls.key (default, tls.key).

Note: When a custom certificate is used for the EventEndpointManagement instance and the EventGateway instance, each EventEndpointManagement instance must trust the CA certificate of the EventGateway instance. If the certificates were not signed by the same Certificate Authority (CA), then set the following fields to ensure the instances trust each other:

• In the EventGateway custom resource, the tls.secretName secret must contain the EventEndpointManagement CA certificate on the ca.crt key.
• In the EventEndpointManagement custom resource, the tls.trustedCertificates must include a secret that contains the CA certificate used to sign the EventGateway.

The following code snippet is an example configuration that uses custom certificates for the EventEndpointManagement instance and the EventGateway instance. In the following code snippet:

• The EventEndpointManagement instance uses the myManagementCert secret and trustedCertificates secret is aSecretContainingTheGatewayCA
• The EventGateway instance uses the myGatewayCert secret and trusts the myManagementCert because the myGatewayCert has the ca.crt of the EventEndpointManagement instance added to it.

apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  manager:
    tls:
      secretName: myManagementCert
      caCertificate: ca.crt
      serverCertificate: tls.crt
      key: tls.key
      trustedCertificates:
        # The ca.crt is the EventGateway's CA certificcate
        - secretName: aSecretContainingTheGatewayCA
          certificate: ca.crt
# ...
---
apiVersion: events.ibm.com/v1beta1
kind: EventGateway
# ...
spec:
  license:
    # ...
  tls:
    # The ca.crt in the myGatewayCert is the EventEndpointManagement's CA Certificate
    secretName: myGatewayCert
    caCertificate: ca.crt  
    serverCertificate: tls.crt
    key: tls.key
# ...

Configuring authentication

Authentication is configured in the EventEndpointManagement configuration.

Two types of authentication are available: LOCAL and OIDC. For more information, see managing access.

Deploy network policies

By default, the operator deploys an instance-specific network policy when an instance of EventEndpointManagement or EventGateway is created. The deployment of these network policies can be turned off by setting the spec.deployNetworkPolicies to false.

The following code snippet is an example of a configuration that turns off the deployment of the network policy:

apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  deployNetworkPolicies: false
# ...
---
apiVersion: events.ibm.com/v1beta1
kind: EventGateway
# ...
spec:
  license:
    # ...
  deployNetworkPolicies: false  

Configuring ingress

If running on the Red Hat OpenShift Container Platform, routes are automatically configured to provide external access. Optional: You can set a host for each exposed route on the EventEndpointManagement and EventGateway instances by setting values under spec.manager.endpoints[] and spec.endpoints[], respectively.

If you are not running on the Red Hat OpenShift Container Platform, the Event Endpoint Management operator will create ingress resources to provide external access. No default hostnames will be assigned to the ingress resource, and you must set hostnames for each exposed endpoint on the EventEndpointManagement and EventGateway instance.

Note: The hostname is limited to 64 characters.

The following code snippet shows an example of configuring the host for the EventEndpointManagement ui and gateway endpoints:

apiVersion: events.ibm.com/v1beta1
kind: EventEndpointManagement
# ...
spec:
  license:
    # ...
  manager:
    endpoints:
      - name: ui
        host: my-eem-ui.mycompany.com
      - name: gateway
        host: my-eem-gw-registration.mycompany.com
#...

For the EventGateway, you can set the gateway endpoint host as shown in the following code snippet:

apiVersion: events.ibm.com/v1beta1
kind: EventGateway
# ...
spec:
  license:
    # ...
  endpoints:
    - name: gateway
      host: my-gateway.mycompany.com
# ... 

Ingress default settings

If you are not running on the Red Hat OpenShift Container Platform, the following ingress defaults are set unless overridden:

• class: The ingress class name is set by default to nginx. Set the class field on endpoints to use a different ingress class.

• annotations: The following annotations are set by default on generated ingress endpoints:

  ingress.kubernetes.io/ssl-passthrough: 'true'
  nginx.ingress.kubernetes.io/backend-protocol: HTTPS
  nginx.ingress.kubernetes.io/ssl-passthrough: 'true'

If you specify a spec.manager.tls.ui.secretName on an EventEndpointManagement instance, the following re-encrypt annotations will be set on the ui ingress. Other ingresses will be configured for passthrough.

    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/configuration-snippet: proxy_ssl_name "<HOSTNAME>";
    nginx.ingress.kubernetes.io/proxy-ssl-protocols: TLSv1.3
    nginx.ingress.kubernetes.io/proxy-ssl-secret: <NAMESPACE>/<SECRETNAME>
    nginx.ingress.kubernetes.io/proxy-ssl-verify: 'on'

Ingress annotations can be overridden by specifying an alternative set of annotations on an endpoint. The following code snippet is an example of overriding the annotations set on an EventGateway gateway endpoint ingress.

apiVersion: events.ibm.com/v1beta1
kind: EventGateway
# ...
spec:
  license:
  # ...
  endpoints:
    - name: gateway
      host: my-gateway.mycompany.com
      annotations:
        some.annotation.foo: "true"
        some.other.annotation: value
# ...