Skip to main contentIBM DataPower Operator Documentation

DataPowerService

The DataPowerService is a Custom Resource Definition used to deploy and manage a DataPower StatefulSet.

Spec

Properties

PropertyDescriptionRequiredDefault
licenseLicense acceptance and useYesNone
versionDataPower firmware versionYesNone
replicasDesired number of Pods in the StatefulsetYesNone
imageCustom DataPower imageNoNone
imagePullSecretsImage pull secretsNoNone
usersList of Users to associate with DataPowerNoSee doc below
domainsList of DataPowerDomains to associate with ServiceNoNone
envEnvironment variablesNoNone
initCmdsCommands to run during user-specified initialization stageNoNone
extraExeList of ConfigMap names to mount containing extra executablesNoNone
debugEnable debug logging on init scriptsNofalse
resourcesResource specifications for StatefulSetNoSee doc below
livenessProbeCustom LivenessProbeNoSee doc below
readinessProbeCustom ReadinessProbeNoSee doc below
affinityCustom Kubernetes Affinity configurationNoSee doc below
storagePersistent volume requestsNoSee doc below
podManagementPolicyControl how pods are deployedNoParallel
serviceAccountNameName of desired service accountNoNone
odTracingOpenTracing Agent and Collector container configurationsNoSee doc below
annotationsCustom annotations to apply to PodsNoNone
labelsCustom labels to apply to PodsNoNone
tolerationsToleration settings to apply when scheduling DataPower PodsNoNone
nodeSelectorNodeSelector key-value pairs to match when scheduling DataPower PodsNoNone
lifecycleContainer lifecycle hook to be executedNoNone

license

Description

The License property contains two required fields:

  • accept: must be boolean true for the DataPower pod(s) to start successfully. This property directly sets the DataPower DATAPOWER_LICENSE_ACCEPT environment variable in pods, which acknowledges the license acceptance.
  • use: the license use. Can be one of the following values:
    • production
    • nonproduction
    • developers
    • developers-limited

The license.use is used in tandem with the version property to determine what DataPower firmware image to pull and use in the deployment, see Image selection.

To view the full license terms, you can:

docker run --rm <image> --show-license

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
license:
accept: true
use: production
...

version

Description

The version property specifies the desired DataPower firmware version to reconcile. This value must match an available version or channel the CRD and operator provide. Subscribing to a channel will allow for automatic updates to future versions on that channel. If the DataPowerService instance is created with a channel specified in version, the latest available version in that channel will be reconciled.

You are also able to lock in a specific version by specifying the full version string, for example 10.0.0.0.

You can view the reconciled and available versions and channels on the DataPowerService instance’s status, once created.

Channels

Supported

The table below shows the supported channels for this CRD version (v1beta2) and operator version (1.2.x). Supported channels indicate channels which can be specified by spec.version for which this version of the DataPower Operator will understand and reconcile.

ChannelStreamLatest
10.0-eus10.0.1.x10.0.1.5-eus
10.0-lts10.0.1.x10.0.1.5
10.0-cd10.0.0.x10.0.0.1

Available

The table below shows the available channels for this CRD version (v1beta2) and operator version (1.2.x). Available channels (and version) are advertised in Cloud Pak for Integration as targets for operand upgrade. To enable and support the Cloud Pak for Integration EUS strategy, version 1.2.x will only advertise EUS operand channels and versions.

ChannelStreamLatest
10.0-eus10.0.1.x10.0.1.5-eus

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
version: 10.0-eus

replicas

Description

The replicas property is an integer that directly sets the replicas field on the StatefulSet of DataPower pods. This property sets the number of DataPower pods you wish to deploy.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
replicas: 3
...

image

Description

The image property allows for a custom built DataPower Docker image to be used in the deployment. This field is optional, and will override the image that would be specified by version and license.use. The value of this property, if provided, will be directly used in the DataPower Pod specs.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
image: my.registry.com/my-custom-datapower:v10
version: 10.0.0.0
...

imagePullSecrets

Description

The imagePullSecrets property specifies the name of Kubernetes Secret objects in the cluster which will provide the Docker registry credentials to pull any images needed during reconciliation and deployment.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
imagePullSecrets:
- datapower-cred-one
- datapower-cred-two
...

users

Description

The users property defines the user accounts to be created. The name field will be the name of the account and the passwordSecret field is the name of the secret that contains the credentials for the user account. The access-level and group property define the access-level, either privileged or group-defined, and the group, if access-level is group-defined. If using groups, it is expected that the configuration provided for the default domain would define the user group in RBM settings. This property does not create the group, it just creates a user asigned to the specified group.

An admin account must be defined. By default, an admin account is defined using the secret name admin-credentials. You can either create a secret with that name containing the credentials for the admin account, or use a different secret name by changing the passwordSecret field for the admin user in the CR.

The following are values in the secret which can be used to define the user’s credentials:

  • password-hashed: The hashed value (see Linux man 3 crypt for format) of the user’s password. Required if password is not defined.
  • password: The user’s password. Required if password-hashed is not defined; ignored if password-hashed is defined.
  • salt: The salt value used when hashing password (see man 3 crypt). Optional; ignored when password-hashed is defined. (Default: 12345678)
  • method: The name of the hashing algorithm used to hash password. Valid options: md5, sha256. Optional; ignored when password-hashed is defined. (Default: md5)

The following examples create Secrets with different values, but result in an user with the same credentials (and the same password hash):

  • kubectl create secret generic userName-credentials --from-literal=password=helloworld --from-literal=salt=12345678 --from-literal=method=md5
  • kubectl create secret generic userName-credentials --from-literal=password=helloworld
  • kubectl create secret generic userName-credentials --from-literal=password-hashed='$1$12345678$8.nskQfP4gQ8tk5xw6Wa8/'

These two examples also result in Secrets with different values but identical user credentials

  • kubectl create secret generic userName-credentials --from-literal=password=hunter2 --from-literal=salt=NaCl --from-literal=method=sha256

  • kubectl create secret generic userName-credentials --from-literal=password-hashed='$5$NaCl$aOrRVimQNvZ2ZLjnAyMvT3WgaUEXoWgwkgyBrhwIg04'

    Notice that, when setting password-hashed, the value must be surrounded by single-quotes

For more information, read the Kubernetes documentation on Secrets.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
users:
- name: admin
passwordSecret: admin-credentials
access-level: privileged

domains

Description

The domains property allows for user-provided domain configuration(s) to be injected into the DataPower pods at runtime, prior to the DataPower process starting.

For an in-depth look at utilizing this property, please see Domain Configuration.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
domains:
- name: "default"
certs:
- certType: "usrcerts"

env

Description

The env property allows you to set custom environment variables that will be set and used within the DataPower containers in the StatefulSet. This property exposes the Kubernetes API for declaring environment variables in the container, and thus follows the same schema. For more information, please see the Kubernetes documentation.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
env:
- name: DATAPOWER_LOG_LEVEL
value: "7"
...

initCmds

Description

The initCmds property is an array of strings that represent initialization commands that you wish to run prior to the DataPower process starting. These commands will be run after the DataPower Operator internal initialization scripts, but before the DataPower start.sh script.

This property can be used in conjunction with the extraExe property to run custom initialization scripts that you’ve added to the DataPower pod(s), or any custom sidecar processes you wish to run in the DataPower containers.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
initCmds:
- /usr/local/extra/test.sh
- ls -al /opt/ibm/datapower/drouter/config
...

extraExe

Description

The extraExe property is an array of strings that represent name(s) of ConfigMaps in the cluster. When the DataPower pod(s) are created, the ConfigMaps specified by extraExe are mounted with r+x permissions at /usr/local/extra.

This property can be used in conjunction with the initCmds property to run custom initialization scripts that you’ve added to the DataPower pod(s), or any custom sidecar processes you wish to run in the DataPower containers.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
extraExe:
- config-map-one
- config-map-two
- config-map-three

debug

Description

The debug property is a user toggleable debugging utility. If not set, it will default to false. For more information on the use of this property, see the Init Scripts troubleshooting guide.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
debug: true

resources

Description

The resources property specifies the desired resources to allocate to the DataPower containers in the statefulset. It is worth noting that you can only request CPU, there is not a limit field, as the DataPower Operator will set:

DATAPOWER_WORKER_THREADS == container.limits.cpu == container.requests.cpu

Thus, when you define the CPU request, all of the above three values will get that value in the statefulset. This ensures best practices are followed in the DataPower runtime statefulset.

You are still able to set both requests and limits for memory. In both cases (cpu and memory) the values must follow Kubernetes formatting constraints.

If this field is not provided, or parts of the resources object are left unset, the example below shows the default value(s) that will be used.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
resources:
requests:
cpu: 4
memory: 4Gi

livenessProbe

Description

The livenessProbe property allows you to set a custom Livensss Probe for the DataPower container(s) in the StatefulSet. The value of this property must be a well formed Kubernetes Probe type. If not specified, the following default will be used:

livenessProbe:
failureThreshold: 3
httpGet:
path: /
port: 7879
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 5
successThreshold: 1

This default livenessProbe leverages the built-in DataPower health-check sidecar process.

Example

In addition to the above example, it’s worth reviewing Kubernetes documentation on Configure Liveness, Readiness and Startup Probes for additional details on what configuration capabilities are available for this field.

readinessProbe

Description

The readinessProbe property allows you to set a custom Readiness Probe for the DataPower container(s) in the statefulset. The value of this property must be a well formed Kubernetes Probe type. If not specified, the following default will be used:

readinessProbe:
failureThreshold: 3
httpGet:
path: /
port: 7879
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 5
successThreshold: 1

This default readinessProbe leverages the built-in DataPower health-check sidecar process.

Example

In addition to the above example, it’s worth reviewing Kubernetes documentation on Configure Liveness, Readiness and Startup Probes for additional details on what configuration capabilities are available for this field.

affinity

Description

Affinity is an interface for specifying custom affinity settings. This field is a literal Affinity object from the Kubernetes library, so it is fully featured and applies directly to the pods. When no Affinity is specified, the defaults shown in the example below are used.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:

storage

Description

The Storage property is a list of storage definitions. Storage definitions fall into two categories: ephermeral and persistent. For persistent storage definitions, the Persistent Volume must use an access mode of ReadWriteOnce.

The following fields can be modified:

  • type
  • path
  • size
  • class
  • volumeMode
  • selector
  • deleteClaim

type

type defines the kind of storage volume that will be created. There are two valid options: ephemeral and persistent. Ephemeral storage takes the form of EmptyDir volumes mounted on path. Persistent storage takes the form of Persistent Volume Claims.

path

path is simply the path internal to the container to persist. The path also informs the names of the volume resources to ensure uniqueness.

This field applies to both ephemeral and persistent types.

size

size is the size of the persisted volume. The expected format is some form of #Gi. For more information on resource definitions, see Kubernetes Resources documentation.

This field applies to both ephermeral and persistent types.

class

class is a direct pass through to the storageClassName field for the Persistent Volume Claim. It determines which class to use for the volume. See Kubernetes StorageClass documentation for more details.

This field applies only to the persistent type.

volumeMode

If a value for volumeMode is provided, it will be overridden and set to Filesystem instead. See Kubernetes VolumeMode documentation for more details.

This field applies only to the persistent type.

selector

selector is a set of label selectors to refine the search for a Persistent Volume to bind.

This field applies only to the persistent type.

deleteClaim

deleteClaim is a boolean value to specify whether a persistent volume claim should be deleted or retained.

This field applies only to the persistent type.

Example

storage:
- type: persistent
path: /opt/ibm/datapower/drouter/config
size: 10Gi
class: some-storage-class
volumeMode: Filesystem
selector:
matchLabels:
example.label: some.value

podManagementPolicy

Description

The podManagementPolicy property maps directly to the StatefulSet PodManagementPolicy. There are two valid options: Parallel and OrderedReady. Parallel, the DataPowerService default, will make pods initially install in parallel to decrease time-to-ready. OrderedReady will make pods install sequentially, each waiting for the previous to become Ready before begining to install. For more information, read the Kubernetes documentation on PodManagementPolicy.

Example

spec:
podManagementPolicy: OrderedReady

serviceAccountName

Description

The serviceAccountName is a pass-through field for a Pod’s ServiceAccountName. This field should be a string that is the name of the Service Account that is to be used by the deployed DataPower Pods.

If no value is provided, a minimal Service Account will be generated for use by the StatefulSet. The generated Service Account will also include reference to the ibm-entitlement-key image pull secret. For more information on this, see the Entitled Registry documentation.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
serviceAccountName: "example-serviceaccount"

odTracing

Description

The odTracing property contains the necessary information to enable OpenTracing and run an agent and collector containers. If odTracing is not defined, OpenTracing will be disabled by default. All of the required properties must be provided. It is recomended to only provide values for the agent and collector probes if instructed by IBM support. The agent and collector probes will default to recomended values shown in the example below when not included. If including custom probe settings, only properties that need to be changed have to be included. When going through the OpenTracing registration process, the secret that stores credentials should be named icp4i-od-store-cred.

Required:

  • enabled
  • imagePullPolicy
  • odTracingRegistrationHostname
  • odTracingDataHostname
  • imageAgent
  • imageCollector

Optional:

  • agent
  • collector

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
odTracing:
enabled: true
imagePullPolicy: "IfNotPresent"
odTracingRegistrationHostname: icp4i-od.tracing.svc

annotations

Description

The annotations field serves as a pass-through for Pod annotations. Users can add any annotation to this field and have it apply to the Pod. The annotations here overwrite the default annotations if provided. For example, if the annotation prometheus.io/scrape was provided, it would overwrite our default value of "true".

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
annotations:
prometheus.io/scrape: "false"

labels

Description

The labels field serves as a pass-through for Pod labels. Users can add any label to this field and have it apply to the Pod. The labels here overwrite the default labels if provided. For example, if the label app.kubernetes.io/instance was provided, it would overwrite our default value of <cr.namespace>-<cr.name>.

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
labels:
app.kubernetes.io/instance: "my-service"

tolerations

Description

The Tolerations field allows users to select which nodes DataPower Pods can be scheduled or not scheduled. By specifying a set of matching tolerations, users can control whether specific nodes can be used for scheduling DataPower Pods. In order for this to work, the corresponding nodes must have been tainted accoringly. For more information, please see the Kubernetes documentation.

Example

The following tolerations allow DataPower Pods to be scheduled on nodes tainted with host=dp:NoSchedule.

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
tolerations:
- key: "host"
value: "db"
operator: "Equal"

nodeSelector

Description

The NodeSelector field allows users to specify a set of key value pair that must be matched against the node labels to decide whether DataPower Pods can be scheduled on that node. Only nodes matching all of these key value pairs in their labels will be selected for scheduling DataPower Pods. For more information, please see the Kubernetes documentation.

Example

The following illustrates how to schedule DataPower Pods only on nodes labeled hostType=gateway.

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
nodeSelector:
hostType: gateway

lifecycle

Description

lifecycle allows scripts to be executed after starting and before stopping the DataPower container. For more details on container lifecycle hooks, please refer to the following Kubernetes documentation:

Example

apiVersion: datapower.ibm.com/v1beta2
kind: DataPowerService
metadata:
name: example-dpservice
spec:
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo Hello from the postStart handler >> ~/lifecycle.log"]

Delaying SIGTERM with preStop

When the DataPower container is terminated by Kubernetes under normal conditions (such as a scaling event or rolling update), any in-flight transactions may be terminated. The use of the preStop lifecycle hook can help to mitigate this by delaying the SIGTERM being sent to the container. For example, a 20 second delay in the below example would delay the shutdown by 20 seconds giving in-flight transactions on the gateway that much time to complete successfully before being terminated.

spec:
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "sleep 20"]

Status

Example

$ oc get dp
NAME PHASE READY SUMMARY VERSION AGE
example Running True StatefulSet replicas ready: 3/3 10.0.1.0 171m
$ oc describe dp/example
...
Status:
Conditions:
Last Transition Time: 2020-09-21T17:19:02Z

Conditions

The DataPowerService supports Kubernetes Conditions. These Conditions can be viewed as a snapshot of the current and most up-to-date status of the DataPowerService instance.

  • The Pending condition will be set True when the DataPowerService is first created and is reconciling the desired StatefulSet. Once the StatefulSet is created and Ready, this Pending condition will be removed.
  • The Ready condition will be set True when all of the Pods in the StatefulSet are Ready, and the number of replicas in the StatefulSet matches the desired number of replicas set by the DataPowerService instance spec.
  • The Error condition will be set True when an error is encountered during reconciliation of the DataPowerService. The Reason and Message should have details regarding what the error may have been, such as a referenced ConfigMap not being found.

Custom Images

The CustomImages status property is a boolean flag that will be set True when a custom image (set via the image property in the CR spec) is being used.

Managment Ports

The MgmtPorts status property is a list of management ports for the DataPowerService. Each item in the list will show the configured port for the respective management interface.

Nodes

The Nodes status provides the list of DataPower Pods currently deployed via the StatefulSet.

Phase

The Phase status property provides the current phase of the DataPowerService. The value can be one the following:

  • Pending - There are no errors, but the DataPowerService is not yet Ready.
  • Running - There are no errors, and the DataPowerService is Ready.
  • Failed - There has been a ReconcileError.

Versions

The Versions status provides information about the currently reconciled version of the operand (DataPower), as well as the available versions and channels the operator supports.

A channel, such as 10.0-lts, allows for a user to subscribe to a stream of releases. When initially deployed with the version property in the CR spec set to a channel, the operator will automatically install the latest available version within that channel. When new versions are released in that channel, the operands (CR instances) will automatically be upgraded by the operator.

The currently reconciled version will always be available in status.versions.reconciled, even if the version specified in the CR spec is a channel.