Deploying a custom Flink session job

You can create and run a Flink session job that is written in Java by using the IBM Operator for Apache Flink.

The FlinkSessionJob custom resource is used to create and manage Flink jobs in a session cluster. The FlinkSessionJob instance can only run with an existing session cluster managed by the FlinkDeployment instance. The FlinkSessionJob custom resource contains the information to submit a session job to the session cluster with following information:

  • deploymentName: The name of the Flink deployment instance deployed by using the FlinkDeployment custom resource.
  • job.jarUri: Specifies the URI of the Flink job JAR file.

The Flink job is deployed to the target session cluster mentioned in the spec. You can then manage the job running in the session cluster.

Prerequisites

Ensure your environment meets the following prerequisites before running a Flink session job:

  • The IBM Operator for Apache Flink installed.
  • A Flink session cluster is deployed by using the FlinkDeployment custom resource.
  • A Flink job written in Java. For examples, see GitHub.

Procedure

  1. Install a FlinkSessionJob instance with your configurations.

  2. Apply the FlinkSessionJob custom resource. To apply the custom resource by using the CLI:

    kubectl apply -f <name-of-the-custom-resource>
    
  3. To ensure that the job is submitted against the session cluster and that the Flink job is running, go to the FlinkSessionJob custom resource and check whether the status of the status.jobStatus.state is RUNNING:

    status:
      jobStatus:
        ...
        state: RUNNING
        ...