Processor nodes

The following processor nodes are available in Event Processing:

Filter

A filter node takes in a stream of events and applies an expression to determine which events to allow to pass and which to block. The output of the filter node is a single stream of events that can be used for analysis or other processing. This node helps to reduce the amount of data by allowing events that match the expression.

Adding a filter node

To add a filter node, complete the following steps:

  1. Ensure that a stream of events is available, from an Event Source node or from the output of any previously configured node.
  2. In the Palette, under Processors, drag the Filter node into the canvas.
  3. Connect the node to an event source by dragging the Output Port from a source node into the Input Port of this node. A purple checkbox unconfigured_node icon is displayed on the filter node indicating that the node is yet to be configured.
  4. Hover over the node and click Edit icon Edit to configure the node.

The Configure a Filter window appears.

Configuring a filter node

To configure a filter node, complete the following steps:

  1. In the Details section, enter a name for your node. The output stream of events from this node will be referred with the name you entered.
  2. Click Next to open the Define Filter pane.

  3. Enter an expression in the Filter Expression text box to filter the events. The expression consists of a property, a mathematical condition, and a value. You can create a simple expression with one condition or a complex expression with multiple conditions based on your requirement. You can create multiple conditions within an expression by using AND or OR.

    Examples:

    • Simple expression:

      example_property > 100

    • Complex expression:

       example_property1 > 50 AND example_property2 > 30 OR example_property3 < 25 AND example_property4 < 250

    • Event Processing 1.1.3 icon Expression with complex object property:

       example_property.nested_property IS NOT NULL AND example_property.nested_property.leaf_property > 100

    • Event Processing 1.1.4 icon Expression with an array property:

       example_array_property[1] IS NOT NULL AND ARRAY_CONTAINS(example_array_property, 10)

      Note: The index of arrays starts at 1, not at 0.

    Note: Expressions are prioritized based on operator precedence.

    Alternatively, you can use the assistant to create an expression. Select Assistant at the right end of the text-box to open the assistant. The assistant provides a drop-down list of properties and conditions that you can use to create the expression.

    Note: The Assistant drop-down menu does not support array properties. However, you can manually write the expressions by selecting the property from the list that appears when you press Ctrl+Space in the Expression field.

  4. Scroll down and click Configure Filter to complete the configuration.

A green checkbox green checkbox appears on the filter node if the node is configured correctly. If there is any error in your configuration, a red checkbox red checkbox appears.

User actions are saved automatically. For save status updates, see the canvas header.

Transform

A transform node takes in a stream of events to modify your existing properties or create new properties. Existing properties can be removed or renamed, and new properties can be added. The value of a new property is determined by the expression you create.

Transform node supports various functions to create an expression. For more information about functions, see the Flink documentation.

Adding a transform node

To add a transform node, complete the following steps:

  1. Ensure that a stream of events is available, from an Event Source node or from the output of any previously configured node.
  2. In the Palette, under Processor, drag-and-drop the Transform node into the canvas.
  3. Connect the node to an event source by dragging the Output Port from a source node into the Input Port of this node. A purple checkbox unconfigured_node icon is displayed on the transform node indicating that the node is yet to be configured.

  4. Hover over the node and click Edit icon Edit to configure the node.

    The Configure Transform window appears.

Configuring a transform node

To configure a transform node, complete the following steps:

  1. In the Details section, enter a name for your node. The output stream of events from this node will be referred with the name you entered.
  2. Click Next to open the Create properties pane.
  3. Click Create new property to add a new property to the table.
  4. Hover over the property name and click the Edit icon edit icon and enter a name for your property.

  5. Enter your expression in the Expression text box to define your property.

    Examples:

    • Simple expressions:

      - example_number_property1 * example_number_property2
- POWER(`example_number_property1`, 2)

    • Complex expressions:

       - REGEXP_EXTRACT(`example_property`, '([A-Z]+)\-([0-9]+)\-([0-9]+)', 1)
 - IF(`example_number_property` >= 20, 'high', 'low')
 - TO_TIMESTAMP(example_string_property, 'EEE MMM dd HH:mm:ss zzz yyyy')
 - DAYOFWEEK( CAST(example_timestamp_property AS DATE) )

    • Event Processing 1.1.3 icon Expressions with complex object property:

       - example_property.nested_object_property
 - example_property.nested_object_property.leaf_property
 - CONCAT( CAST(example_property.nested_object_property.leaf_property AS STRING), ‘$’)

    • Event Processing 1.1.4 icon Expressions with array properties:

       - ARRAY_JOIN(`example_string_array_property`, ', ')
 - ARRAY_REMOVE(example_property.nested_object_property.leaf_array_property, NULL)
 - example_property BETWEEN example_array_property[1] AND example_array_property[2]
 - example_property.nested_object_property.leaf_array_property

      Event Processing 1.1.3 icon Note: The expression can also simply refer to existing properties. This is particularly useful when renaming complex object properties or when moving nested properties to the top-level.

  6. Optional: To use the assistant to enter an expression, complete the following steps:
    1. Click the Assistant drop-down menu to open the assistant.
    2. From the Select function list, select a function with which you want to create an expression.
    3. In the function you selected previously, enter the required values.

    4. Select Insert into expression to add the expression in the text-box.

      Note: Ensure you choose the values with correct data type for your expression. Properties that are used as values in the comparison operations must be of the same data type. Arithmetic operations are only allowed on integer and double data types. String operations are only possible with properties of string data type. Some temporal functions require a timestamp data type.

    Note: The Assistant drop-down menu does not support array properties. However, you can manually write the expressions by selecting the property from the list that appears when you press Ctrl+Space in the Expression field.

  7. Optional: Repeat steps 3 to 6 to create more properties.
  8. Click Next to open the Output properties pane. You can manage the properties that come from this node to suit your requirements.

  9. Event Processing 1.1.3 icon Only leaf properties are listed in the Output properties table. You can remove specific properties from an object, or if you want to remove the entire object, remove all the properties related to it one by one.

    Click Remove property remove icon to remove a property so that it is not displayed in the output.

    Note: Nested properties are displayed by using a forward slash (/) as a separator for each level of nesting. For example, product / id or customer / address / city.

  10. Event Processing 1.1.3 icon Only leaf properties are listed in the Output properties table. You can only rename the last nested property within nested properties, you cannot rename the top-level or any other properties in between. For example, in the case of customer / address / city, you can only rename city, but not address or customer.

    Note: To rename the top-level property customer/address/city (in this case, customer), create a new property and add details in the following fields:

    • Property name: Add a new name for the property.
    • Expression: Add the name of the property to rename, for example, customer.
  11. Optional: To rename a property, hover over the property name and click the Edit icon edit icon.
    • In the text-box, enter a new name for your property.
    • Click outside the text-box or press Enter on your keyboard to rename the property.
  12. Optional: To add a property that was previously removed, go to the Properties to remove table that lists the removed properties. For the property you want to add back, click the Add icon add icon.
  13. To complete the transform node configuration, click Configure.

A green checkbox green checkbox appears on the transform node if the node is configured correctly. If there is any error in your configuration, a red checkbox red checkbox appears.

User actions are saved automatically. For save status updates, see the canvas header.