2 - Transform events to create or remove properties

Transform

When processing events we can modify events to remove some properties from the events. Transforms work on individual events in the stream.

Scenario: Redact personal information from order events

The operations team wants to enable another team to analyze order events, however this other team is not permitted access to personally-identifiable information (PII) about customers. They need to be able to process order events without customer PII.

Before you begin

The instructions in this tutorial use the Tutorial environment, which includes a selection of topics each with a live stream of events, created to allow you to explore features in IBM Event Automation. Following the setup instructions to deploy the demo environment gives you a complete instance of IBM Event Automation that you can use to follow this tutorial for yourself.

Versions

This tutorial uses the following versions of Event Automation capabilities. Screenshots may differ from the current interface if you are using a newer version.

• Event Streams 11.5.0
• Event Endpoint Management 11.3.0
• Event Processing 1.2.0

Instructions

Step 1 : Create a flow

1. Go to the Event Processing home page.

   

   If you need a reminder of how to access the Event Processing home page, you can review Accessing the tutorial environment.

2. Create a flow, and give it a name and description to explain that you will use it to redact customer information from orders.

   

Step 2 : Provide a source of events

The next step is to bring the stream of events to process into the flow. We will reuse the topic connection information from an earlier tutorial.

1. Update the Event source node.

   

   Hover over the node and click Edit to configure the node.

2. Choose the ORDERS topic that you used in the Identify orders from a specific region tutorial.

   

   Tip: If you haven’t followed that tutorial, you can click Add new event source instead, and follow the Provide a source of events steps in the previous tutorial to define a new Event source from scratch.

   Click Next.

3. In the Event details pane, the schema for events on this topic defined before is displayed.

   

   Click Configure.

Step 3 : Define the transformation

The next step is to define the transformation that will remove the customer personal information from events.

1. Add a Transform node and link it to your event source.

   

   Create a transform node by dragging one onto the canvas. You can find this in the Processors section of the left panel.

   Click and drag from the small gray dot on the event source to the matching dot on the transform node.

   Did you know? Instead of dragging the node, you can add a node onto the canvas and automatically connect it to the last added node by double-clicking a node within the palette. For example, after configuring an event source node, double-click any processor node to add and connect the processor node to your previously configured event source node.

2. Give the transform node a name that describes what it will do: redact personal info.

   

   Hover over the node and click Edit to configure the node.

3. As you aren’t creating any new properties, click Next.

   (You can learn about creating custom properties in the next tutorial.)

4. Remove the customer and customerid properties from the events.

   

   Remove properties from events by clicking the remove property icon next to their name.

Step 4 : Test the flow

The final step is to run your event processing flow and view the results.

1. Use the Run menu, and select Include historical to run your filter on the history of order events available on this Kafka topic.

   

2. Click the redact transform node to see a live view of results from your transform. It is updated as new events are emitted onto the orders topic.

   

Recap

You used a transform node to redact sensitive data from order events.

Next step

In the next tutorial, you will try a windowed operation to process events that occur within a time window.