Lab 2: Create rules to analyze data¶
Information Server enables users to better understand their data. It uses pre-built and custom rules that will apply meaning and quality measurements, which are available for users of the data and interested parties.
This lab, will give you hands-on experience using Information Server's rule capabilities. In this lab, you will learn the following:
- How to create a data rule
- Apply the rule to a data set
- View data that does not apply to the rule
This section is comprised of the following steps:
1. Import and view the data¶
Launch Firefox and click on the Launchpad bookmark. When the Information Server launchpad shows up click on the Information Governance Catalog New tile.
Log in with the username isadmin and password inf0Xerver.
The overview page will appear.
Click on the Connections tab and ensure the connection JK_ORD appears and the host is IIS-SERVER.IBM.DEMO. This connection is pre-established for convenience and was imported to the JK Orders workspace, let's take a look at it.
To find the JK Orders workspace, go to Quality tab and click on the right tile.
The workspace overview will load. Take a few moments to browse the graphics on the page and click on the Data sets link to view the data in this exercise.
Before we create new rules let's look at the two data sets that will be used in this example. Take a look at the Sales data set first.
Click on the Columns tab to view findings from the analyzer. It found many things when the data was imported, like maximum and minimum values, distinct values, format, and uniqueness.
Repeat the same for the CUSTOMERS data set.
We're now ready to create our first data rule!
2. Create a data rule¶
From the JK Orders workspace click on the Data rules tab.
Expand the Published Rules section, then the 01 Personal Identity section, and then the Age section to reveal the AdultInRangeNumeric rule.
Navigate to the right to show the ⋯ menu. Choose the Manage in workspace option from the menu.
The rule should now appear under the All section. Navigate to the right to show the ⋯ menu. Choose the Edit option from the menu.
Now we're going to edit the rule so that it flags ages that are under 18 or over 120. To do that, edit the text boxes to the right that show minage and maxage and update them with 18 and 120 respectively. The formula at the bottom should read:
age >= 18 and age < 120
Save the rule.
When you are brought back to the Data rules tab you'll notice that the new rule has an error. We need to publish the rule. To do so navigate to the right to show the ⋯ menu. Choose the Publish option from the menu.
Confirm the publish action.
3. Re-analyze and view results¶
Go back to the JK Orders workspace and click on the Data sets links
Click on the SALES data set.
We can now apply the rule by clicking the + Add rule button.
Choose the AdultInRangeNumeric rule and click Next.
We now have to bind the rule to a specific column in our data set. Click on the age variable name on the left, this is the rule. And click on the AGE column name on the right, this is our data source. Click on the Bind button and once bound you can click Next.
We don't need to test the rule this time, we can just click on Save.
You should now see the rule in the data set view. Click on the Analyze button to restart the analysis with the new rule.
Confirm to start the analysis.
You may need to refresh your browser a few times for this step. It will take a few minutes. You'll see the state go to Running to Successful when done.
Once complete, go to the Data quality tab. You'll see that the new rule has one finding. Click on the findings to view the exception.
Scrolling to the right you'll see that there is one entry whos age is 17, this is less than the required 18 for our rule!
CONGRATULATIONS!! You have completed this lab!