Create Utilities Predict Group and Upload Sensor Data
Maximo Predict comes with notebook templates to assist in streamlining data uploads to Maximo. This notebook will create the following resources using provided csv files:
- Device Data in Monitor
- Device Asset Mapping
- Predict Groups
These instructions use the notebook named 2_FastStartLoader-Predict.ipynb
file with the Substation Transformer for Health and Predict for Utilities Demo Assets. Note that this uses simulated Pump Data for the sensor readings.
In this exercise you will use Watson Studio and Health and Predict - Utilities to:
- Gather notebook and CSV files for all data to be uploaded
- Upload the and Run the Fast Start HPU Data Loader Notebooks using a template to upload new Device Data and Predict Groups into Maximo
- Confirm the Data Has been uploaded for your assets
- Handle Errors that may come up in the process
Note
You must complete the previous exercise for Setup Watson Studio before you start this exercise.
Pre-requisites
- Review Predict documentation for the list of available models.
- Ensure your MAS Predict environment is running and you have access. Try your server URL that might look something like: https://main.predict.ivt11rel87.ivt.suite.myhost.com/ibm/pmi/service/rest/system/info
- Ensure you have Access to Asset data files for the Health and Predict Utilities Demo Data
- Complete the Load Data into Manage lab for the Utilities data.
- Complete the Load Utilities Health Scores via Notebook lab
- Have the following information from the previous lab:
SITE_ID
Note
It is best to perform this lab in your own Watson Studio Project created using Setup Watson Studio instructions. If you are using a shared project, ensure you append each file uploaded with your initials and update the file paths in the notebooks to include that change.
Gather notebooks and CSV Files
-
Ensure you have the
2_FastStartLoader-Predict.ipynb
notebook from the github as outlined in Load Data into Manage. -
In that folder, ensure you have 3 csv files under
csv > predict_csv_st
containing asset installation information, asset failure dates and sensor data -
Open the
hpu_st_failure_data_afm_vel_timeshifted_v17.csv
and update the Site ID to your site ID from Load Data into Manage -
In the
notebooks
folder there should be a file titled2_FastStartLoader-Predict.ipynb
Upload and Start the Fast Start Data Loader Notebook
-
Upload or open the Fast Start Data Loader template notebook to your Project. Use the steps from the previous exercise Add Notebook From File to a Watson Studio Project Rename the notebook template by prepending your initials to the template. If you already have uploaded the notebook, open it with Watson Studio. Select the
2_FastStartLoader-Predict.ipynb
notebook template. -
Open the notebook. Click on the
pencil
icon next to your notebook -
If the notebook fails to start, restart it. Click on the
i
icon ,Environment
tab,Running status
drop down select box and chooseRestart
Run the Notebook
-
Run the first cell two cells. These cells are setting up the files required to complete the notebook.
-
Read the introduction for details on the notebook and how it fits into the Maximo Predict process.
-
Ensure you have a 'Predict_Envs.json' file uploaded. If one is not provided, follow the instructions in set up Watson Studiounder the 'Get URL' section to gather the
APM_ID
,APM_API_BASE_URL
, andAPM_API_KEY
to create a JSON file containing the credentials and upload to Watson Studio with your initials prepended to the file name. -
Run the cell to open the JSON file.
-
Run the cell to uninstall the pmlib. This is done to ensure the right version is installed later in the process.
-
Run the following two cells to set up variables required for the environment and the structure of the JSON file that will be stored on completion of the notebook. Note that both should produce an output without error.
-
Run the following 3 cells to install pmlib and import the additional required libraries
-
Update
default_site_id = 'EULARGE1'
todefault_site_id = '{your site id}'
-
Run the cell to define your device type name, site id, and your asset group label. Make a note of the resulting output. This is the asset group label that will be the name of your predict group.
-
Run the following cell to import the csv containing the asset data and update the dataframe's data types
-
Run the next cell to import the sensor data and update the data frame to match the desired format. This will replace the device type with the one defined two cells above and adds the asset id for each row.
-
Run the following cell to rename the
timestamp
column to the desired name and data type. -
Run the following cell to create a list of the different Device Ids that will be created. Note that not all the assets have an associated device data.
-
Run the next cell to upload the failure data as a dataframe
-
Run the next cell to create add the devices to the asset data table imported in step 10
-
Run the following cell to construct the asset device mapping table that will be needed for later in the notebook
-
Run the next 6 cells to delete the data. These cells do NOT have to be run if this is the first time uploading data. If you need to re-upload the sensor data or recreate the group then the cells are required to be run. The final cell has a 5 minute delay to ensure the data gets fully deleted. Notice there is a commented out cell that would delete your assets. This is because we uploaded assets using the Load Data into Manage and did not want to delete the extra data we uploaded.
-
Run the next cell following the delete cells to import the Maximo api calls and clean up the data frames.
-
Run the next cell to confirm the asset_group_label_afm that will be the name of your predict group
-
The next few cells will be inserting the asset data into Maximo. Run the first cell to confirm the asset data to be imported
-
Run the next two cells to shift the time of the sensor data to two days ago (to be more recent) and the failure dates accordingly
-
Note that the data is now in the current year
-
The next two commented out cells will create the assets and asset attributes. Since this information was uploaded earlier in the lab, they do not have to be run
-
Run the next cell to check if the failure code exists in your organization. If it does not, it will create one.
-
Run the next cell to import the asset failure history
-
Run the following cell to define the functions to create the asset group and filter for that group
-
Run the next cell to use those functions to create the asset group filter and create teh asset group.
-
Run the next cell to get the asset group id for the recently created predict group. This will be stored in a json file and used in future notebooks.
-
Run the final cell in this section to view and note the asset group id
-
Run the first cell in the
Setup IOT Devices
section to update the column headers and view the dataframe along with the amount of data to be uploaded -
Run the following cell to load in the training data for the models. This will set up the IOT devices and the associated sensor data.
-
Run the next cell to view the asset device mappings to be uploaded
-
Run the next cell to load the asset device mappings
-
Run the final three cells to save and view the data in the
fast_execution.json
file. Ensure theasset_group_id
and the 34. Run the final three cells to save and view the data in thefast_execution.json
file. Ensure theasset_group_id
and thedevice_type
match the values displayed throughout the notebook
- The remaining cells will create scoring data for the predict group created. This is not being done since in the following labs, models will be created and assets will be scored for the created predict group at that time.
Confirm Data Upload and Prediction Groups
Confirm that the historical data was uploaded to Monitor. Confirm that the Prediction group was created and linked to the model notebook template you created.
-
Navigate to Maximo Health and Predict for Utilities within your environment
-
Use the left-hand menu to go into
Predict Grouping
-
Ensure your asset group is available in the list and confirm your asset is available within the group.
-
Navigate to IOT within your environment. Under
devices
, search to ensure yourdevice
is there -
Navigate to
Device Types
and find your device type. Check that yourphysical
andlogical interfaces
are active. -
Navigate to Monitor and ensure your devices have data
Error Handling
Update DateFrame Header
If you receive an object attribute error for pandas, with a column header initialized in Part 2 step 10 listed similar to: Then follow these steps to replace your dataframe header to resolve this error. 1. Click into the cell that resulted in an error.
-
In the menu, go to
Insert > Cell Above
-
Rename the columns by adding the following code in the cell:
{DataFrame_to_Change}.rename(columns={'{current_column_header}': site_id_col_name}, inplace=True)
{DataFrame_to_Change}.head()
-
Run the cell. The output should show the new header in the table. For example: in order to change the
failure_data_afm_df
column header from'site'
to the preferred header:
Congratulations you have loaded historical data and created Predict Groups linking your device's metrics inputs to list of assets and notebook template that will be used to score predictions using Predict with Monitor device data!