Cloud Pak for Data Watson Machine Learning Development Space Job Information

Monitor Summary

This monitor will generate the following observations:

Number of Cloud Pak for Data Waston Machine Learning Deployment Jobs across all spaces
Number of Acitve Cloud Pak for Data Waston Machine Learning Deployment Jobs per space
Status of Last Cloud Pak for Data Waston Machine Learning Deployment Job per space

Monitor Requirements

In order for Watson Studio Job Information to be available, CCS (Common Core Services) must be available on the Cloud Pak for Data instance. Catridges like Watson Studio and Watson Machine Learning automatically install CCS as part of their dependencies. If CCS is not available, no watson studio job information is available.

Administrative Credential Requirements

In order to retrieve the monitor data, the credential of admin user will be used. The default password of admin user is saved in the secret admin-user-details. You could use the below command to query the default password of the admin user..

oc extract secret/admin-user-details --keys=initial_admin_password --to=-

Once the password of the admin user is changed, please also change the password in the secret admin-user-details.

oc set data secret/admin-user-details initial_admin_password=<new password>

Deployment of the monitor

This monitor can be deployed using the IBM Cloud Pak Deployer or manually deployed using oc commands:

Monitor Source

Source folder containing the script can be found here (cp4d-wml-deployment-space-job-info).

Generated observations

Once the monitor is deployed, the following observations are available in IBM Cloud Pak for Data Metrics:

Using the IBM Cloud Pak for Data Platform Management Events:

https://<CP4D-BASE-URL>/zen/#/platformManagement/events

On the Platform Management Events page the following entries are added:

Event	Event Type	Possible values	Description
Cloud Pak for Data Waston Machine Learning Deployment Space Jobs Overall Count	Number of Cloud Pak for Data Waston Machine Learning Deployment Jobs across all spaces	info	The Cloud Pak for Data Waston Machine Learning Spaces information is requested.
Cloud Pak for Data Waston Machine Learning Deployment Space Active Count	Number of Acitve Cloud Pak for Data Waston Machine Learning Deployment Jobs per space	info	The Cloud Pak for Data Waston Machine Learning Spaces information is requested.
Cloud Pak for Data Waston Machine Learning Deployment Space Job Last Status	Status of Last Cloud Pak for Data Waston Machine Learning Deployment Job per space	info	The Cloud Pak for Data Waston Machine Learning Spaces and Jobs information are requested.

Using the IBM Cloud Pak for Data Prometheus endpoint

https://<CP4D-BASE-URL>/zen/metrics

It will generate 2 types of metrics:

Actual Result metrics
These metrics contain the observed values
Zen Watchdog metrics
These metrics are used by the Cloud Pak for Data Zen Watchdog to monitor the state and trigger notifications

Actual Result metrics:

cp4d_wml_deployment_space_jobs_overall_count Provides number of Cloud Pak for Data Waston Machine Learning Deployment Jobs across all spaces
cp4d_wml_deployment_space_jobs_active_count Provides number of Acitve Cloud Pak for Data Waston Machine Learning Deployment Jobs per space
cp4d_wml_deployment_space_job_last_status Provides status of Last Cloud Pak for Data Waston Machine Learning Deployment Job per space

# HELP cp4d_wml_deployment_space_job_last_start_epoch_time 
# TYPE cp4d_wml_deployment_space_job_last_start_epoch_time gauge
cp4d_wml_deployment_space_job_last_start_epoch_time{event_type="cp4d_wml_deployment_space_job_last_status",monitor_type="cp4dplatformwmldeploymentspacejobinfo",reference="test-demo_test-batch-job"} 1.653974483e+09
# HELP cp4d_wml_deployment_space_job_last_status_duration_in_seconds 
# TYPE cp4d_wml_deployment_space_job_last_status_duration_in_seconds gauge
cp4d_wml_deployment_space_job_last_status_duration_in_seconds{event_type="cp4d_wml_deployment_space_job_last_status",monitor_type="cp4dplatformwmldeploymentspacejobinfo",reference="test-demo_test-batch-job"} 1
# HELP cp4d_wml_deployment_space_job_last_status_success 
# TYPE cp4d_wml_deployment_space_job_last_status_success gauge
cp4d_wml_deployment_space_job_last_status_success{event_type="cp4d_wml_deployment_space_job_last_status",monitor_type="cp4dplatformwmldeploymentspacejobinfo",reference="test-demo_test-batch-job"} 1

Zen Watchdog metrics (used in platform management events)

watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_jobs_overall_count
watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_jobs_active_count
watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_job_last_status

Zen Watchdog metrics can have the following values:

2 (info)
1 (warning)
0 (critical)

# HELP watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_job_last_status 
# TYPE watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_job_last_status gauge
watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_job_last_status{event_type="cp4d_wml_deployment_space_job_last_status",monitor_type="cp4dplatformwmldeploymentspacejobinfo",reference="test-demo_test-batch-job"} 2
# HELP watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_jobs_active_count 
# TYPE watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_jobs_active_count gauge
watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_jobs_active_count{event_type="cp4d_wml_deployment_space_jobs_active_count",monitor_type="cp4dplatformwmldeploymentspacejobinfo",reference="test-demo"} 2
# HELP watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_jobs_overall_count 
# TYPE watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_jobs_overall_count gauge
watchdog_cp4dplatformwmldeploymentspacejobinfo_cp4d_wml_deployment_space_jobs_overall_count{event_type="cp4d_wml_deployment_space_jobs_overall_count",monitor_type="cp4dplatformwmldeploymentspacejobinfo",reference="All Deployment Space"} 2

Edit this page on GitHub

Monitors: Watson Machine Learning Development Space Information

Setup Prometheus