Engines

In IBM StreamSets, an engine is a runtime environment where flows are executed. Settings for engine resources are taken from the configuration of the Environment that the engine belongs to, such as CPU, memory, and storage. They can be configured with specific settings, like Java version, memory allocation, and maximum concurrent flows, to optimize performance and resource utilization.

Create an Engine

The SDK provides functionality for interacting with an engine, however an engine cannot be created directly. Instead, you can retrieve the installation command to install and start an Engine directly from it’s Environment via the Environment.get_installation_command() method.

For more information, refer to our Environment documentation.

Retrieving Existing Engines in a Project

To retrieve existing engines in a project, use the engines property of the Project class. This will return a list of Engine objects.

>>> # Get all engines associated with the project
>>> engines = project.engines

Retrieving an Existing Engine in a Project

To retrieve existing engines in a project, use the Project.get_engine() method of the Project class. You must specify engine_id of the existing engine. This will return a Engine object.

In the UI, you can retrieve an Engine ID by navigating to Manage -> StreamSets -> Environment and click on your environment name for details.

>>> # Get a single engine by engine_id
>>> engine = project.get_engine(engine_id='C25F1E5F-A901-44F7-A27E-2B0E94C23224')

Retrieving Existing Engines in an Environment

To retrieve existing engines in an Environment, use the Environment.engines property of the Environment class. This will return a list of Engine objects.

>>> # Get all engines associated with the environment
>>> engines =  environment.engines

Deleting an Existing Engine

To delete an engine instance, pass the Engine object you want to delete into the Project.delete_engine() method to delete it.

>>> # Delete an engine
>>> project.delete_engine(engine)