Connections

Connections#


A connection is an object that is used to store connection related data for a single datasource (Kafka, Azure, etc.) like credentials, secrets and urls. Each connection is defined by exactly one datasource.

Tip

To read more about datasources see Retrieving a DatasourceType.

The SDK provides functionality to interact with connections.

This includes operations such as:

  • Creating a connection

  • Retrieving connection(s)

  • Updating a connection

  • Deleting a connection

Creating a Connection#

In the UI, you can create a new Connection by navigating to Assets -> New asset -> Connect to a datasource.

Screenshot of the Connection creation in the UI - Step 1 Screenshot of the Connection creation in the UI - Step 2

You will need to choose the desired datasource from the list

Screenshot of the Connection creation in the UI - Step 3

and provide a Name and other additional configuration (depending on the selected datasource).

Screenshot of the Connection creation in the UI - Step 4

In the SDK, you can create a new Connection object within a Project, by selecting the appropriate project from the Platform and then using the Project.create_connection() method to instantiate the connection.

You must provide a name and datasource type to create connection. DatasourceType is an object defining the data source for which the connection will be created.

By default before creating any connection, server will validate it. If the connection cannot be estabilished with provided parameters, you will get an error and connection will not be saved. To skip validation and create connection anyway set test parameter of Project.create_connection() to False.

Note

To get available datasources see: Retrieving a DatasourceType.

>>> datasource_type = platform.datasources.get(name='http')
>>> connection = project.create_connection(
...     name='Connection Name',
...     datasource_type=datasource_type,
...     description='Description ...',
...     properties={'url': 'https://my/connection/url'},
... )
>>> connection
Connection(name='Connection Name')

Important

Each datasource_type defines its own set of required properties.

To get list of available properties see: Retrieving a DatasourceType connection properties.

Retrieving an existing Connection#

In the UI, you can get all Connections by navigating to Assets -> Data access -> Connections.

Screenshot of the Connection listing in the UI

In the SDK, Connection can be retrieved using Project.connections property. You can also further filter and refine the connections returned based on attributes including name, context, properties and datasource_type.

This property returns a Connections object.

>>> # Returns the first connection matching given `name`
>>> project.connections.get(name='Connection Name')
Connection(name='Connection Name')

>>> # Return a list of all connections that match `properties`
>>> project.connections.get_all(
...     properties={'url': 'https://my/connection/url'}
... )
[Connection(name='Connection Name')]

>>> # Return a list of all connections
>>> project.connections 
[Connection(name='Connection Name')]

Tip

For detailed information about parameters and values refer to https://cloud.ibm.com/apidocs/data-ai-common-core#listconnections.

Updating a Connection#

In the UI, you can update a Connection by navigating to Assets -> Data access -> Connections and clicking the edit button on the Connection you want to edit. This button is visible only when cursor is over the Connection object.

Screenshot of the Connection updating in the UI

To update a connection in the SDK, we first modify properties of the connection and then pass the instance to the Project.update_connection() method.

This method returns an HTTP response indicating the status of the update operation.

Similar to Creating a Connection you can skip the validation by setting test parameter of Project.update_connection() to False.

>>> connection.name = 'New Connection Name'
>>> project.update_connection(connection)
<Response [200]>
>>> project.connections.get(name='New Connection Name') 
Connection(name='New Connection Name')

Using a Connection (Batch Only)#

In the UI, you can use a Connection in a flow by adding a connector stage and choosing the Connection in the view connection property.

Screenshot of the Connection usage in the UI

In the SDK to use a Connection instance you can pass it to the use_connection method of the connector stage.

>>> new_flow = project.create_flow(name='New flow', environment=None, flow_type='batch')
>>> connection = project.connections.get(name='New Connection Name')
>>> connection 
Connection(name='New Connection Name')
>>> http = new_flow.add_stage(type='HTTP', label='HTTP_Stage_1')
>>> http.use_connection(connection)

Using a local Connection (Batch Only)#

An alternate way to use a connector stage is by putting connection parameters directly into the stage’s properties. In the UI this can be done under the Connection details section of the stage.

If you are already Using a Connection the Connection details section will not be visible.

Screenshot of the Local Connection usage in the UI

In the SDK each connector stage’s configuration has a connection property. To use a local connection you can directly edit the desired fields of this connection property.

>>> http = new_flow.add_stage(type='HTTP', label='HTTP_Stage_1')
>>> http.configuration.connection.url = 'yoururl.com'
>>> http.configuration.connection.ssl_certificate = 'yoursslcertificate'

Deleting a Connection#

In the UI, you can delete an existing Connection by navigating to the Assets -> Data access -> Connections.

Screenshot of the Connection deletion in the UI

In the SDK to delete a connection instance you can pass it to Project.delete_connection() method.

This method returns an HTTP response indicating the status of the delete operation.

>>> project.delete_connection(connection)
<Response [204]>
Section Contents