Models

Section Contents

Models#

Access Groups#

Module containing AccessGroup Model.

pydantic model ibm_watsonx_data_integration.cpd_models.access_groups_model.AccessGroup#

Bases: BaseModel, ProjectCollaboratable

Model representing an Access Group, including Rules.

The __init__ of the AccessGroup class.

Parameters:

  • platform (Optional[Platform], default: None) – The Platform object.

  • access_group_json (dict) – The JSON for the AccessGroup.

field access_group_id: str [Optional]#
field created_at: str [Required]#
field created_by_id: str [Required]#
field description: str [Required]#
field id: str [Required]#
field last_modified_at: str [Required]#
field last_modified_by_id: str [Required]#
field name: str [Required]#
add_members_to_access_group(members: UserProfile | ServiceID | TrustedProfile | list[UserProfile | ServiceID | TrustedProfile])#

Adds members to an AccessGroup.

Parameters:

members (UserProfile | ServiceID | TrustedProfile | list[UserProfile | ServiceID | TrustedProfile]) – The members to be added to the access group.

Return type:

Response

Returns:

A HTTP response.

check_membership(member: UserProfile | ServiceID | TrustedProfile)#

Checks the membership of a member in an access group.

Parameters:

member (UserProfile | ServiceID | TrustedProfile) – The member object to check membership of.

Return type:

Response

Returns:

A HTTP response.

get_access_group_members()#

Gets all members of an AccessGroup.

Return type:

list

Returns:

A list of current members of an Access Group

get_service_ids(service_id_set: set)#

Gets all service IDs with membership to the current access group.

Parameters:

service_id_set (set) – The set of all service IDs in the current account

Returns:

List of all service IDs with membership to current access group.

Return type:

list_of_service_ids

get_trusted_profiles(trusted_profile_set: set)#

Gets all trusted profiles with membership to the current access group.

Parameters:

trusted_profile_set (set) – The set of all trusted profiles in the current account

Returns:

List of all trusted profiles with membership to current access group.

Return type:

list_of_trusted_profles

get_users(user_id_set: set)#

Gets all users with membership to the current access group.

Parameters:

user_id_set (set) – The set of all users in the current account

Returns:

List of all users with membership to current access group.

Return type:

list_of_users

remove_members_from_access_group(members: UserProfile | ServiceID | TrustedProfile | list[UserProfile | ServiceID | TrustedProfile])#

Removes members from access group.

Parameters:

members (UserProfile | ServiceID | TrustedProfile | list[UserProfile | ServiceID | TrustedProfile]) – List of members to remove from access group.

Return type:

Response

Returns:

A HTTP response.

class ibm_watsonx_data_integration.cpd_models.access_groups_model.AccessGroups(platform: Platform | None = None)#

Bases: CollectionModel[AccessGroup]

Collection of AccessGroup instances.

The __init__ of the AccessGroups class.

Parameters:

platform (Optional[Platform], default: None) – The Platform object.

Access Groups On Prem#

Module containing AccessGroupOnPrem Model.

pydantic model ibm_watsonx_data_integration.cpd_models.access_groups_model_on_prem.AccessGroupOnPrem#

Bases: BaseModel, ProjectCollaboratable

On-prem Access Group model.

The __init__ of the AccessGroup class.

field created_at: str [Required]#
field created_by: str [Required]#
field description: str = None#
field group_id: int | str [Required]#
field members_count: int = 0#
field misc: dict [Optional]#
field name: str [Required]#
field permissions: list[str] [Optional]#
field roles: list[dict] [Optional]#
field updated_at: str [Required]#
add_members_to_access_group(users: list[object])#

Adds members to an AccessGroup.

Parameters:

users (list[object]) – The members to be added to the access group.

Return type:

Response

Returns:

A HTTP response.

remove_member_from_access_group(user: object)#

Removes members from access group.

Parameters:

user (object) – Member to be removed from access group.

Return type:

Response

Returns:

A HTTP response.

class ibm_watsonx_data_integration.cpd_models.access_groups_model_on_prem.AccessGroupsOnPrem(platform: Platform | None = None)#

Bases: CollectionModel[AccessGroupOnPrem]

Collection of AccessGroupOnPrem instances.

The __init__ of the AccessGroupsOnPrem class.

Account#

Module containing Account Model.

pydantic model ibm_watsonx_data_integration.cpd_models.account_model.Account#

Bases: BaseModel

Model representing an account, including metadata and flattened entity details.

The __init__ of the Account.

Parameters:

  • platform (Optional[Platform], default: None) – The Platform object.

  • account_json (dict) – The JSON for the Account.

field account_id: str [Optional] (alias 'customer_id')#
field account_type: str [Required] (alias 'type')#
field billing_country_code: str [Required]#
field bluemix_subscriptions: list[BluemixSubscription] [Required]#
field configuration_id: str | None = None#
field country_code: str [Required]#
field currency_code: str [Required]#
field current_billing_system: str [Required]#
field iam_id: str [Required] (alias 'owner_iam_id')#
field isIBMer: bool [Required]#
field linkages: list[dict] [Required]#
field metadata: AccountMetadata [Required]#
field name: str [Required]#
field onboarded: int [Required]#
field organizations_region: list[Any] [Optional]#
field origin: str [Required]#
field owner: str | None = ''#
field owner_unique_id: str [Required]#
field owner_userid: str [Required]#
field state: str [Required]#
field subscription_id: str [Required]#
field tags: list[Any] [Optional]#
field team_directory_enabled: bool [Required]#
field terms_and_conditions: dict [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

pydantic model ibm_watsonx_data_integration.cpd_models.account_model.AccountMetadata#

Bases: BaseModel

Model representing metadata for an account.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field created_at: str [Required]#
field created_by: str | None = None#
field guid: str [Required]#
field update_comments: str [Required]#
field updated_at: str [Required]#
field updated_by: str [Required]#
field url: str [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

class ibm_watsonx_data_integration.cpd_models.account_model.Accounts(platform: Platform)#

Bases: CollectionModel[Account]

Collection of Account instances.

The __init__ of the Accounts class.

Parameters:

platform (Platform) – The Platform object.

pydantic model ibm_watsonx_data_integration.cpd_models.account_model.BluemixSubscription#

Bases: BaseModel

Model representing a Bluemix subscription with payment details.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field billing_system: str [Required]#
field category: str [Required]#
field current_state_timestamp: str | None = None#
field history: list[Any] | None = None#
field part_number: str [Required]#
field payment_method: dict [Required]#
field state: str [Required]#
field subscriptionTags: list[Any] [Required]#
field subscription_id: str [Required]#
field type: str [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

property payment_ended: str | None#

Access the ended in payment_method.

Returns:

A ended payment type.

property payment_started: str | None#

Access the started in payment_method.

Returns:

A started payment type.

property payment_type: str | None#

Access the type in payment_method.

Returns:

A payment type.

Account Owner#

Module containing AccountOwner Model.

pydantic model ibm_watsonx_data_integration.cpd_models.account_owner_model.AccountOwner#

Bases: BaseModel

Model representing the owner of an account.

The __init__ of the Account Owner.

Parameters:

  • platform (Optional[Platform], default: None) – The Platform object.

  • account_owner_json (dict) – The JSON for the Account Owner.

field email: str [Required]#
field family_name: str [Required] (alias 'familyName')#
field given_name: str [Required] (alias 'givenName')#
field user_name: str [Required] (alias 'userName')#

Authentication#

Authentication module.

class ibm_watsonx_data_integration.common.auth.BaseAuthenticator(disable_ssl_verification: bool = False, proxies: dict[str, str] | None = None)#

Bases: CodeGeneratable, ABC

Base Authenticator classes to be inherited by other authenticators.

Initializes BaseAuthenticator.

Parameters:

  • disable_ssl_verification (bool, default: False) – Whether to disable SSL Verification.

  • proxies (Optional[dict[str, str]], default: None) – Optional proxy configuration for requests.

abstract get_authorization_header()#

Returns the token formatted to be plugged into the Authorization header of a request.

Return type:

str

Returns:

The value to be plugged into the Authorization header of a request.

class ibm_watsonx_data_integration.common.auth.BearerTokenAuthenticator(bearer_token: str, disable_ssl_verification: bool = False, proxies: dict[str, str] | None = None)#

Bases: BaseAuthenticator

Authentication class for directly using a bearer token.

bearer_token#

bearer token to use for authentication.

disable_ssl_verification#

Whether to disable SSL Verification.

__init__ for the class.

Parameters:

  • bearer_token (str) – bearer token to use for authentication.

  • disable_ssl_verification (bool, default: False) – Whether to disable SSL Verification.

  • proxies (Optional[dict[str, str]], default: None) – Optional proxy configuration for requests.

get_authorization_header()#

Returns the token formatted to be plugged into the Authorization header of a request.

Return type:

str

Returns:

The value to be plugged into the Authorization header of a request.

class ibm_watsonx_data_integration.common.auth.IAMAuthenticator(api_key: str, base_auth_url: str = 'https://cloud.ibm.com', disable_ssl_verification: bool = False, proxies: dict[str, str] | None = None)#

Bases: BaseAuthenticator

Authenticator class to authenticate using an IBM Cloud IBM API Key.

api_key#

API key being used to authenticate.

token_url#

URL being used to authenticate against.

iam_token#

The token generated using the api_key and url.

token_expiry_time#

UNIX time in seconds for when the token will expire.

Initializes IAM Authenticator.

Parameters:

  • api_key (str) – The API key to be used for authentication.

  • base_auth_url (str, default: 'https://cloud.ibm.com') – The base URL of the IBM cloud instance to be used for authentication.

  • disable_ssl_verification (bool, default: False) – Whether to disable SSL Verification.

  • proxies (Optional[dict[str, str]], default: None) – Optional proxy configuration for requests.

Raises:

  • InvalidApiKeyError – If api_key is not of type str, or is an empty str.

  • InvalidUrlError – If base_auth_url is not of type str, or is an empty str.

  • requests.exceptions.HTTPError – If there is an error getting a valid token.

get_authorization_header()#

Returns the token formatted to be plugged into the Authorization header of a request.

Return type:

str

Returns:

The value to be plugged into the Authorization header of a request.

get_token()#

Get existing token, or request a new one if the current token is expired.

Return type:

str

Returns:

The current bearer token used for auth.

request_token()#

Request a token from the servers using the API key.

Raises:

  • InvalidApiKeyError – If api_key is invalid.

  • InvalidUrlError – If token_url is invalid.

  • requests.exceptions.HTTPError – If there is an error getting a valid token.

Return type:

None

class ibm_watsonx_data_integration.common.auth.ICP4DAuthenticator(url: str, username: str, password: str | None = None, zen_api_key: str | None = None, disable_ssl_verification: bool = False, proxies: dict[str, str] | None = None)#

Bases: BaseAuthenticator

Authenticator for on-prem cpd using IAM or API credentials.

url#

CPD instance URL.

username#

Username of the user trying to authenticate.

password#

Password of the user.

zen_api_key#

ZenApiKey of the user.

disable_ssl_verification#

Whether to disable SSL verification.

Init for ICP4D Authenticator.

Parameters:

  • url (str) – CPD instance URL

  • username (str) – Username of the user trying to authenticate

  • password (Optional[str], default: None) – Password of the user, if provided zen_api_key should be left None

  • zen_api_key (Optional[str], default: None) – ZenApiKey of the user, if provided password should be left None

  • disable_ssl_verification (bool, default: False) – Whether to disable SSL verification. Default: False.

  • proxies (Optional[dict[str, str]], default: None) – Optional proxy configuration for requests.

get_authorization_header()#

Get the authorization header to be used for requests.

Return type:

str

get_token()#

Get existing token, or request a new one if the current token is expired.

Return type:

str

Returns:

The current bearer token used for auth.

request_token()#

Requests a new token from the endpoint.

Return type:

None

class ibm_watsonx_data_integration.common.auth.ZenApiKeyAuthenticator(username: str, zen_api_key: str, disable_ssl_verification: bool = False, proxies: dict[str, str] | None = None)#

Bases: BaseAuthenticator

Authentication class for using a Zen API Key.

username#

Username to use for authentication.

zen_api_key#

Zen API Key to use for authentication.

disable_ssl_verification#

Whether to disable SSL Verification.

__init__ for the class.

Parameters:

  • username (str) – Username to use for authentication.

  • zen_api_key (str) – Zen API Key to use for authentication.

  • disable_ssl_verification (bool, default: False) – Whether to disable SSL Verification.

  • proxies (Optional[dict[str, str]], default: None) – Optional proxy configuration for requests.

encode()#

Encodes the username and Zen API key in base64.

Returns:

<zen_api_key>”.

Return type:

The base64-encoded string “<username>

get_authorization_header()#

Returns the token formatted to be plugged into the Authorization header of a request.

Return type:

str

Returns:

The value to be plugged into the Authorization header of a request.

Batch Connectors#

This module defines configuration or the Amazon RDS for PostgreSQL stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.amazon_postgresql.amazon_postgresql#

Bases: BaseStage

Properties for the Amazon RDS for PostgreSQL stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_proccode_column: bool | None = False#
field batch_size: int | None = 2000#
field before_after_after_node_fail_on_error: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field byte_limit: str | None = None#
field call_each_row: bool | None = True#
field call_statement: str | None = None#
field connection: AmazonPostgresqlConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "auto_discovery": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "host": null,     "login_timeout": null,     "password": null,     "port": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_user": null,     "query_timeout": 300,     "retry_limit": 2,     "ssl": true,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_statement: str | None = None#
field default_max_string_binary_precision: int | None = 20000#
field defer_credentials: bool | None = False#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field execmode: Execmode | None = Execmode.default_par#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field forward_row_data: bool | None = False#
field generate_unicode_columns: bool | None = False#
field has_ref_output: bool | None = False#
field has_reject_output: bool | None = False#
field input_count: int | None = 0#
field key_column_names: str | None = None#
field login_timeout: int | None = None#
field lookup_type: LookupType | None = LookupType.empty#
field output_count: int | None = 0#
field preserve: Preserve | None = Preserve.default_propagate#
field procedure_name: str | None = None#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field rcp: bool | None = True#
field read_after_sql_node_statements_from_file: bool | None = False (alias 'before_after.after_node.read_from_file_after_sql_node')#
field read_after_sql_statements_from_file: bool | None = False (alias 'before_after.after.read_from_file_after_sql')#
field read_before_sql_node_statement_from_file: bool | None = False (alias 'before_after.before_node.read_from_file_before_sql_node')#
field read_before_sql_statements_from_file: bool | None = False (alias 'before_after.before.read_from_file_before_sql')#
field read_mode: ReadMode | None = ReadMode.general#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool = False#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field select_statement_read_from_file_select: bool | None = False (alias 'select_statement.read_from_file_select')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field update_statement: str | None = None#
field update_statement_read_from_file_update: bool | None = False (alias 'update_statement.read_from_file_update')#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/postgresql-amazon.svg'#
label: ClassVar[str] = 'Amazon RDS for PostgreSQL'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'postgresql-amazon'#

This module defines configuration or the Amazon Redshift stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.amazon_redshift.amazon_redshift#

Bases: BaseStage

Properties for the Amazon Redshift stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field access_key: str = None#
field batch_size: int | None = 2000#
field bucket: str = None#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: AmazonRedshiftConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "extended_column_metadata": null,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "port_is_ssl_enabled": true,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "time_type": "timestamp",     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disable_partitioning: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field file_name: str = None#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field node_count: int | None = 1#
field node_number: int | None = 0#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field region: str = None#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field secret_key: str = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/RedshiftPX.svg'#
label: ClassVar[str] = 'Amazon Redshift'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'RedshiftPX'#

This module defines configuration or the Amazon RDS for Oracle stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.amazonrds_oracle.amazonrds_oracle#

Bases: BaseStage

Properties for the Amazon RDS for Oracle stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field conn_impersonate_user: str | None = None#
field conn_impersonate_user_password: str | None = None#
field connection: AmazonrdsOracleConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "alternate_servers": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "conn_impersonate_user": null,     "conn_impersonate_user_password": null,     "connection_mode": null,     "defer_credentials": false,     "failover_mode": null,     "hostname_or_ip_address": null,     "load_balancing": null,     "metadata_discovery": "no_remarks",     "number_type": null,     "password": null,     "port": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_username": null,     "retry_limit": 2,     "service_name": null,     "database_sid": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field number_type: NumberType | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/oracle-amazon.svg'#
label: ClassVar[str] = 'Amazon RDS for Oracle'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'oracle-amazon'#

This module defines configuration or the Amazon S3 stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.amazons3.amazons3#

Bases: BaseStage

Properties for the Amazon S3 stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field append_unique_identifier: bool | None = False (alias 'append_uid')#
field bucket: str | None = None#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: Amazons3Conn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "access_key": null,     "bucket": null,     "cluster_access_token": null,     "cluster_user_name": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_username": null,     "region": null,     "secret_key": null,     "url": null,     "vaulted_properties": null,     "ds_host": null,     "ds_port": null,     "authentication_method": null,     "defer_credentials": false,     "duration_seconds": null,     "external_id": null,     "role_arn": null,     "role_session_name": null,     "session_token": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_bucket: bool | None = False#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field delete_bucket: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_codec_avro: DSCodecAvro | None = None (alias '_codec_avro')#
field ds_codec_csv: DSCodecCsv | None = None (alias '_codec_csv')#
field ds_codec_delimited: DSCodecDelimited | None = None (alias '_codec_delimited')#
field ds_create_bucket_append_u_u_i_d: bool | None = False (alias '_create_bucket._append_u_u_i_d')#
field ds_exclude_files: str | None = None (alias '_exclude_files')#
field ds_file_attributes: bool | None = None (alias '_file_attributes')#
field ds_file_attributes_content_type: str | None = None (alias '_file_attributes._content_type')#
field ds_file_attributes_encryption: DSFileAttributesEncryption | None = DSFileAttributesEncryption.none (alias '_file_attributes._encryption')#
field ds_file_attributes_life_cycle_rule: bool | None = False (alias '_file_attributes._life_cycle_rule')#
field ds_file_attributes_life_cycle_rule_expiration: bool | None = False (alias '_file_attributes._life_cycle_rule._expiration')#
field ds_file_attributes_life_cycle_rule_expiration_expiration_date: str = None (alias '_file_attributes._life_cycle_rule._expiration._expiration_date')#
field ds_file_attributes_life_cycle_rule_expiration_expiration_duration: int = None (alias '_file_attributes._life_cycle_rule._expiration._expiration_duration')#
field ds_file_attributes_life_cycle_rule_l_c_rule_format: DSFileAttributesLifeCycleRuleLCRuleFormat | None = DSFileAttributesLifeCycleRuleLCRuleFormat.days_from_creation (alias '_file_attributes._life_cycle_rule._l_c_rule_format')#
field ds_file_attributes_life_cycle_rule_l_c_rule_scope: DSFileAttributesLifeCycleRuleLCRuleScope | None = DSFileAttributesLifeCycleRuleLCRuleScope.file (alias '_file_attributes._life_cycle_rule._l_c_rule_scope')#
field ds_file_attributes_life_cycle_rule_transition: bool | None = False (alias '_file_attributes._life_cycle_rule._transition')#
field ds_file_attributes_life_cycle_rule_transition_transition_date: str = None (alias '_file_attributes._life_cycle_rule._transition._transition_date')#
field ds_file_attributes_life_cycle_rule_transition_transition_duration: int = None (alias '_file_attributes._life_cycle_rule._transition._transition_duration')#
field ds_file_attributes_storage_class: DSFileAttributesStorageClass | None = DSFileAttributesStorageClass.standard_class (alias '_file_attributes._storage_class')#
field ds_file_attributes_user_metadata: str | None = None (alias '_file_attributes._user_metadata')#
field ds_file_exists: DSFileExists | None = DSFileExists.overwrite (alias '_file_exists')#
field ds_file_format: DSFileFormat | None = DSFileFormat.delimited (alias '_file_format')#
field ds_file_format_avro_source: bool | None = None (alias '_file_format._avro_source')#
field ds_file_format_avro_source_output_j_s_o_n: bool | None = False (alias '_file_format._avro_source._output_j_s_o_n')#
field ds_file_format_avrotarget: bool | None = None (alias '_file_format._avrotarget')#
field ds_file_format_avrotarget_avro_array_keys: str | None = None (alias '_file_format._avrotarget._avro_array_keys')#
field ds_file_format_avrotarget_avro_schema: str | None = None (alias '_file_format._avrotarget._avro_schema')#
field ds_file_format_avrotarget_input_j_s_o_n: bool | None = False (alias '_file_format._avrotarget._input_j_s_o_n')#
field ds_file_format_avrotarget_use_schema: bool | None = False (alias '_file_format._avrotarget._use_schema')#
field ds_file_format_delimited_syntax: bool | None = None (alias '_file_format._delimited_syntax')#
field ds_file_format_delimited_syntax_data_format: DSFileFormatDelimitedSyntaxDataFormat | None = DSFileFormatDelimitedSyntaxDataFormat.test (alias '_file_format._delimited_syntax._data_format')#
field ds_file_format_delimited_syntax_encoding_output_b_o_m: bool | None = False (alias '_file_format._delimited_syntax._encoding._output_b_o_m')#
field ds_file_format_delimited_syntax_escape: str | None = None (alias '_file_format._delimited_syntax._escape')#
field ds_file_format_delimited_syntax_field_delimiter: str | None = ',' (alias '_file_format._delimited_syntax._field_delimiter')#
field ds_file_format_delimited_syntax_record_def: DSFileFormatDelimitedSyntaxRecordDef | None = DSFileFormatDelimitedSyntaxRecordDef.none (alias '_file_format._delimited_syntax._record_def')#
field ds_file_format_delimited_syntax_record_def_record_def_source: str | None = None (alias '_file_format._delimited_syntax._record_def._record_def_source')#
field ds_file_format_delimited_syntax_row_delimiter: str | None = '<NL>' (alias '_file_format._delimited_syntax._row_delimiter')#
field ds_file_format_delimited_syntax_trace_file: str | None = None (alias '_file_format._delimited_syntax._trace_file')#
field ds_file_format_o_r_c_source: bool | None = None (alias '_file_format._o_r_c_source')#
field ds_file_format_o_r_c_source_temp_staging_area: str | None = None (alias '_file_format._o_r_c_source._temp_staging_area')#
field ds_file_format_o_r_c_target: bool | None = None (alias '_file_format._o_r_c_target')#
field ds_file_format_o_r_c_target_orc_buffer_size: int | None = 10000 (alias '_file_format._o_r_c_target._orc_buffer_size')#
field ds_file_format_o_r_c_target_orc_compress: DSFileFormatORCTargetOrcCompress | None = DSFileFormatORCTargetOrcCompress.gzip (alias '_file_format._o_r_c_target._orc_compress')#
field ds_file_format_o_r_c_target_orc_stripe_size: int | None = 100000 (alias '_file_format._o_r_c_target._orc_stripe_size')#
field ds_file_format_o_r_c_target_temp_staging_area: str | None = None (alias '_file_format._o_r_c_target._temp_staging_area')#
field ds_file_format_parquet_source: bool | None = None (alias '_file_format._parquet_source')#
field ds_file_format_parquet_source_temp_staging_area: str | None = None (alias '_file_format._parquet_source._temp_staging_area')#
field ds_file_format_parquet_target: bool | None = None (alias '_file_format._parquet_target')#
field ds_file_format_parquet_target_parquet_block_size: int | None = 10000000 (alias '_file_format._parquet_target._parquet_block_size')#
field ds_file_format_parquet_target_parquet_compress: DSFileFormatParquetTargetParquetCompress | None = DSFileFormatParquetTargetParquetCompress.snappy (alias '_file_format._parquet_target._parquet_compress')#
field ds_file_format_parquet_target_parquet_page_size: int | None = 10000 (alias '_file_format._parquet_target._parquet_page_size')#
field ds_file_format_parquet_target_temp_staging_area: str | None = None (alias '_file_format._parquet_target._temp_staging_area')#
field ds_filename_column: str | None = None (alias '_filename_column')#
field ds_first_line_header: bool | None = False (alias '_first_line_header')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field ds_log_interval: int | None = None (alias '_log_interval')#
field ds_read_mode: DSReadMode | None = DSReadMode.single_file (alias '_read_mode')#
field ds_recurse: bool | None = True (alias '_recurse')#
field ds_reject_mode: DSRejectMode | None = DSRejectMode.cont (alias '_reject_mode')#
field ds_thread_count: int | None = 5 (alias '_thread_count')#
field ds_use_datastage: bool | None = True (alias '_use_datastage')#
field ds_write_mode: DSWriteMode | None = DSWriteMode.write (alias '_write_mode')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field endpoint_folder: str | None = None (alias 'table_folder_name')#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str | None = None#
field file_size_threshold: int | None = 1#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_avro_as_json: bool | None = None#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_a_file_to_a_row: bool | None = False (alias 'read_file_to_row')#
field read_mode: ReadMode | None = ReadMode.read_single#
field read_part_size: int | None = None#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field table_action: TableAction | None = TableAction.append#
field table_data_file_compression_codec: TableDataFileCompressionCodec | None = None#
field table_data_file_format: TableDataFileFormat | None = TableDataFileFormat.avro#
field table_format: TableFormat | None = None#
field table_name: str | None = None#
field table_namespace: str | None = None#
field the_cache_expiration: str | None = None (alias 'table_partition_cache_expiration')#
field the_cache_size: int | None = None (alias 'table_partition_cache_size')#
field the_data_path: str | None = None (alias 'table_data_path')#
field the_partition_columns: str | None = None (alias 'table_partition_columns')#
field the_partition_paths: str | None = None (alias 'table_partition_path')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field write_part_size: int | None = None#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/AmazonS3PX.svg'#
label: ClassVar[str] = 'Amazon S3'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'AmazonS3PX'#

This module defines configuration or the Apache HBase stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.apache_hbase.apache_hbase#

Bases: BaseStage

Properties for the Apache HBase stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field autoflush_enabled: bool | None = True (alias 'hbase.write.is_auto_flush')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: ApacheHbaseConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "cas_lite_service_authorization_header": null,     "cluster_access_token": null,     "cluster_user_name": null,     "core_site_xml": null,     "core_site_xml_path": null,     "defer_credentials": false,     "input_method_for_xml_files": "content",     "hadoop_identity": "",     "hbase_site_xml": null,     "hbase_site_xml_path": null,     "hbase_identity": "",     "simple_authentication_sasl_username": null,     "use_cas_lite_service": true,     "use_kerberos_authentication": false,     "krb5_conf_location": null,     "password": "password",     "user_principal_name": "user@REALM",     "ticket_cache_location": null,     "use_keytab": false,     "keytab": "",     "use_ticket_cache": false,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field detect_regions_changes_row_size: int | None = 100000 (alias 'detect_splits_and_merges_group.detection_step_size')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_detecting_regions_changes: bool | None = True (alias 'detect_splits_and_merges_group.detect_splits_and_merges')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field hbase_namespace: str = 'default' (alias 'hbase.namespace')#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field strict_type_checking: bool | None = True (alias 'hbase.read.strict_type_checking')#
field target_table: str = 'table' (alias 'hbase.table')#
field type_of_row_keys_in_the_target_table: TypeOfRowKeysInTheTargetTable | None = TypeOfRowKeysInTheTargetTable.uniform_byte_arrays (alias 'row_key_type')#
field unique: bool | None = None (alias 'part_unique')#
field use_cas_lite_service: bool | None = True (alias 'use_cas_lite')#
field use_parallel_read: bool | None = True#
field write_mode: HbaseWriteMode | None = HbaseWriteMode.put (alias 'hbase.write.mode')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/HBaseConnectorPX.svg'#
label: ClassVar[str] = 'Apache HBase'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'HBaseConnectorPX'#

This module defines configuration or the Apache Hive stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.apache_hive.apache_hive#

Bases: BaseStage

Properties for the Apache Hive stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: ApacheHiveConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "ds_url": "",     "cluster_access_token": null,     "cluster_user_name": null,     "vaulted_properties": null,     "authentication_method": "password",     "discover_data_assets": null,     "database": null,     "defer_credentials": false,     "host": null,     "http_path": null,     "impersonate_user": null,     "kerberos_sso": null,     "kerberos_sso_keytab": null,     "kerberos_sso_principal": null,     "keytab_file": null,     "login_timeout": null,     "password": null,     "port": null,     "row_limit_support": true,     "service_principal_name": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "user_principal_name": null,     "username": null,     "use_zoo_keeper_discovery": false,     "zoo_keeper_namespace": null,     "comma_separated_list_of_alternative_servers": null,     "additional_properties": null,     "login_config_name": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_before_after: bool | None = False (alias '_before_after')#
field ds_before_after_after_sql: str | None = None (alias '_before_after._after_sql')#
field ds_before_after_after_sql_fail_on_error: bool | None = True (alias '_before_after._after_sql._fail_on_error')#
field ds_before_after_after_sql_node: str | None = None (alias '_before_after._after_sql_node')#
field ds_before_after_after_sql_node_fail_on_error: bool | None = True (alias '_before_after._after_sql_node._fail_on_error')#
field ds_before_after_after_sql_node_read_from_file_after_sql_node: bool | None = False (alias '_before_after._after_sql_node._read_from_file_after_sql_node')#
field ds_before_after_after_sql_read_from_file_after_sql: bool | None = False (alias '_before_after._after_sql._read_from_file_after_sql')#
field ds_before_after_before_sql: str | None = None (alias '_before_after._before_sql')#
field ds_before_after_before_sql_fail_on_error: bool | None = True (alias '_before_after._before_sql._fail_on_error')#
field ds_before_after_before_sql_node: str | None = None (alias '_before_after._before_sql_node')#
field ds_before_after_before_sql_node_fail_on_error: bool | None = True (alias '_before_after._before_sql_node._fail_on_error')#
field ds_before_after_before_sql_node_read_from_file_before_sql_node: bool | None = False (alias '_before_after._before_sql_node._read_from_file_before_sql_node')#
field ds_before_after_before_sql_read_from_file_before_sql: bool | None = False (alias '_before_after._before_sql._read_from_file_before_sql')#
field ds_custom_statements: str | None = None (alias '_custom_statements')#
field ds_custom_statements_read_from_file_custom: bool | None = False (alias '_custom_statements._read_from_file_custom')#
field ds_delete_statement: str = None (alias '_delete_statement')#
field ds_delete_statement_read_from_file_delete: bool | None = False (alias '_delete_statement._read_from_file_delete')#
field ds_enable_partitioned_reads: bool | None = False (alias '_enable_partitioned_reads')#
field ds_enable_partitioned_reads_column_name: str | None = None (alias '_enable_partitioned_reads._column_name')#
field ds_enable_partitioned_reads_partition_method: DSEnablePartitionedReadsPartitionMethod | None = DSEnablePartitionedReadsPartitionMethod._hive_partition (alias '_enable_partitioned_reads._partition_method')#
field ds_enable_partitioned_reads_table_name: str | None = None (alias '_enable_partitioned_reads._table_name')#
field ds_enable_partitioned_write: bool | None = False (alias '_enable_partitioned_write')#
field ds_enable_quoted_ids: bool | None = False (alias '_enable_quoted_ids')#
field ds_generate_sql: bool | None = True (alias '_generate_sql')#
field ds_hive_parameters: str | None = None (alias '_hive_parameters')#
field ds_hive_parameters_fail_on_error: bool | None = False (alias '_hive_parameters._fail_on_error')#
field ds_insert_statement: str = None (alias '_insert_statement')#
field ds_insert_statement_read_from_file_insert: bool | None = False (alias '_insert_statement._read_from_file_insert')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field ds_limit_rows_limit: int | None = None (alias '_limit_rows._limit')#
field ds_read_mode: DSReadMode | None = DSReadMode._select (alias '_read_mode')#
field ds_record_ordering: DSRecordOrdering | None = DSRecordOrdering.zero (alias '_record_ordering')#
field ds_record_ordering_key_column: list | None = [] (alias '_record_ordering._key_column')#
field ds_select_statement: str = None (alias '_select_statement')#
field ds_select_statement_other_clause: str | None = None (alias '_select_statement._other_clause')#
field ds_select_statement_read_from_file_select: bool | None = False (alias '_select_statement._read_from_file_select')#
field ds_select_statement_where_clause: str | None = None (alias '_select_statement._where_clause')#
field ds_session_array_size: int | None = 1 (alias '_session._array_size')#
field ds_session_batch_size: int | None = 2000 (alias '_session._batch_size')#
field ds_session_character_set_for_non_unicode_columns: DSSessionCharacterSetForNonUnicodeColumns | None = DSSessionCharacterSetForNonUnicodeColumns._default (alias '_session._character_set_for_non_unicode_columns')#
field ds_session_character_set_for_non_unicode_columns_character_set_name: str = None (alias '_session._character_set_for_non_unicode_columns._character_set_name')#
field ds_session_default_length_for_columns: int | None = 200 (alias '_session._default_length_for_columns')#
field ds_session_default_length_for_long_columns: int | None = 20000 (alias '_session._default_length_for_long_columns')#
field ds_session_drop_unmatched_fields: bool | None = False (alias '_session._drop_unmatched_fields')#
field ds_session_fail_on_truncation: bool | None = True (alias '_session._fail_on_truncation')#
field ds_session_fetch_size: int | None = 0 (alias '_session._fetch_size')#
field ds_session_generate_all_columns_as_unicode: bool | None = False (alias '_session._generate_all_columns_as_unicode')#
field ds_session_keep_conductor_connection_alive: bool | None = True (alias '_session._keep_conductor_connection_alive')#
field ds_session_report_schema_mismatch: bool | None = False (alias '_session._report_schema_mismatch')#
field ds_table_action: DSTableAction = DSTableAction._append (alias '_table_action')#
field ds_table_action_generate_create_statement: bool | None = True (alias '_table_action._generate_create_statement')#
field ds_table_action_generate_create_statement_create_statement: str = None (alias '_table_action._generate_create_statement._create_statement')#
field ds_table_action_generate_create_statement_fail_on_error: bool | None = True (alias '_table_action._generate_create_statement._fail_on_error')#
field ds_table_action_generate_create_statement_row_format: DSTableActionGenerateCreateStatementRowFormat | None = DSTableActionGenerateCreateStatementRowFormat._storage_format (alias '_table_action._generate_create_statement._row_format')#
field ds_table_action_generate_create_statement_row_format_field_terminator: str | None = None (alias '_table_action._generate_create_statement._row_format._field_terminator')#
field ds_table_action_generate_create_statement_row_format_line_terminator: str | None = None (alias '_table_action._generate_create_statement._row_format._line_terminator')#
field ds_table_action_generate_create_statement_row_format_serde_library: str = None (alias '_table_action._generate_create_statement._row_format._serde_library')#
field ds_table_action_generate_create_statement_storage_format: DSTableActionGenerateCreateStatementStorageFormat | None = DSTableActionGenerateCreateStatementStorageFormat._text_file (alias '_table_action._generate_create_statement._storage_format')#
field ds_table_action_generate_create_statement_table_location: str | None = None (alias '_table_action._generate_create_statement._table_location')#
field ds_table_action_generate_drop_statement: bool | None = True (alias '_table_action._generate_drop_statement')#
field ds_table_action_generate_drop_statement_drop_statement: str = None (alias '_table_action._generate_drop_statement._drop_statement')#
field ds_table_action_generate_drop_statement_fail_on_error: bool | None = False (alias '_table_action._generate_drop_statement._fail_on_error')#
field ds_table_action_generate_truncate_statement: bool | None = True (alias '_table_action._generate_truncate_statement')#
field ds_table_action_generate_truncate_statement_fail_on_error: bool | None = True (alias '_table_action._generate_truncate_statement._fail_on_error')#
field ds_table_action_generate_truncate_statement_truncate_statement: str = None (alias '_table_action._generate_truncate_statement._truncate_statement')#
field ds_table_action_table_action_first: bool | None = True (alias '_table_action._table_action_first')#
field ds_table_name: str = None (alias '_table_name')#
field ds_transaction_auto_commit_mode: DSTransactionAutoCommitMode | None = DSTransactionAutoCommitMode._disable (alias '_transaction._auto_commit_mode')#
field ds_transaction_begin_end: bool | None = False (alias '_transaction._begin_end')#
field ds_transaction_begin_end_begin_sql: str | None = None (alias '_transaction._begin_end._begin_sql')#
field ds_transaction_begin_end_end_sql: str | None = None (alias '_transaction._begin_end._end_sql')#
field ds_transaction_begin_end_run_end_sql_if_no_records_processed: bool | None = False (alias '_transaction._begin_end._run_end_sql_if_no_records_processed')#
field ds_transaction_end_of_wave: DSTransactionEndOfWave | None = DSTransactionEndOfWave._no (alias '_transaction._end_of_wave')#
field ds_transaction_isolation_level: DSTransactionIsolationLevel | None = DSTransactionIsolationLevel._default (alias '_transaction._isolation_level')#
field ds_transaction_record_count: int | None = 2000 (alias '_transaction._record_count')#
field ds_update_statement: str = None (alias '_update_statement')#
field ds_update_statement_read_from_file_update: bool | None = False (alias '_update_statement._read_from_file_update')#
field ds_url: str | None = '' (alias '_url')#
field ds_use_datastage: bool | None = True (alias '_use_datastage')#
field ds_write_mode: DSWriteMode | None = DSWriteMode._insert (alias '_write_mode')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field file_format: FileFormat | None = FileFormat.delimited#
field file_name: str | None = None#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/HiveConnectorPX.svg'#
label: ClassVar[str] = 'Apache Hive'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'HiveConnectorPX'#

This module defines configuration or the Apache Kafka stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.apache_kafka.apache_kafka#

Bases: BaseStage

Properties for the Apache Kafka stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field advanced_kafka_config_options: str | None = None#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field conn_registry_key_chain_pem: str | None = None#
field conn_registry_key_password: str = None#
field conn_registry_key_pem: str | None = None#
field conn_registry_keystore_location: str | None = None#
field conn_registry_keystore_password: str = None#
field conn_registry_keytab: str | None = None#
field conn_registry_password: str = None#
field conn_registry_principal_name: str | None = None#
field conn_registry_truststore_pem: str | None = None#
field conn_registry_username: str | None = None#
field conn_schema_registry_authentication: ConnSchemaRegistryAuthentication | None = ConnSchemaRegistryAuthentication.none#
field conn_schema_registry_secure: ConnSchemaRegistrySecure | None = ConnSchemaRegistrySecure.none#
field connection: ApacheKafkaConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "key_chain_pem": null,     "key_password": null,     "key_pem": null,     "keystore_location": null,     "keystore_password": null,     "keytab": null,     "legacy_registry_security_conf": null,     "legacy_security_conf": null,     "additional_properties": null,     "client_id": null,     "client_secret": null,     "scope": null,     "server_url": null,     "password": null,     "registry_key_chain_pem": null,     "registry_key_password": null,     "registry_key_pem": null,     "registry_keystore_location": null,     "registry_keystore_password": null,     "registry_keytab": null,     "registry_password": null,     "registry_principal_name": null,     "registry_truststore_location": null,     "registry_truststore_password": null,     "registry_truststore_pem": null,     "registry_username": null,     "oauth_bearer_authentication": "SASL_OAUTH2",     "schema_registry_authentication": "none",     "schema_registry_secure": "none",     "schema_registry_type": "confluent",     "registry_url": null,     "secure_connection": "None",     "kafka_server_host_name": null,     "truststore_location": null,     "truststore_password": null,     "truststore_pem": null,     "use_schema_registry_for_message_format": false,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field consumer_group: str | None = None (alias 'consumer_group_name')#
field continuous_mode: bool | None = False#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_advanced_client_logging: bool | None = False (alias '_advanced_client_logging')#
field ds_advanced_kafka_config_options: bool | None = False (alias '_advanced_kafka_config_options')#
field ds_client_logging_level: DSClientLoggingLevel | None = DSClientLoggingLevel.off (alias '_client_logging_level')#
field ds_consumer_group_name: str | None = None (alias '_consumer_group_name')#
field ds_isolation_level: DSIsolationLevel | None = DSIsolationLevel.read_uncommitted (alias '_isolation_level')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field ds_kafka_config_options: str | None = None (alias '_kafka_config_options')#
field ds_key_serializer_type: DSKeySerializerType | None = DSKeySerializerType.string (alias '_key_serializer_type')#
field ds_max_messages: int | None = None (alias '_max_messages')#
field ds_max_poll_records: int | None = 100 (alias '_max_poll_records')#
field ds_reset_policy: DSResetPolicy | None = DSResetPolicy.latest (alias '_reset_policy')#
field ds_start_offset: str | None = None (alias '_start_offset')#
field ds_stop_message: str | None = None (alias '_stop_message')#
field ds_timeout: int | None = None (alias '_timeout')#
field ds_timeout_after_last_message: int | None = 30 (alias '_timeout_after_last_message')#
field ds_use_datastage: bool | None = True (alias '_use_datastage')#
field ds_value_serializer_type: DSValueSerializerType | None = DSValueSerializerType.string (alias '_value_serializer_type')#
field ds_warn_and_error_log: DSWarnAndErrorLog | None = DSWarnAndErrorLog.log_as_informational (alias '_warn_and_error_log')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field end_of_data: bool | None = False#
field end_of_wave: bool | None = False#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field heap_size: int | None = 256#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field isolation_level: IsolationLevel | None = IsolationLevel.read_uncommitted#
field job_timeout_in_seconds: int | None = 30 (alias 'timeout')#
field kafka_client_logging_level: KafkaClientLoggingLevel | None = KafkaClientLoggingLevel.off#
field kafka_config_options: str | None = None#
field kafka_start_offset: str | None = None (alias 'start_offset')#
field kafka_warning_and_error_logs: KafkaWarningAndErrorLogs | None = KafkaWarningAndErrorLogs.log_as_informational#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_serializer: KeySerializer | None = KeySerializer.string (alias 'key_serializer_type')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field messages_read_within_single_request: int | None = 100 (alias 'max_poll_records')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field record_count: int | None = 0#
field registry_trust_location: str | None = None#
field registry_trust_password: str | None = None#
field reset_policy: ResetPolicy | None = ResetPolicy.earliest#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_registry_url: str | None = None (alias 'conn_schema_registry_url')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field stop_message_pattern: str | None = None (alias 'stop_message')#
field time_interval: int | None = 0#
field topic_name: str = None#
field total_number_of_messages: int | None = 100 (alias 'max_messages')#
field unique: bool | None = None (alias 'part_unique')#
field value_serializer: ValueSerializer | None = ValueSerializer.string (alias 'value_serializer_type')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/KafkaConnectorPX.svg'#
label: ClassVar[str] = 'Apache Kafka'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'KafkaConnectorPX'#

This module defines configuration or the Microsoft Azure Blob Storage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.azure_blob_storage.azure_blob_storage#

Bases: BaseStage

Properties for the Microsoft Azure Blob Storage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field append_unique_identifier: bool | None = False (alias 'wave_handling.append_uid')#
field blob_type: BlobType = BlobType.block#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cas_api_key: str = None#
field cas_endpoint: str = None#
field cas_instance_id: str = None#
field catalog_name: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: AzureBlobStorageConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": "connection_string",     "cas_api_key": null,     "cas_endpoint": null,     "cas_instance_id": null,     "catalog_name": null,     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "connection_string": null,     "container": null,     "defer_credentials": false,     "mds_rest_endpoint": null,     "password": null,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "storage_account_url": null,     "tenant_id": null,     "use_watsonx_credential_provider": null,     "username": null,     "vaulted_properties": null,     "ds_host": null,     "ds_port": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_user": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field container_for_temporary_files: str | None = None (alias 'parallel_write.temp_container')#
field container_source: str | None = None#
field create_container: bool | None = False#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_temporary_container: bool | None = False (alias 'parallel_write.create_temp_container')#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field definition_source: str | None = None (alias 'delimited_syntax.record_def.record_def_source')#
field delete_container: bool | None = False#
field delimited_syntax_encoding: str | None = None (alias 'delimited_syntax.encoding')#
field delimited_syntax_escape: str | None = None (alias 'delimited_syntax.escape')#
field delimited_syntax_field_delimiter: str | None = ',' (alias 'delimited_syntax.field_delimiter')#
field delimited_syntax_field_formats_date_format: str | None = None (alias 'delimited_syntax.field_formats.date_format')#
field delimited_syntax_field_formats_decimal_format: str | None = None (alias 'delimited_syntax.field_formats.decimal_format')#
field delimited_syntax_field_formats_time_format: str | None = None (alias 'delimited_syntax.field_formats.time_format')#
field delimited_syntax_field_formats_timestamp_format: str | None = None (alias 'delimited_syntax.field_formats.timestamp_format')#
field delimited_syntax_null_value: str | None = None (alias 'delimited_syntax.null_value')#
field delimited_syntax_row_delimiter: str | None = '<NL>' (alias 'delimited_syntax.row_delimiter')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_container: str | None = None (alias '_container')#
field ds_create_container: bool | None = False (alias '_create_container')#
field ds_file_format: DSFileFormat | None = DSFileFormat.delimited (alias '_file_format')#
field ds_file_name: str = None (alias '_file_name')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field ds_read_mode: DSReadMode = DSReadMode.read_single_file (alias '_read_mode')#
field ds_use_datastage: bool | None = True (alias '_use_datastage')#
field ds_write_mode: DSWriteMode = DSWriteMode.write (alias '_write_mode')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_parallel_write: bool | None = False (alias 'parallel_write')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field endpoint_folder: str | None = None (alias 'table_folder_name')#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_exists: FileExists | None = FileExists.overwrite_file#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field file_name_column: str | None = None (alias 'filename_column')#
field file_name_source: str = None#
field file_size_threshold: int | None = 1 (alias 'wave_handling.file_size_threshold')#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field first_row_is_header: bool | None = False (alias 'delimited_syntax.header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field hide: bool | None = False#
field include_byte_order_mark_for_unicode_encoding: bool | None = False (alias 'delimited_syntax.encoding.output_bom')#
field include_child_folders: bool | None = True (alias 'recurse')#
field include_data_types: bool | None = False (alias 'delimited_syntax.header.include_types')#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field mds_rest_endpoint: str | None = None#
field names_as_labels: bool | None = None#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proxy: bool | None = False#
field proxy_host: str = None#
field proxy_password: str | None = None#
field proxy_port: int = None#
field proxy_user: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quotation_mark: DelimitedSyntaxQuotes | None = DelimitedSyntaxQuotes.none (alias 'delimited_syntax.quotes')#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field read_part_size: int | None = None#
field record_definition: DelimitedSyntaxRecordDef | None = DelimitedSyntaxRecordDef.none (alias 'delimited_syntax.record_def')#
field record_limit: int | None = None (alias 'delimited_syntax.record_limit')#
field reject_mode: RejectMode | None = RejectMode.cont#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field ssl_certificate: str | None = None#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field table_action: TableAction | None = TableAction.append#
field table_data_file_compression_codec: TableDataFileCompressionCodec | None = None#
field table_data_file_format: TableDataFileFormat | None = TableDataFileFormat.avro#
field table_format: TableFormat | None = None#
field table_name: str | None = None#
field table_namespace: str | None = None#
field the_cache_expiration: str | None = None (alias 'table_partition_cache_expiration')#
field the_cache_size: int | None = None (alias 'table_partition_cache_size')#
field the_data_path: str | None = None (alias 'table_data_path')#
field the_partition_columns: str | None = None (alias 'table_partition_columns')#
field the_partition_paths: str | None = None (alias 'table_partition_path')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field use_watsonx_credential_provider: bool | None = None#
field validate_ssl_certificate: bool | None = None (alias 'ssl_certificate_validation')#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field write_part_size: int | None = None#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/azureblobstorage.svg'#
label: ClassVar[str] = 'Microsoft Azure Blob Storage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'azureblobstorage'#

This module defines configuration or the Microsoft Azure Cosmos DB stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.azure_cosmos.azure_cosmos#

Bases: BaseStage

Properties for the Microsoft Azure Cosmos DB stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field collection: str = None#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: AzureCosmosConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": "master_key",     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "hostname": null,     "master_key": null,     "password": null,     "port": 443,     "tenant_id": null,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_collection: bool | None = None#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field database: str = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field file_name: str | None = None#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field input_count: int | None = 0#
field input_format: InputFormat | None = InputFormat.relational#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field offer_throughput: int | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_key_path: str | None = None#
field partition_key_value: str | None = None#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field unique: bool | None = None (alias 'part_unique')#
field write_mode: WriteMode | None = WriteMode.write#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/cosmos.svg'#
label: ClassVar[str] = 'Microsoft Azure Cosmos DB'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'cosmos'#

This module defines configuration or the Microsoft Azure Databricks stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.azure_databricks.azure_databricks#

Bases: BaseStage

Properties for the Microsoft Azure Databricks stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field catalog_name: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: AzureDatabricksConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "microsoft_entra_id_token": null,     "hostname_or_ip_address": null,     "http_path": null,     "client_id_of_service_principal": null,     "client_secret_of_service_principal": null,     "password": null,     "port": null,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/databricks.svg'#
label: ClassVar[str] = 'Microsoft Azure Databricks'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'databricks'#

This module defines configuration or the Microsoft Azure File Storage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.azure_file_storage.azure_file_storage#

Bases: BaseStage

Properties for the Microsoft Azure File Storage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field append_unique_identifier: bool | None = False (alias 'wave_handling.append_uid')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: AzureFileStorageConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": "connection_string",     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "connection_string": null,     "container": null,     "defer_credentials": false,     "password": null,     "storage_account_url": null,     "tenant_id": null,     "username": null,     "vaulted_properties": null,     "ds_host": null,     "ds_port": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field copy_timeout: int | None = None#
field create_container: bool | None = False#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_file_share: bool | None = False#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field definition_source: str | None = None (alias 'delimited_syntax.record_def.record_def_source')#
field delete_container: bool | None = False#
field delimited_syntax_encoding: str | None = None (alias 'delimited_syntax.encoding')#
field delimited_syntax_escape: str | None = None (alias 'delimited_syntax.escape')#
field delimited_syntax_field_delimiter: str | None = ',' (alias 'delimited_syntax.field_delimiter')#
field delimited_syntax_field_formats_date_format: str | None = None (alias 'delimited_syntax.field_formats.date_format')#
field delimited_syntax_field_formats_decimal_format: str | None = None (alias 'delimited_syntax.field_formats.decimal_format')#
field delimited_syntax_field_formats_time_format: str | None = None (alias 'delimited_syntax.field_formats.time_format')#
field delimited_syntax_field_formats_timestamp_format: str | None = None (alias 'delimited_syntax.field_formats.timestamp_format')#
field delimited_syntax_null_value: str | None = None (alias 'delimited_syntax.null_value')#
field delimited_syntax_row_delimiter: str | None = '<NL>' (alias 'delimited_syntax.row_delimiter')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_file_format: DSFileFormat | None = DSFileFormat.delimited (alias '_file_format')#
field ds_file_name: str = None (alias '_file_name')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field ds_read_mode: DSReadMode = DSReadMode.read_single_file (alias '_read_mode')#
field ds_use_datastage: bool | None = True (alias '_use_datastage')#
field ds_write_mode: DSWriteMode = DSWriteMode.write (alias '_write_mode')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field endpoint_folder: str | None = None (alias 'table_folder_name')#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_exists: FileExists | None = FileExists.overwrite_file#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field file_name_column: str | None = None (alias 'filename_column')#
field file_name_source: str = None#
field file_share: str = None#
field file_share_source: str = None#
field file_size_threshold: int | None = 1 (alias 'wave_handling.file_size_threshold')#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field first_row_is_header: bool | None = False (alias 'delimited_syntax.header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field hide: bool | None = False#
field include_byte_order_mark_for_unicode_encoding: bool | None = False (alias 'delimited_syntax.encoding.output_bom')#
field include_data_types: bool | None = False (alias 'delimited_syntax.header.include_types')#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quotation_mark: DelimitedSyntaxQuotes | None = DelimitedSyntaxQuotes.none (alias 'delimited_syntax.quotes')#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field read_part_size: int | None = None#
field record_definition: DelimitedSyntaxRecordDef | None = DelimitedSyntaxRecordDef.none (alias 'delimited_syntax.record_def')#
field record_limit: int | None = None (alias 'delimited_syntax.record_limit')#
field reject_mode: RejectMode | None = RejectMode.cont#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field table_action: TableAction | None = TableAction.append#
field table_data_file_compression_codec: TableDataFileCompressionCodec | None = None#
field table_data_file_format: TableDataFileFormat | None = TableDataFileFormat.avro#
field table_format: TableFormat | None = None#
field table_name: str | None = None#
field table_namespace: str | None = None#
field the_cache_expiration: str | None = None (alias 'table_partition_cache_expiration')#
field the_cache_size: int | None = None (alias 'table_partition_cache_size')#
field the_data_path: str | None = None (alias 'table_data_path')#
field the_partition_columns: str | None = None (alias 'table_partition_columns')#
field the_partition_paths: str | None = None (alias 'table_partition_path')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field write_part_size: int | None = None#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/azurefilestorage.svg'#
label: ClassVar[str] = 'Microsoft Azure File Storage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'azurefilestorage'#

This module defines configuration or the Azure PostgreSQL stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.azure_postgresql.azure_postgresql#

Bases: BaseStage

Properties for the Azure PostgreSQL stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: AzurePostgresqlConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": "user_credentials",     "discover_data_assets": null,     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "login_timeout": null,     "password": null,     "port": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_username": null,     "query_timeout": 300,     "retry_limit": 2,     "port_is_ssl_enabled": true,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "tenant_id": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_ref_output: bool | None = False#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field login_timeout: int | None = None#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_after_sql_node_statements_from_file: bool | None = False (alias 'before_after.after_node.read_from_file_after_sql_node')#
field read_after_sql_statements_from_file: bool | None = False (alias 'before_after.after.read_from_file_after_sql')#
field read_before_sql_node_statement_from_file: bool | None = False (alias 'before_after.before_node.read_from_file_before_sql_node')#
field read_before_sql_statements_from_file: bool | None = False (alias 'before_after.before.read_from_file_before_sql')#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field read_select_statement_from_file: bool | None = False (alias 'select_statement.read_from_file_select')#
field read_update_statement_from_file: bool | None = False (alias 'update_statement.read_from_file_update')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/postgresql-azure.svg'#
label: ClassVar[str] = 'Azure PostgreSQL'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'postgresql-azure'#

This module defines configuration or the Microsoft Azure Data Lake Storage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.azuredatalake.azuredatalake#

Bases: BaseStage

Properties for the Microsoft Azure Data Lake Storage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: AzuredatalakeConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "authentication_method": "client_credentials",     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "no_proxy": null,     "password": null,     "proxy": false,     "proxy_host": null,     "proxy_port": null,     "proxy_protocol": null,     "encrypted_proxy_communication": null,     "ssl_certificate": null,     "tenant_id": null,     "url": null,     "use_home_as_root": true,     "username": null,     "vaulted_properties": null,     "endpoint": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field endpoint_folder: str | None = None (alias 'table_folder_name')#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field node_count: int | None = 1#
field node_number: int | None = 0#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field table_action: TableAction | None = TableAction.append#
field table_data_file_compression_codec: TableDataFileCompressionCodec | None = None#
field table_data_file_format: TableDataFileFormat | None = TableDataFileFormat.avro#
field table_format: TableFormat | None = None#
field table_name: str | None = None#
field table_namespace: str | None = None#
field the_cache_expiration: str | None = None (alias 'table_partition_cache_expiration')#
field the_cache_size: int | None = None (alias 'table_partition_cache_size')#
field the_data_path: str | None = None (alias 'table_data_path')#
field the_partition_columns: str | None = None (alias 'table_partition_columns')#
field the_partition_paths: str | None = None (alias 'table_partition_path')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_home_as_root: bool | None = True#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/AzureDatalakePX.svg'#
label: ClassVar[str] = 'Microsoft Azure Data Lake Storage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'AzureDatalakePX'#

This module defines configuration or the Microsoft Azure SQL Database stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.azuresql.azuresql#

Bases: BaseStage

Properties for the Microsoft Azure SQL Database stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: AzuresqlConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": "user_credentials",     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_username": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": true,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "use_active_directory": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field inout_parameters: str | None = 'false'#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/azuresql.svg'#
label: ClassVar[str] = 'Microsoft Azure SQL Database'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'azuresql'#

This module defines configuration or the Microsoft Azure Synapse Analytics stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.azuresynapse.azuresynapse#

Bases: BaseStage

Properties for the Microsoft Azure Synapse Analytics stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: AzuresynapseConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": "user_credentials",     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_username": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": true,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "use_active_directory": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hashing_column: str = None#
field hashing_required: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field inout_parameters: str | None = 'false'#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/azuresynapse.svg'#
label: ClassVar[str] = 'Microsoft Azure Synapse Analytics'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'azuresynapse'#

This module defines configuration or the Google BigQuery stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.bigquery.bigquery#

Bases: BaseStage

Properties for the Google BigQuery stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field big_query_temp_bucket_name: str | None = None#
field bucket: str = None#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: BigqueryConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "access_token": null,     "api_service_url": null,     "authentication_method": null,     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "credentials": null,     "credentials_file_path": null,     "defer_credentials": false,     "output_json_string_format": null,     "metadata_discovery": null,     "project_id": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_protocol": "https",     "proxy_username": null,     "refresh_token": null,     "service_account_email": null,     "service_account_token_lifetime": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "security_token_service_audience": null,     "token_field_name": null,     "token_format": "text",     "token_type": "id_token",     "token_url": null,     "request_body": null,     "http_headers": null,     "http_method": "get",     "vaulted_properties": null,     "additional_properties": null,     "oauth_endpoint_uri": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field database_name: str | None = None#
field dataset_name: str = None (alias 'schema_name')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_partitioned_reads: bool | None = False (alias 'partitioned')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field file_format: FileFormat | None = None#
field file_name_prefix: str | None = None (alias 'file_name')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field node_count: int | None = 1#
field node_number: int | None = 0#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_after_sql_node_statements_from_file: bool | None = False (alias 'before_after.after_node.read_from_file_after_sql_node')#
field read_after_sql_statements_from_file: bool | None = False (alias 'before_after.after.read_from_file_after_sql')#
field read_before_sql_node_statement_from_file: bool | None = False (alias 'before_after.before_node.read_from_file_before_sql_node')#
field read_before_sql_statements_from_file: bool | None = False (alias 'before_after.before.read_from_file_before_sql')#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field read_part_size: int | None = None#
field read_select_statement_from_file: bool | None = False (alias 'select_statement.read_from_file_select')#
field read_update_statement_from_file: bool | None = False (alias 'update_statement.read_from_file_update')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str = None#
field use_gcs_staging: bool | None = False#
field write_mode: WriteMode | None = WriteMode.insert#
field write_part_size: int | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/bigqueryPX.svg'#
label: ClassVar[str] = 'Google BigQuery'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'bigqueryPX'#

This module defines configuration or the IBM Db2 Big SQL stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.bigsql.bigsql#

Bases: BaseStage

Properties for the IBM Db2 Big SQL stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: BigsqlConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "access_token": null,     "alternate_hostname": null,     "alternate_port": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "use_my_platform_login_credentials": false,     "max_transport_objects": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "use_alternate_server": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/bigsql.svg'#
label: ClassVar[str] = 'IBM Db2 Big SQL'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'bigsql'#

This module defines configuration or the Box stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.box.box#

Bases: BaseStage

Properties for the Box stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: BoxConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "access_token": null,     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "enterprise_id": null,     "private_key": null,     "private_key_password": null,     "public_key": null,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/box.svg'#
label: ClassVar[str] = 'Box'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'box'#

This module defines configuration or the Apache Cassandra stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.cassandra.cassandra#

Bases: BaseStage

Properties for the Apache Cassandra stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: CassandraConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "keyspace": null,     "password": null,     "port": null,     "read_consistency": "quorum",     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "write_consistency": "quorum",     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/cassandra.svg'#
label: ClassVar[str] = 'Apache Cassandra'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'cassandra'#

This module defines configuration or the Apache Cassandra for DataStage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.cassandra_datastage.cassandra_datastage#

Bases: BaseStage

Properties for the Apache Cassandra for DataStage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field check_schema_agreement: bool | None = True#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: CassandraDatastageConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "authentication_method": "allow_all_authenticator",     "cluster_access_token": null,     "cluster_contact_points": null,     "cluster_user_name": null,     "compression_type": "no_compression",     "defer_credentials": false,     "local_datacenter": "datacenter1",     "password": null,     "protocol_version": "newest_supported",     "keystore_password": null,     "keystore_path": null,     "truststore_password": null,     "truststore_path": null,     "use_ssl_tls": false,     "use_client_certificate_authentication": null,     "use_client_to_node_encryption": null,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field custom_type_codecs: str | None = '' (alias 'custom_typecodecs')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_cql_statement_tracing: bool | None = False (alias 'tracing_statements')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_quoted_identifiers: bool | None = False#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field hide: bool | None = False#
field ignore_blob_column_value_truncation_errors: bool | None = False (alias 'ignore_blob_truncation_errors')#
field ignore_string_column_value_truncation_errors: bool | None = False (alias 'ignore_string_truncation_errors')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field keyspace_name: str = None (alias 'cassandra_keyspace')#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field modification_type: ModificationType | None = ModificationType.insert (alias 'mutation_type')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field page_size: int | None = 10#
field parallel_read_strategy: ParallelReadStrategy | None = ParallelReadStrategy.equal_splitter#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field pre_fetching_threshold: int | None = 2 (alias 'prefetching_threshold')#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_consistency_level: ReadConsistencyLevel | None = ReadConsistencyLevel.quorum#
field reconcile_tables_definitions: bool | None = False#
field runtime_column_propagation: bool | None = None#
field save_null_values: bool | None = True#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field table_name: str = None (alias 'cassandra_table')#
field unique: bool | None = None (alias 'part_unique')#
field use_json_encoded_map_for_a_single_row: bool | None = False (alias 'use_json_mapped_rows')#
field use_parallel_read: bool | None = False#
field write_consistency_level: WriteConsistencyLevel | None = WriteConsistencyLevel.quorum#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/CassandraConnectorPX.svg'#
label: ClassVar[str] = 'Apache Cassandra for DataStage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'CassandraConnectorPX'#

This module defines configuration or the IBM Cloud Object Storage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.cloud_object_storage.cloud_object_storage#

Bases: BaseStage

Properties for the IBM Cloud Object Storage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field append_unique_identifier: bool | None = False (alias 'append_uid')#
field bucket: str | None = None#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: CloudObjectStorageConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "access_key": null,     "api_key": null,     "authentication_method": null,     "discover_data_assets": null,     "bucket": null,     "cluster_access_token": null,     "cluster_user_name": null,     "service_credentials": null,     "credentials_file_path": null,     "defer_credentials": false,     "iam_cos_url": null,     "iam_url": null,     "region_deprecated": null,     "resource_instance_id": null,     "secret_key": null,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "trust_all_ssl_certificates": false,     "url": null,     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_bucket: bool | None = False#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field delete_bucket: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field endpoint_folder: str | None = None (alias 'table_folder_name')#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str | None = None#
field file_size_threshold: int | None = 1#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field iam_cos_url: str | None = None#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field node_count: int | None = 1#
field node_number: int | None = 0#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field read_part_size: int | None = None#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field storage_class: StorageClass | None = StorageClass.standard#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field table_action: TableAction | None = TableAction.append#
field table_data_file_compression_codec: TableDataFileCompressionCodec | None = None#
field table_data_file_format: TableDataFileFormat | None = TableDataFileFormat.avro#
field table_format: TableFormat | None = None#
field table_name: str | None = None#
field table_namespace: str | None = None#
field the_cache_expiration: str | None = None (alias 'table_partition_cache_expiration')#
field the_cache_size: int | None = None (alias 'table_partition_cache_size')#
field the_data_path: str | None = None (alias 'table_data_path')#
field the_partition_columns: str | None = None (alias 'table_partition_columns')#
field the_partition_paths: str | None = None (alias 'table_partition_path')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field trust_all_ssl_certificates: bool | None = False (alias 'trust_all_ssl_cert')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field write_part_size: int | None = None#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/cloudobjectstoragePX.svg'#
label: ClassVar[str] = 'IBM Cloud Object Storage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'cloudobjectstoragePX'#

This module defines configuration or the IBM Cognos Analytics stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.cognos_analytics.cognos_analytics#

Bases: BaseStage

Properties for the IBM Cognos Analytics stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: CognosAnalyticsConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_physical_collection": null,     "ds_port": null,     "access_token": null,     "api_key": null,     "authentication_method": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "use_my_platform_login_credentials": false,     "namespace_id": null,     "password": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "ssl_certificate": null,     "url": null,     "use_anonymous_access": false,     "username": null,     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field file_name: str = None#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field unique: bool | None = None (alias 'part_unique')#
field use_anonymous_access: bool | None = False#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/cognos-analytics.svg'#
label: ClassVar[str] = 'IBM Cognos Analytics'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'cognos-analytics'#

This module defines configuration or the DataStax Enterprise stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.datastax.datastax#

Bases: BaseStage

Properties for the DataStax Enterprise stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: DatastaxConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "keyspace": null,     "password": null,     "port": null,     "read_consistency": "quorum",     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "write_consistency": "quorum",     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/datastax.svg'#
label: ClassVar[str] = 'DataStax Enterprise'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'datastax'#

This module defines configuration or the IBM Db2 stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.db2.db2#

Bases: BaseStage

Properties for the IBM Db2 stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: Db2Conn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "access_token": null,     "application_name": null,     "discover_data_assets": null,     "avoid_timestamp_conversion": null,     "client_accounting_information": null,     "client_hostname": null,     "client_user": null,     "cluster_access_token": null,     "cluster_user_name": null,     "command_timeout": 600,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "impersonate_user": null,     "use_my_platform_login_credentials": false,     "kerberos_sso": null,     "kerberos_sso_keytab": null,     "kerberos_sso_principal": null,     "kerberos_user_principal_name": null,     "kerberos_user_principal_password": null,     "max_transport_objects": null,     "password": null,     "port": null,     "service_principal_name": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "username": null,     "username_and_password_encryption_algorithm": "default",     "security_mechanism": "default",     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/db2.svg'#
label: ClassVar[str] = 'IBM Db2'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'db2'#

This module defines configuration or the IBM Db2 on Cloud stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.db2cloud.db2cloud#

Bases: BaseStage

Properties for the IBM Db2 on Cloud stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: Db2cloudConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "access_token": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "command_timeout": 600,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "use_my_platform_login_credentials": false,     "max_transport_objects": null,     "password": null,     "port": 50001,     "port_is_ssl_enabled": true,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/db2cloud.svg'#
label: ClassVar[str] = 'IBM Db2 on Cloud'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'db2cloud'#

This module defines configuration or the IBM Db2 for DataStage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.db2fordatastage.db2fordatastage#

Bases: BaseStage

Properties for the IBM Db2 for DataStage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field advanced_connection_settings: bool | None = True (alias 'advanced')#
field allow_access_mode: LoadControlAllowAccessMode | None = LoadControlAllowAccessMode.no_access (alias 'load_control.allow_access_mode')#
field allow_changes: bool | None = False (alias 'load_to_zos.image_copy_function.allow_changes')#
field array_size: int | None = 2000 (alias 'session.array_size')#
field atomic_arrays: SessionInsertBufferingAtomicArrays | None = SessionInsertBufferingAtomicArrays.auto (alias 'session.insert_buffering.atomic_arrays')#
field auto_commit_mode: SessionAutocommitMode | None = SessionAutocommitMode.off (alias 'session.autocommit_mode')#
field batch_pipe_system_id: str = None (alias 'load_to_zos.batch_pipe_system_id')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffer_pool: str | None = None (alias 'table_action.generate_create_statement.create_table_bufferpool')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field bulk_load_to_db2_on_z_os: bool | None = False (alias 'load_to_zos')#
field bulk_load_with_lob_or_xml_columns: bool | None = False (alias 'load_control.bulkload_with_lob_xml')#
field ccsid: str = None (alias 'load_to_zos.encoding.ccsid')#
field change_limit_percent_1: int | None = None (alias 'load_to_zos.image_copy_function.change_limit_percent1')#
field change_limit_percent_2: int | None = None (alias 'load_to_zos.image_copy_function.change_limit_percent2')#
field check_pending_cascade: LoadControlCheckPendingCascade | None = LoadControlCheckPendingCascade.deferred (alias 'load_control.check_pending_cascade')#
field check_truncation: bool | None = False (alias 'load_control.partitioned_db_config.check_truncation')#
field clean_up_on_failure: bool | None = False (alias 'load_control.cleanup_on_fail')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_delimiter: LoggingLogColumnValuesDelimiter | None = LoggingLogColumnValuesDelimiter.space (alias 'logging.log_column_values.delimiter')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field columns: str = '' (alias 'session.pass_lob_locator.column')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field compress: TableActionGenerateCreateStatementCreateTableCompress | None = TableActionGenerateCreateStatementCreateTableCompress.database_default (alias 'table_action.generate_create_statement.create_table_compress')#
field concurrent_access_level: LoadToZosShrLevel | None = LoadToZosShrLevel.none (alias 'load_to_zos.shr_level')#
field connection: Db2fordatastageConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "advanced_connection_settings": true,     "api_key": null,     "hostname": null,     "options": null,     "port": 50000,     "ssl_certificate_arm": null,     "ssl_connection": false,     "credentials": "username_and_password",     "cas_lite_service_authorization_header": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "keep_conductor_connection_alive": false,     "password": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "original_hostname_of_the_resource": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "use_cas_lite_service": true,     "use_direct_connections": false,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field copy_loaded_data: LoadControlCopyLoadedData | None = LoadControlCopyLoadedData.no_copy (alias 'load_control.copy_loaded_data')#
field cpu_parallelism: int | None = 0 (alias 'load_control.cpu_parallelism')#
field create_table_statement: str = '' (alias 'table_action.generate_create_statement.create_statement')#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_buffer_size: int | None = 0 (alias 'load_control.data_buffer_size')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field delete_statement: str = None (alias 'sql.delete_statement')#
field device_type: str | None = 'SYSDA' (alias 'load_to_zos.device_type')#
field direct_insert: bool | None = True (alias 'sql.direct_insert')#
field directory_for_data_and_command_files: str = None (alias 'load_control.data_file_path')#
field directory_for_data_files: str | None = None (alias 'load_to_zos.transfer.data_file_path')#
field directory_for_log_files: str | None = None (alias 'session.use_external_tables.log_directory')#
field directory_for_named_pipe: str | None = '/tmp' (alias 'session.use_external_tables.directory_for_named_pipe')#
field directory_for_named_pipe_unix_only: str | None = '/tmp' (alias 'load_control.directory_for_named_pipe')#
field disk_parallelism: int | None = 0 (alias 'load_control.disk_parallelism')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field distribute_by: TableActionGenerateCreateStatementCreateTableDistributeBy | None = TableActionGenerateCreateStatementCreateTableDistributeBy.none (alias 'table_action.generate_create_statement.create_table_distribute_by')#
field drop_table: bool | None = True (alias 'session.temporary_work_table.drop_table')#
field drop_table_statement: str = '' (alias 'table_action.generate_drop_statement.drop_statement')#
field drop_unmatched_fields: bool | None = True (alias 'session.schema_reconciliation.drop_unmatched_fields')#
field dsn_prefix: str | None = None (alias 'load_to_zos.dsn_prefix')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_and_after_sql: bool | None = False (alias 'before_after')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_lob_references: bool | None = False (alias 'session.pass_lob_locator')#
field enable_partitioned_reads: bool | None = False (alias 'sql.enable_partitioning')#
field enable_quoted_identifiers: bool | None = True (alias 'enable_quoted_i_ds')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: LoadToZosEncoding | None = LoadToZosEncoding.ebcdic (alias 'load_to_zos.encoding')#
field end_of_data: bool | None = False (alias 'transaction.end_of_wave.end_of_data')#
field end_of_wave: TransactionEndOfWave | None = TransactionEndOfWave.none (alias 'transaction.end_of_wave')#
field exception_table_name: str | None = None (alias 'load_control.exception_table')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field external_table_collect_statistics_during_load: bool | None = False (alias 'session.use_external_tables.statistics')#
field external_tables_other_options: str | None = '' (alias 'session.use_external_tables.other_options')#
field fail_on_code_page_mismatch: bool | None = False (alias 'session.schema_reconciliation.fail_on_code_page_mismatch')#
field fail_on_error: bool | None = True (alias 'sql.user_defined_sql.fail_on_error')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fail_on_error_create_statement: bool | None = True (alias 'table_action.generate_create_statement.fail_on_error')#
field fail_on_error_drop_statement: bool | None = True (alias 'table_action.generate_drop_statement.fail_on_error')#
field fail_on_error_truncate_statement: bool | None = True (alias 'table_action.generate_truncate_statement.fail_on_error')#
field fail_on_row_error: bool | None = True (alias 'session.fail_on_row_error_px')#
field fail_on_size_mismatch: bool | None = True (alias 'session.schema_reconciliation.fail_on_size_mismatch')#
field fail_on_type_mismatch: bool | None = True (alias 'session.schema_reconciliation.fail_on_type_mismatch')#
field file_type: LoadControlFileType | None = LoadControlFileType.asc (alias 'load_control.file_type')#
field flow_dirty: str | None = 'false'#
field generate_create_statement_at_runtime: bool | None = True (alias 'table_action.generate_create_statement')#
field generate_create_statement_distribute_by_hash_key_column_names: str = None (alias 'table_action.generate_create_statement.create_table_distribute_by.hash_key_columns')#
field generate_drop_statement_at_runtime: bool | None = True (alias 'table_action.generate_drop_statement')#
field generate_partitioning_sql: bool | None = True (alias 'sql.enable_partitioning.partitioning_method.gen_partitioning_sql')#
field generate_sql_at_runtime: bool | None = False (alias 'generate_sql')#
field generate_truncate_statement_at_runtime: bool | None = True (alias 'table_action.generate_truncate_statement')#
field graphic_character_set: str | None = None (alias 'load_to_zos.encoding.graphic_character_set')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field has_reject_output: bool | None = False#
field hfs_file_directory: str = None (alias 'load_to_zos.transfer.uss_file_directory')#
field hide: bool | None = False#
field higher_port_number: int = None (alias 'load_control.partitioned_db_config.port_range.max_value')#
field hold_quiesce: bool | None = False (alias 'load_control.hold_quiesce')#
field image_copy_function: LoadToZosImageCopyFunction | None = LoadToZosImageCopyFunction.no (alias 'load_to_zos.image_copy_function')#
field index_in: str | None = None (alias 'table_action.generate_create_statement.create_table_index_in')#
field indexing_mode: LoadControlIndexingMode | None = LoadControlIndexingMode.automatic_selection (alias 'load_control.indexing_mode')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field insert_buffering: SessionInsertBuffering | None = SessionInsertBuffering.default (alias 'session.insert_buffering')#
field insert_statement: str = None (alias 'sql.insert_statement')#
field interval_between_retries: int = 10 (alias 'load_to_zos.transfer.retry_connection.retry_interval')#
field isolate_partition_errors: LoadControlPartitionedDbConfigIsolatePartErrors | None = LoadControlPartitionedDbConfigIsolatePartErrors.load_errors_only (alias 'load_control.partitioned_db_config.isolate_part_errors')#
field isolation_level: SessionIsolationLevel | None = SessionIsolationLevel.cursor_stability (alias 'session.isolation_level')#
field keep_existing_records_in_table_space: bool | None = True (alias 'load_to_zos.resume')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column: list | None = [] (alias 'record_ordering.key_column')#
field key_columns: str | None = None (alias 'sql.key_columns')#
field library_used_to_copy: str = '' (alias 'load_control.copy_loaded_data.copy_load_library_name')#
field limit: int | None = 1000 (alias 'limit_rows.limit')#
field limit_number_of_returned_rows: bool | None = False (alias 'limit_rows')#
field limit_parallelism: bool | None = False#
field load_control_dump_file: str | None = None (alias 'load_control.file_type_modifiers.dump_file')#
field load_control_files_only: bool | None = False (alias 'load_control.files_only')#
field load_control_load_method: LoadControlLoadMethod | None = LoadControlLoadMethod.named_pipes (alias 'load_control.load_method')#
field load_control_statistics: bool | None = False (alias 'load_control.statistics')#
field load_mode: LoadControlLoadMode | None = LoadControlLoadMode.insert (alias 'load_control.load_mode')#
field load_timeout: int | None = 300 (alias 'load_control.load_timeout')#
field load_to_zos_data_file_attributes_discard_data_set_data_class: str | None = None (alias 'load_to_zos.data_file_attributes.discard_data_set.data_class')#
field load_to_zos_data_file_attributes_discard_data_set_dataset_name: str | None = None (alias 'load_to_zos.data_file_attributes.discard_data_set.dataset_name')#
field load_to_zos_data_file_attributes_discard_data_set_file_disposition_abnormal_termination: LoadToZosDataFileAttributesDiscardDataSetFileDispositionAbnormalTermination | None = LoadToZosDataFileAttributesDiscardDataSetFileDispositionAbnormalTermination.catalog (alias 'load_to_zos.data_file_attributes.discard_data_set.file_disposition.abnormal_termination')#
field load_to_zos_data_file_attributes_discard_data_set_file_disposition_normal_termination: LoadToZosDataFileAttributesDiscardDataSetFileDispositionNormalTermination | None = LoadToZosDataFileAttributesDiscardDataSetFileDispositionNormalTermination.catalog (alias 'load_to_zos.data_file_attributes.discard_data_set.file_disposition.normal_termination')#
field load_to_zos_data_file_attributes_discard_data_set_file_disposition_status: LoadToZosDataFileAttributesDiscardDataSetFileDispositionStatus | None = LoadToZosDataFileAttributesDiscardDataSetFileDispositionStatus.replace (alias 'load_to_zos.data_file_attributes.discard_data_set.file_disposition.status')#
field load_to_zos_data_file_attributes_discard_data_set_management_class: str | None = None (alias 'load_to_zos.data_file_attributes.discard_data_set.management_class')#
field load_to_zos_data_file_attributes_discard_data_set_number_of_buffers: int | None = None (alias 'load_to_zos.data_file_attributes.discard_data_set.number_of_buffers')#
field load_to_zos_data_file_attributes_discard_data_set_primary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.discard_data_set.primary_allocation')#
field load_to_zos_data_file_attributes_discard_data_set_secondary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.discard_data_set.secondary_allocation')#
field load_to_zos_data_file_attributes_discard_data_set_space_type: LoadToZosDataFileAttributesDiscardDataSetSpaceType | None = LoadToZosDataFileAttributesDiscardDataSetSpaceType.cylinders (alias 'load_to_zos.data_file_attributes.discard_data_set.space_type')#
field load_to_zos_data_file_attributes_discard_data_set_storage_class: str | None = None (alias 'load_to_zos.data_file_attributes.discard_data_set.storage_class')#
field load_to_zos_data_file_attributes_discard_data_set_unit: str | None = None (alias 'load_to_zos.data_file_attributes.discard_data_set.unit')#
field load_to_zos_data_file_attributes_discard_data_set_volumes: str | None = None (alias 'load_to_zos.data_file_attributes.discard_data_set.volumes')#
field load_to_zos_data_file_attributes_error_data_set_data_class: str | None = None (alias 'load_to_zos.data_file_attributes.error_data_set.data_class')#
field load_to_zos_data_file_attributes_error_data_set_dataset_name: str | None = None (alias 'load_to_zos.data_file_attributes.error_data_set.dataset_name')#
field load_to_zos_data_file_attributes_error_data_set_file_disposition_abnormal_termination: LoadToZosDataFileAttributesErrorDataSetFileDispositionAbnormalTermination | None = LoadToZosDataFileAttributesErrorDataSetFileDispositionAbnormalTermination.catalog (alias 'load_to_zos.data_file_attributes.error_data_set.file_disposition.abnormal_termination')#
field load_to_zos_data_file_attributes_error_data_set_file_disposition_normal_termination: LoadToZosDataFileAttributesErrorDataSetFileDispositionNormalTermination | None = LoadToZosDataFileAttributesErrorDataSetFileDispositionNormalTermination.catalog (alias 'load_to_zos.data_file_attributes.error_data_set.file_disposition.normal_termination')#
field load_to_zos_data_file_attributes_error_data_set_file_disposition_status: LoadToZosDataFileAttributesErrorDataSetFileDispositionStatus | None = LoadToZosDataFileAttributesErrorDataSetFileDispositionStatus.replace (alias 'load_to_zos.data_file_attributes.error_data_set.file_disposition.status')#
field load_to_zos_data_file_attributes_error_data_set_management_class: str | None = None (alias 'load_to_zos.data_file_attributes.error_data_set.management_class')#
field load_to_zos_data_file_attributes_error_data_set_number_of_buffers: int | None = None (alias 'load_to_zos.data_file_attributes.error_data_set.number_of_buffers')#
field load_to_zos_data_file_attributes_error_data_set_primary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.error_data_set.primary_allocation')#
field load_to_zos_data_file_attributes_error_data_set_secondary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.error_data_set.secondary_allocation')#
field load_to_zos_data_file_attributes_error_data_set_space_type: LoadToZosDataFileAttributesErrorDataSetSpaceType | None = LoadToZosDataFileAttributesErrorDataSetSpaceType.cylinders (alias 'load_to_zos.data_file_attributes.error_data_set.space_type')#
field load_to_zos_data_file_attributes_error_data_set_storage_class: str | None = None (alias 'load_to_zos.data_file_attributes.error_data_set.storage_class')#
field load_to_zos_data_file_attributes_error_data_set_unit: str | None = None (alias 'load_to_zos.data_file_attributes.error_data_set.unit')#
field load_to_zos_data_file_attributes_error_data_set_volumes: str | None = None (alias 'load_to_zos.data_file_attributes.error_data_set.volumes')#
field load_to_zos_data_file_attributes_input_data_files_data_class: str | None = None (alias 'load_to_zos.data_file_attributes.input_data_files.data_class')#
field load_to_zos_data_file_attributes_input_data_files_dataset_name: str | None = None (alias 'load_to_zos.data_file_attributes.input_data_files.dataset_name')#
field load_to_zos_data_file_attributes_input_data_files_file_disposition_abnormal_termination: LoadToZosDataFileAttributesInputDataFilesFileDispositionAbnormalTermination | None = LoadToZosDataFileAttributesInputDataFilesFileDispositionAbnormalTermination.keep (alias 'load_to_zos.data_file_attributes.input_data_files.file_disposition.abnormal_termination')#
field load_to_zos_data_file_attributes_input_data_files_file_disposition_normal_termination: LoadToZosDataFileAttributesInputDataFilesFileDispositionNormalTermination | None = LoadToZosDataFileAttributesInputDataFilesFileDispositionNormalTermination.keep (alias 'load_to_zos.data_file_attributes.input_data_files.file_disposition.normal_termination')#
field load_to_zos_data_file_attributes_input_data_files_file_disposition_status: LoadToZosDataFileAttributesInputDataFilesFileDispositionStatus | None = LoadToZosDataFileAttributesInputDataFilesFileDispositionStatus.replace (alias 'load_to_zos.data_file_attributes.input_data_files.file_disposition.status')#
field load_to_zos_data_file_attributes_input_data_files_management_class: str | None = None (alias 'load_to_zos.data_file_attributes.input_data_files.management_class')#
field load_to_zos_data_file_attributes_input_data_files_number_of_buffers: int | None = None (alias 'load_to_zos.data_file_attributes.input_data_files.number_of_buffers')#
field load_to_zos_data_file_attributes_input_data_files_primary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.input_data_files.primary_allocation')#
field load_to_zos_data_file_attributes_input_data_files_secondary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.input_data_files.secondary_allocation')#
field load_to_zos_data_file_attributes_input_data_files_space_type: LoadToZosDataFileAttributesInputDataFilesSpaceType | None = LoadToZosDataFileAttributesInputDataFilesSpaceType.cylinders (alias 'load_to_zos.data_file_attributes.input_data_files.space_type')#
field load_to_zos_data_file_attributes_input_data_files_storage_class: str | None = None (alias 'load_to_zos.data_file_attributes.input_data_files.storage_class')#
field load_to_zos_data_file_attributes_input_data_files_unit: str | None = None (alias 'load_to_zos.data_file_attributes.input_data_files.unit')#
field load_to_zos_data_file_attributes_input_data_files_volumes: str | None = None (alias 'load_to_zos.data_file_attributes.input_data_files.volumes')#
field load_to_zos_data_file_attributes_map_data_set_data_class: str | None = None (alias 'load_to_zos.data_file_attributes.map_data_set.data_class')#
field load_to_zos_data_file_attributes_map_data_set_dataset_name: str | None = None (alias 'load_to_zos.data_file_attributes.map_data_set.dataset_name')#
field load_to_zos_data_file_attributes_map_data_set_file_disposition_abnormal_termination: LoadToZosDataFileAttributesMapDataSetFileDispositionAbnormalTermination | None = LoadToZosDataFileAttributesMapDataSetFileDispositionAbnormalTermination.catalog (alias 'load_to_zos.data_file_attributes.map_data_set.file_disposition.abnormal_termination')#
field load_to_zos_data_file_attributes_map_data_set_file_disposition_normal_termination: LoadToZosDataFileAttributesMapDataSetFileDispositionNormalTermination | None = LoadToZosDataFileAttributesMapDataSetFileDispositionNormalTermination.catalog (alias 'load_to_zos.data_file_attributes.map_data_set.file_disposition.normal_termination')#
field load_to_zos_data_file_attributes_map_data_set_file_disposition_status: LoadToZosDataFileAttributesMapDataSetFileDispositionStatus | None = LoadToZosDataFileAttributesMapDataSetFileDispositionStatus.replace (alias 'load_to_zos.data_file_attributes.map_data_set.file_disposition.status')#
field load_to_zos_data_file_attributes_map_data_set_management_class: str | None = None (alias 'load_to_zos.data_file_attributes.map_data_set.management_class')#
field load_to_zos_data_file_attributes_map_data_set_number_of_buffers: int | None = None (alias 'load_to_zos.data_file_attributes.map_data_set.number_of_buffers')#
field load_to_zos_data_file_attributes_map_data_set_primary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.map_data_set.primary_allocation')#
field load_to_zos_data_file_attributes_map_data_set_secondary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.map_data_set.secondary_allocation')#
field load_to_zos_data_file_attributes_map_data_set_space_type: LoadToZosDataFileAttributesMapDataSetSpaceType | None = LoadToZosDataFileAttributesMapDataSetSpaceType.cylinders (alias 'load_to_zos.data_file_attributes.map_data_set.space_type')#
field load_to_zos_data_file_attributes_map_data_set_storage_class: str | None = None (alias 'load_to_zos.data_file_attributes.map_data_set.storage_class')#
field load_to_zos_data_file_attributes_map_data_set_unit: str | None = None (alias 'load_to_zos.data_file_attributes.map_data_set.unit')#
field load_to_zos_data_file_attributes_map_data_set_volumes: str | None = None (alias 'load_to_zos.data_file_attributes.map_data_set.volumes')#
field load_to_zos_data_file_attributes_work1_data_set_data_class: str | None = None (alias 'load_to_zos.data_file_attributes.work1_data_set.data_class')#
field load_to_zos_data_file_attributes_work1_data_set_dataset_name: str | None = None (alias 'load_to_zos.data_file_attributes.work1_data_set.dataset_name')#
field load_to_zos_data_file_attributes_work1_data_set_file_disposition_abnormal_termination: LoadToZosDataFileAttributesWork1DataSetFileDispositionAbnormalTermination | None = LoadToZosDataFileAttributesWork1DataSetFileDispositionAbnormalTermination.delete (alias 'load_to_zos.data_file_attributes.work1_data_set.file_disposition.abnormal_termination')#
field load_to_zos_data_file_attributes_work1_data_set_file_disposition_normal_termination: LoadToZosDataFileAttributesWork1DataSetFileDispositionNormalTermination | None = LoadToZosDataFileAttributesWork1DataSetFileDispositionNormalTermination.delete (alias 'load_to_zos.data_file_attributes.work1_data_set.file_disposition.normal_termination')#
field load_to_zos_data_file_attributes_work1_data_set_file_disposition_status: LoadToZosDataFileAttributesWork1DataSetFileDispositionStatus | None = LoadToZosDataFileAttributesWork1DataSetFileDispositionStatus.replace (alias 'load_to_zos.data_file_attributes.work1_data_set.file_disposition.status')#
field load_to_zos_data_file_attributes_work1_data_set_management_class: str | None = None (alias 'load_to_zos.data_file_attributes.work1_data_set.management_class')#
field load_to_zos_data_file_attributes_work1_data_set_number_of_buffers: int | None = None (alias 'load_to_zos.data_file_attributes.work1_data_set.number_of_buffers')#
field load_to_zos_data_file_attributes_work1_data_set_primary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.work1_data_set.primary_allocation')#
field load_to_zos_data_file_attributes_work1_data_set_secondary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.work1_data_set.secondary_allocation')#
field load_to_zos_data_file_attributes_work1_data_set_space_type: LoadToZosDataFileAttributesWork1DataSetSpaceType | None = LoadToZosDataFileAttributesWork1DataSetSpaceType.cylinders (alias 'load_to_zos.data_file_attributes.work1_data_set.space_type')#
field load_to_zos_data_file_attributes_work1_data_set_storage_class: str | None = None (alias 'load_to_zos.data_file_attributes.work1_data_set.storage_class')#
field load_to_zos_data_file_attributes_work1_data_set_unit: str | None = None (alias 'load_to_zos.data_file_attributes.work1_data_set.unit')#
field load_to_zos_data_file_attributes_work1_data_set_volumes: str | None = None (alias 'load_to_zos.data_file_attributes.work1_data_set.volumes')#
field load_to_zos_data_file_attributes_work2_data_set_data_class: str | None = None (alias 'load_to_zos.data_file_attributes.work2_data_set.data_class')#
field load_to_zos_data_file_attributes_work2_data_set_dataset_name: str | None = None (alias 'load_to_zos.data_file_attributes.work2_data_set.dataset_name')#
field load_to_zos_data_file_attributes_work2_data_set_file_disposition_abnormal_termination: LoadToZosDataFileAttributesWork2DataSetFileDispositionAbnormalTermination | None = LoadToZosDataFileAttributesWork2DataSetFileDispositionAbnormalTermination.delete (alias 'load_to_zos.data_file_attributes.work2_data_set.file_disposition.abnormal_termination')#
field load_to_zos_data_file_attributes_work2_data_set_file_disposition_normal_termination: LoadToZosDataFileAttributesWork2DataSetFileDispositionNormalTermination | None = LoadToZosDataFileAttributesWork2DataSetFileDispositionNormalTermination.delete (alias 'load_to_zos.data_file_attributes.work2_data_set.file_disposition.normal_termination')#
field load_to_zos_data_file_attributes_work2_data_set_file_disposition_status: LoadToZosDataFileAttributesWork2DataSetFileDispositionStatus | None = LoadToZosDataFileAttributesWork2DataSetFileDispositionStatus.replace (alias 'load_to_zos.data_file_attributes.work2_data_set.file_disposition.status')#
field load_to_zos_data_file_attributes_work2_data_set_management_class: str | None = None (alias 'load_to_zos.data_file_attributes.work2_data_set.management_class')#
field load_to_zos_data_file_attributes_work2_data_set_number_of_buffers: int | None = None (alias 'load_to_zos.data_file_attributes.work2_data_set.number_of_buffers')#
field load_to_zos_data_file_attributes_work2_data_set_primary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.work2_data_set.primary_allocation')#
field load_to_zos_data_file_attributes_work2_data_set_secondary_allocation: int | None = None (alias 'load_to_zos.data_file_attributes.work2_data_set.secondary_allocation')#
field load_to_zos_data_file_attributes_work2_data_set_space_type: LoadToZosDataFileAttributesWork2DataSetSpaceType | None = LoadToZosDataFileAttributesWork2DataSetSpaceType.cylinders (alias 'load_to_zos.data_file_attributes.work2_data_set.space_type')#
field load_to_zos_data_file_attributes_work2_data_set_storage_class: str | None = None (alias 'load_to_zos.data_file_attributes.work2_data_set.storage_class')#
field load_to_zos_data_file_attributes_work2_data_set_unit: str | None = None (alias 'load_to_zos.data_file_attributes.work2_data_set.unit')#
field load_to_zos_data_file_attributes_work2_data_set_volumes: str | None = None (alias 'load_to_zos.data_file_attributes.work2_data_set.volumes')#
field load_to_zos_encoding_character_set: str | None = None (alias 'load_to_zos.encoding.character_set')#
field load_to_zos_files_only: bool | None = False (alias 'load_to_zos.files_only')#
field load_to_zos_image_copy_function_image_copy_backup_file_data_class: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_backup_file.data_class')#
field load_to_zos_image_copy_function_image_copy_backup_file_dataset_name: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_backup_file.dataset_name')#
field load_to_zos_image_copy_function_image_copy_backup_file_file_disposition_abnormal_termination: LoadToZosImageCopyFunctionImageCopyBackupFileFileDispositionAbnormalTermination | None = LoadToZosImageCopyFunctionImageCopyBackupFileFileDispositionAbnormalTermination.catalog (alias 'load_to_zos.image_copy_function.image_copy_backup_file.file_disposition.abnormal_termination')#
field load_to_zos_image_copy_function_image_copy_backup_file_file_disposition_normal_termination: LoadToZosImageCopyFunctionImageCopyBackupFileFileDispositionNormalTermination | None = LoadToZosImageCopyFunctionImageCopyBackupFileFileDispositionNormalTermination.catalog (alias 'load_to_zos.image_copy_function.image_copy_backup_file.file_disposition.normal_termination')#
field load_to_zos_image_copy_function_image_copy_backup_file_file_disposition_status: LoadToZosImageCopyFunctionImageCopyBackupFileFileDispositionStatus | None = LoadToZosImageCopyFunctionImageCopyBackupFileFileDispositionStatus.replace (alias 'load_to_zos.image_copy_function.image_copy_backup_file.file_disposition.status')#
field load_to_zos_image_copy_function_image_copy_backup_file_management_class: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_backup_file.management_class')#
field load_to_zos_image_copy_function_image_copy_backup_file_number_of_buffers: int | None = None (alias 'load_to_zos.image_copy_function.image_copy_backup_file.number_of_buffers')#
field load_to_zos_image_copy_function_image_copy_backup_file_primary_allocation: int | None = None (alias 'load_to_zos.image_copy_function.image_copy_backup_file.primary_allocation')#
field load_to_zos_image_copy_function_image_copy_backup_file_secondary_allocation: int | None = None (alias 'load_to_zos.image_copy_function.image_copy_backup_file.secondary_allocation')#
field load_to_zos_image_copy_function_image_copy_backup_file_space_type: LoadToZosImageCopyFunctionImageCopyBackupFileSpaceType | None = LoadToZosImageCopyFunctionImageCopyBackupFileSpaceType.cylinders (alias 'load_to_zos.image_copy_function.image_copy_backup_file.space_type')#
field load_to_zos_image_copy_function_image_copy_backup_file_storage_class: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_backup_file.storage_class')#
field load_to_zos_image_copy_function_image_copy_backup_file_unit: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_backup_file.unit')#
field load_to_zos_image_copy_function_image_copy_backup_file_volumes: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_backup_file.volumes')#
field load_to_zos_image_copy_function_image_copy_file_data_class: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_file.data_class')#
field load_to_zos_image_copy_function_image_copy_file_dataset_name: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_file.dataset_name')#
field load_to_zos_image_copy_function_image_copy_file_file_disposition_abnormal_termination: LoadToZosImageCopyFunctionImageCopyFileFileDispositionAbnormalTermination | None = LoadToZosImageCopyFunctionImageCopyFileFileDispositionAbnormalTermination.catalog (alias 'load_to_zos.image_copy_function.image_copy_file.file_disposition.abnormal_termination')#
field load_to_zos_image_copy_function_image_copy_file_file_disposition_normal_termination: LoadToZosImageCopyFunctionImageCopyFileFileDispositionNormalTermination | None = LoadToZosImageCopyFunctionImageCopyFileFileDispositionNormalTermination.catalog (alias 'load_to_zos.image_copy_function.image_copy_file.file_disposition.normal_termination')#
field load_to_zos_image_copy_function_image_copy_file_file_disposition_status: LoadToZosImageCopyFunctionImageCopyFileFileDispositionStatus | None = LoadToZosImageCopyFunctionImageCopyFileFileDispositionStatus.replace (alias 'load_to_zos.image_copy_function.image_copy_file.file_disposition.status')#
field load_to_zos_image_copy_function_image_copy_file_management_class: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_file.management_class')#
field load_to_zos_image_copy_function_image_copy_file_number_of_buffers: int | None = None (alias 'load_to_zos.image_copy_function.image_copy_file.number_of_buffers')#
field load_to_zos_image_copy_function_image_copy_file_primary_allocation: int | None = None (alias 'load_to_zos.image_copy_function.image_copy_file.primary_allocation')#
field load_to_zos_image_copy_function_image_copy_file_secondary_allocation: int | None = None (alias 'load_to_zos.image_copy_function.image_copy_file.secondary_allocation')#
field load_to_zos_image_copy_function_image_copy_file_space_type: LoadToZosImageCopyFunctionImageCopyFileSpaceType | None = LoadToZosImageCopyFunctionImageCopyFileSpaceType.cylinders (alias 'load_to_zos.image_copy_function.image_copy_file.space_type')#
field load_to_zos_image_copy_function_image_copy_file_storage_class: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_file.storage_class')#
field load_to_zos_image_copy_function_image_copy_file_unit: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_file.unit')#
field load_to_zos_image_copy_function_image_copy_file_volumes: str | None = None (alias 'load_to_zos.image_copy_function.image_copy_file.volumes')#
field load_to_zos_image_copy_function_recovery_backup_data_class: str | None = None (alias 'load_to_zos.image_copy_function.recovery_backup.data_class')#
field load_to_zos_image_copy_function_recovery_backup_dataset_name: str | None = None (alias 'load_to_zos.image_copy_function.recovery_backup.dataset_name')#
field load_to_zos_image_copy_function_recovery_backup_file_disposition_abnormal_termination: LoadToZosImageCopyFunctionRecoveryBackupFileDispositionAbnormalTermination | None = LoadToZosImageCopyFunctionRecoveryBackupFileDispositionAbnormalTermination.catalog (alias 'load_to_zos.image_copy_function.recovery_backup.file_disposition.abnormal_termination')#
field load_to_zos_image_copy_function_recovery_backup_file_disposition_normal_termination: LoadToZosImageCopyFunctionRecoveryBackupFileDispositionNormalTermination | None = LoadToZosImageCopyFunctionRecoveryBackupFileDispositionNormalTermination.catalog (alias 'load_to_zos.image_copy_function.recovery_backup.file_disposition.normal_termination')#
field load_to_zos_image_copy_function_recovery_backup_file_disposition_status: LoadToZosImageCopyFunctionRecoveryBackupFileDispositionStatus | None = LoadToZosImageCopyFunctionRecoveryBackupFileDispositionStatus.replace (alias 'load_to_zos.image_copy_function.recovery_backup.file_disposition.status')#
field load_to_zos_image_copy_function_recovery_backup_management_class: str | None = None (alias 'load_to_zos.image_copy_function.recovery_backup.management_class')#
field load_to_zos_image_copy_function_recovery_backup_number_of_buffers: int | None = None (alias 'load_to_zos.image_copy_function.recovery_backup.number_of_buffers')#
field load_to_zos_image_copy_function_recovery_backup_primary_allocation: int | None = None (alias 'load_to_zos.image_copy_function.recovery_backup.primary_allocation')#
field load_to_zos_image_copy_function_recovery_backup_secondary_allocation: int | None = None (alias 'load_to_zos.image_copy_function.recovery_backup.secondary_allocation')#
field load_to_zos_image_copy_function_recovery_backup_space_type: LoadToZosImageCopyFunctionRecoveryBackupSpaceType | None = LoadToZosImageCopyFunctionRecoveryBackupSpaceType.cylinders (alias 'load_to_zos.image_copy_function.recovery_backup.space_type')#
field load_to_zos_image_copy_function_recovery_backup_storage_class: str | None = None (alias 'load_to_zos.image_copy_function.recovery_backup.storage_class')#
field load_to_zos_image_copy_function_recovery_backup_unit: str | None = None (alias 'load_to_zos.image_copy_function.recovery_backup.unit')#
field load_to_zos_image_copy_function_recovery_backup_volumes: str | None = None (alias 'load_to_zos.image_copy_function.recovery_backup.volumes')#
field load_to_zos_image_copy_function_recovery_file_data_class: str | None = None (alias 'load_to_zos.image_copy_function.recovery_file.data_class')#
field load_to_zos_image_copy_function_recovery_file_dataset_name: str | None = None (alias 'load_to_zos.image_copy_function.recovery_file.dataset_name')#
field load_to_zos_image_copy_function_recovery_file_file_disposition_abnormal_termination: LoadToZosImageCopyFunctionRecoveryFileFileDispositionAbnormalTermination | None = LoadToZosImageCopyFunctionRecoveryFileFileDispositionAbnormalTermination.catalog (alias 'load_to_zos.image_copy_function.recovery_file.file_disposition.abnormal_termination')#
field load_to_zos_image_copy_function_recovery_file_file_disposition_normal_termination: LoadToZosImageCopyFunctionRecoveryFileFileDispositionNormalTermination | None = LoadToZosImageCopyFunctionRecoveryFileFileDispositionNormalTermination.catalog (alias 'load_to_zos.image_copy_function.recovery_file.file_disposition.normal_termination')#
field load_to_zos_image_copy_function_recovery_file_file_disposition_status: LoadToZosImageCopyFunctionRecoveryFileFileDispositionStatus | None = LoadToZosImageCopyFunctionRecoveryFileFileDispositionStatus.replace (alias 'load_to_zos.image_copy_function.recovery_file.file_disposition.status')#
field load_to_zos_image_copy_function_recovery_file_management_class: str | None = None (alias 'load_to_zos.image_copy_function.recovery_file.management_class')#
field load_to_zos_image_copy_function_recovery_file_number_of_buffers: int | None = None (alias 'load_to_zos.image_copy_function.recovery_file.number_of_buffers')#
field load_to_zos_image_copy_function_recovery_file_primary_allocation: int | None = None (alias 'load_to_zos.image_copy_function.recovery_file.primary_allocation')#
field load_to_zos_image_copy_function_recovery_file_secondary_allocation: int | None = None (alias 'load_to_zos.image_copy_function.recovery_file.secondary_allocation')#
field load_to_zos_image_copy_function_recovery_file_space_type: LoadToZosImageCopyFunctionRecoveryFileSpaceType | None = LoadToZosImageCopyFunctionRecoveryFileSpaceType.cylinders (alias 'load_to_zos.image_copy_function.recovery_file.space_type')#
field load_to_zos_image_copy_function_recovery_file_storage_class: str | None = None (alias 'load_to_zos.image_copy_function.recovery_file.storage_class')#
field load_to_zos_image_copy_function_recovery_file_unit: str | None = None (alias 'load_to_zos.image_copy_function.recovery_file.unit')#
field load_to_zos_image_copy_function_recovery_file_volumes: str | None = None (alias 'load_to_zos.image_copy_function.recovery_file.volumes')#
field load_to_zos_image_copy_image_copy_backup_file: bool | None = False (alias 'load_to_zos.image_copy_function.image_copy_backup_file')#
field load_to_zos_image_copy_recovery_recovery_backup_file: bool | None = False (alias 'load_to_zos.image_copy_function.recovery_backup')#
field load_to_zos_image_copy_recovery_recovery_file: bool | None = False (alias 'load_to_zos.image_copy_function.recovery_file')#
field load_to_zos_load_method: LoadToZosLoadMethod | None = LoadToZosLoadMethod.mvs_datasets (alias 'load_to_zos.load_method')#
field load_to_zos_statistics: LoadToZosStatistics | None = LoadToZosStatistics.none (alias 'load_to_zos.statistics')#
field load_to_zos_transfer_password: str | None = None (alias 'load_to_zos.transfer.password')#
field load_with_logging: bool | None = False (alias 'load_to_zos.load_with_logging')#
field loaded_data_copy_location: str = '' (alias 'load_control.copy_loaded_data.copy_to_device_or_directory')#
field lob_path_list: str | None = None (alias 'load_control.lob_path_list')#
field lock_wait_mode: LockWaitMode | None = LockWaitMode.use_the_lock_timeout_database_configuration_parameter#
field lock_wait_time: int = None (alias 'lock_wait_mode.lock_wait_mode_time')#
field lock_with_force: bool | None = False (alias 'load_control.lock_with_force')#
field log_column_values_on_first_row_error: bool | None = False (alias 'logging.log_column_values')#
field log_key_values_only: bool | None = False (alias 'logging.log_column_values.log_keys_only')#
field lookup_type: LookupType | None = LookupType.empty#
field lower_port_number: int = None (alias 'load_control.partitioned_db_config.port_range.min_value')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field maximum_partitioning_agents: int | None = 25 (alias 'load_control.partitioned_db_config.max_num_part_agents')#
field maximum_reject_count: int | None = 1 (alias 'session.use_external_tables.max_errors')#
field message_file: str = 'loadMsgs.out' (alias 'load_control.message_file')#
field migrated_job: bool | None = False#
field name_of_table_space: str | None = None (alias 'load_control.allow_access_mode.table_space')#
field non_recoverable_load: bool | None = False (alias 'load_control.non_recoverable_tx')#
field number_of_retries: int = 3 (alias 'load_to_zos.transfer.retry_connection.retry_count')#
field omit_header: bool | None = False (alias 'load_control.partitioned_db_config.omit_header')#
field organize_by: TableActionGenerateCreateStatementCreateTableOrganizeBy | None = TableActionGenerateCreateStatementCreateTableOrganizeBy.database_default (alias 'table_action.generate_create_statement.create_table_organize_by')#
field other_options: str | None = '' (alias 'table_action.generate_create_statement.create_table_other_options')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field output_partition_numbers: str | None = None (alias 'load_control.partitioned_db_config.output_db_part_nums')#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field pad_character: str | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_collecting_statistics: int | None = -1 (alias 'load_control.partitioned_db_config.run_stat_db_partnum')#
field partition_number: int | None = None (alias 'load_to_zos.partition_number')#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned_database_configuration: bool | None = False (alias 'load_control.partitioned_db_config')#
field partitioned_distribution_file: str | None = None (alias 'load_control.partitioned_db_config.dist_file')#
field partitioned_reads_column_name: str = None (alias 'sql.enable_partitioning.partitioning_method.key_field')#
field partitioned_reads_method: SqlEnablePartitioningPartitioningMethod | None = SqlEnablePartitioningPartitioningMethod.minimum_and_maximum_range (alias 'sql.enable_partitioning.partitioning_method')#
field partitioned_reads_table_name: str = None (alias 'sql.enable_partitioning.partitioning_method.table_name')#
field partitioning_partition_numbers: str | None = None (alias 'load_control.partitioned_db_config.partitioning_db_part_nums')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field perform_table_action_first: bool | None = True (alias 'table_action.table_action_first')#
field port_range: bool | None = None (alias 'load_control.partitioned_db_config.port_range')#
field prefix_for_expression_columns: str | None = 'EXPR'#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_after_sql_node_statements_from_file: bool | None = False (alias 'before_after.after_node.read_from_file_after_sql_node')#
field read_after_sql_statements_from_file: bool | None = False (alias 'before_after.after.read_from_file_after_sql')#
field read_before_sql_node_statement_from_file: bool | None = False (alias 'before_after.before_node.read_from_file_before_sql_node')#
field read_before_sql_statements_from_file: bool | None = False (alias 'before_after.before.read_from_file_before_sql')#
field read_create_statement_from_file: bool | None = False (alias 'table_action.generate_create_statement.read_create_statement_from_file')#
field read_drop_statement_from_file: bool | None = False (alias 'table_action.generate_drop_statement.read_drop_statement_from_file')#
field read_select_statement_from_file: bool | None = False (alias 'sql.select_statement.read_from_file_select')#
field read_truncate_statement_from_file: bool | None = False (alias 'table_action.generate_truncate_statement.read_truncate_statement_from_file')#
field record_count: int | None = 2000 (alias 'transaction.record_count')#
field record_ordering: RecordOrdering | None = RecordOrdering.zero#
field reject_condition_row_not_updated: bool | None = False#
field reject_condition_sql_error: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field remove_intermediate_data_file: bool | None = True (alias 'load_control.remove_intermediate_data_file')#
field reoptimization: Reoptimization | None = Reoptimization.none (alias 're_optimization')#
field report_only: bool | None = False (alias 'load_to_zos.image_copy_function.report_only')#
field restart_phase: LoadControlRestartPhase | None = LoadControlRestartPhase.load (alias 'load_control.restart_phase')#
field retry_on_connection_failure: bool | None = True (alias 'load_to_zos.transfer.retry_connection')#
field row_count: int | None = 0 (alias 'load_control.i_row_count')#
field row_count_estimate: int | None = 1000 (alias 'load_to_zos.row_count_estimate')#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field save_count: int | None = 0 (alias 'load_control.save_count')#
field schema_name: str | None = None#
field scope: LoadToZosImageCopyFunctionScope | None = LoadToZosImageCopyFunctionScope.full (alias 'load_to_zos.image_copy_function.scope')#
field select_statement: str = None (alias 'sql.select_statement')#
field select_statement_column: str | None = None (alias 'sql.select_statement.columns.column')#
field set_copy_pending: bool | None = False (alias 'load_to_zos.set_copy_pending')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_buffer_size: int | None = 0 (alias 'load_control.sort_buffer_size')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field sql_delete_statement: str | None = None (alias 'sql.delete_statement.tables.table')#
field sql_delete_statement_parameters: str | None = None (alias 'sql.delete_statement.parameters.parameter')#
field sql_delete_statement_where_clause: str | None = None (alias 'sql.delete_statement.where_clause')#
field sql_insert_statement: str | None = None (alias 'sql.insert_statement.tables.table')#
field sql_insert_statement_parameters: str | None = None (alias 'sql.insert_statement.parameters.parameter')#
field sql_insert_statement_where_clause: str | None = None (alias 'sql.insert_statement.where_clause')#
field sql_other_clause: str | None = None (alias 'sql.other_clause')#
field sql_select_statement_other_clause: str | None = None (alias 'sql.select_statement.other_clause')#
field sql_select_statement_parameters: str | None = None (alias 'sql.select_statement.parameters.parameter')#
field sql_select_statement_table_name: str | None = None (alias 'sql.select_statement.tables.table')#
field sql_select_statement_where_clause: str | None = None (alias 'sql.select_statement.where_clause')#
field sql_update_statement: str | None = None (alias 'sql.update_statement.tables.table')#
field sql_update_statement_parameters: str | None = None (alias 'sql.update_statement.parameters.parameter')#
field sql_update_statement_where_clause: str | None = None (alias 'sql.update_statement.where_clause')#
field sql_user_defined_sql_file_character_set: str | None = None (alias 'sql.user_defined_sql.file.character_set')#
field sql_where_clause: str | None = None (alias 'sql.where_clause')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field statistics_on_columns: str | None = None (alias 'session.use_external_tables.statistics.run_stats_on_columns')#
field status_interval: int | None = 100 (alias 'load_control.partitioned_db_config.status_interval')#
field system_pages: bool | None = True (alias 'load_to_zos.image_copy_function.system_pages')#
field table_action: TableAction = TableAction.append#
field table_action_generate_create_statement_create_table_in: str | None = None (alias 'table_action.generate_create_statement.create_table_in')#
field table_name: str = None#
field target_table_on_db2_for_z_os: bool | None = False (alias 'table_action.generate_create_statement.create_table_on_zos')#
field temporary_files_directory: str | None = None (alias 'load_control.directory_for_tmp_files')#
field temporary_work_table_mode: SessionTemporaryWorkTable | None = SessionTemporaryWorkTable.automatic (alias 'session.temporary_work_table')#
field temporary_work_table_name: str = None (alias 'session.temporary_work_table.table_name')#
field time_commit_interval: int | None = 0 (alias 'transaction.time_interval')#
field total_number_of_player_processes: int = None (alias 'limit_parallelism.player_process_limit')#
field trace: int | None = 0 (alias 'load_control.partitioned_db_config.trace')#
field transfer_command: str | None = None (alias 'load_to_zos.transfer.transfer_cmd')#
field transfer_to: str = None (alias 'load_to_zos.transfer.transfer_to')#
field transfer_type: LoadToZosTransferTransferType | None = LoadToZosTransferTransferType.ftp (alias 'load_to_zos.transfer.transfer_type')#
field truncate_table: bool | None = False (alias 'session.temporary_work_table.truncate_table')#
field truncate_table_statement: str = '' (alias 'table_action.generate_truncate_statement.truncate_statement')#
field type: TableActionGenerateCreateStatementCreateTableCompressCreateTableCompressLuw | None = TableActionGenerateCreateStatementCreateTableCompressCreateTableCompressLuw.adaptive (alias 'table_action.generate_create_statement.create_table_compress.create_table_compress_luw')#
field unique: bool | None = None (alias 'part_unique')#
field unique_key_column: str = None (alias 'sql.use_unique_key_column.unique_key_column')#
field update_columns: str | None = None (alias 'sql.update_columns')#
field update_statement: str = None (alias 'sql.update_statement')#
field use_cas_lite_service: bool | None = True (alias 'use_cas_lite')#
field use_direct_connections: bool | None = False#
field use_et_source: bool | None = True#
field use_external_tables: bool | None = False (alias 'session.use_external_tables')#
field use_unique_key_column: bool | None = False (alias 'sql.use_unique_key_column')#
field user: str | None = None (alias 'load_to_zos.transfer.user')#
field user_defined_sql: SqlUserDefinedSql = SqlUserDefinedSql.statements (alias 'sql.user_defined_sql')#
field user_defined_sql_file_name: str = None (alias 'sql.user_defined_sql.file')#
field user_defined_sql_statements: str = None (alias 'sql.user_defined_sql.statements')#
field user_defined_sql_supress_warnings: bool | None = False (alias 'sql.user_defined_sql.suppress_warnings')#
field uss_pipe_directory: str = None (alias 'load_to_zos.uss_pipe_directory')#
field utility_id: str | None = 'DB2ZLOAD' (alias 'load_to_zos.utility_id')#
field value_compression: bool | None = False (alias 'table_action.generate_create_statement.create_table_value_compression')#
field warning_count: int | None = 0 (alias 'load_control.warning_count')#
field without_prompting: bool | None = False (alias 'load_control.without_prompting')#
field write_mode: WriteMode = WriteMode.insert#
field xml_column_as_lob: bool | None = False#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/DB2ConnectorPX.svg'#
label: ClassVar[str] = 'IBM Db2 for DataStage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'DB2ConnectorPX'#

This module defines configuration or the IBM Db2 for i stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.db2iseries.db2iseries#

Bases: BaseStage

Properties for the IBM Db2 for i stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: Db2iseriesConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "command_timeout": 600,     "location": null,     "driver": "jcc",     "defer_credentials": false,     "gateway_url": null,     "hostname_or_ip_address": null,     "jdbc_driver_files": null,     "max_transport_objects": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/db2iseries.svg'#
label: ClassVar[str] = 'IBM Db2 for i'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'db2iseries'#

This module defines configuration or the IBM Db2 Warehouse stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.db2warehouse.db2warehouse#

Bases: BaseStage

Properties for the IBM Db2 Warehouse stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: Db2warehouseConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "access_token": null,     "api_key": null,     "application_name": null,     "authentication_method": null,     "discover_data_assets": null,     "avoid_timestamp_conversion": null,     "client_accounting_information": null,     "client_hostname": null,     "client_user": null,     "cluster_access_token": null,     "cluster_user_name": null,     "command_timeout": 600,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "use_my_platform_login_credentials": false,     "max_transport_objects": null,     "password": null,     "port": 50001,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": true,     "ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/dashdb.svg'#
label: ClassVar[str] = 'IBM Db2 Warehouse'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'dashdb'#

This module defines configuration or the IBM Db2 for z/OS stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.db2zos.db2zos#

Bases: BaseStage

Properties for the IBM Db2 for z/OS stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: Db2zosConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "application_name": null,     "avoid_timestamp_conversion": null,     "client_accounting_information": null,     "client_hostname": null,     "client_user": null,     "cluster_access_token": null,     "cluster_user_name": null,     "collection_id": null,     "command_timeout": 600,     "location": null,     "defer_credentials": false,     "gateway_url": null,     "hostname_or_ip_address": null,     "jdbc_driver_files": null,     "max_transport_objects": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/db2zos.svg'#
label: ClassVar[str] = 'IBM Db2 for z/OS'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'db2zos'#

This module defines configuration or the Denodo stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.denodo.denodo#

Bases: BaseStage

Properties for the Denodo stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: DenodoConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "login_timeout": null,     "password": null,     "port": null,     "query_timeout": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "ssl_certificate_hostname": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_statement: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field login_timeout: int | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.update_statement#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/denodo.svg'#
label: ClassVar[str] = 'Denodo'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'denodo'#

This module defines configuration or the Apache Derby stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.derby.derby#

Bases: BaseStage

Properties for the Apache Derby stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: DerbyConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/derby.svg'#
label: ClassVar[str] = 'Apache Derby'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'derby'#

This module defines configuration or the Dremio stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.dremio.dremio#

Bases: BaseStage

Properties for the Dremio stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: DremioConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_type": "user_pass",     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "personal_access_token": null,     "port": null,     "dremio_cloud_project_id": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_statement: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/dremio.svg'#
label: ClassVar[str] = 'Dremio'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'dremio'#

This module defines configuration or the Dropbox stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.dropbox.dropbox#

Bases: BaseStage

Properties for the Dropbox stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: DropboxConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "access_token": null,     "app_key_client_id": null,     "app_secret_client_secret": null,     "authentication_method": "accesstoken",     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "refresh_token": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/dropbox.svg'#
label: ClassVar[str] = 'Dropbox'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'dropbox'#

This module defines configuration or the IBM Data Virtualization stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.dv.dv#

Bases: BaseStage

Properties for the IBM Data Virtualization stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: DvConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "access_token": null,     "api_key": null,     "application_name": null,     "auth_method": null,     "auto_discovery": null,     "avoid_timestamp_conversion": null,     "client_accounting_information": null,     "client_hostname": null,     "client_user": null,     "cluster_access_token": null,     "cluster_user_name": null,     "command_timeout": 600,     "database": "bigsql",     "defer_credentials": false,     "host": null,     "inherit_access_token": false,     "instance_environment": null,     "instance_id": null,     "password": null,     "port": null,     "service_name": null,     "sg_gateway_id": null,     "sg_host_original": null,     "sg_http_proxy": null,     "sg_security_token": null,     "sg_service_url": null,     "sl_client_cert": null,     "sl_client_private_key": null,     "sl_connector_id": null,     "sl_endpoint_host": null,     "sl_endpoint_name": null,     "sl_endpoint_port": null,     "sl_host_original": null,     "sl_http_proxy": null,     "sl_location_id": null,     "sl_service_url": null,     "ssl": false,     "ssl_certificate": null,     "use_s2s_ssl_certificate": false,     "username": null,     "vaulted_properties": null,     "properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field database: str = 'bigsql'#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_data_size: int | None = 262143#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field use_s2s_ssl_certificate: bool | None = False#
image: ClassVar[str] = '/data-intg/flows/graphics/alternateCanvasSVG/DataVirtualization_Alt.svg'#
label: ClassVar[str] = 'IBM Data Virtualization'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'dv'#

This module defines configuration or the IBM Data Virtualization Manager for z/OS stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.dvm.dvm#

Bases: BaseStage

Properties for the IBM Data Virtualization Manager for z/OS stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field byte_limit: str | None = None#
field connection: DvmConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "auto_discovery": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "host": null,     "password": null,     "port": null,     "query_timeout": null,     "sg_gateway_id": null,     "sg_host_original": null,     "sg_http_proxy": null,     "sg_security_token": null,     "sg_service_url": null,     "sl_client_cert": null,     "sl_client_private_key": null,     "sl_connector_id": null,     "sl_endpoint_host": null,     "sl_endpoint_name": null,     "sl_endpoint_port": null,     "sl_host_original": null,     "sl_http_proxy": null,     "sl_location_id": null,     "sl_service_url": null,     "ssl": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_max_string_binary_precision: int | None = 20000#
field defer_credentials: bool | None = False#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field execmode: Execmode | None = Execmode.default_par#
field generate_unicode_columns: bool | None = False#
field has_reject_output: bool | None = False#
field input_count: int | None = 0#
field key_column_names: str | None = None#
field output_count: int | None = 0#
field preserve: Preserve | None = Preserve.default_propagate#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field rcp: bool | None = True#
field read_mode: ReadMode | None = ReadMode.general#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool = False#
field schema_name: str | None = None#
field select_statement: str = None#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/dvm.svg'#
label: ClassVar[str] = 'IBM Data Virtualization Manager for z/OS'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'dvm'#

This module defines configuration or the Elasticsearch stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.elasticsearch.elasticsearch#

Bases: BaseStage

Properties for the Elasticsearch stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: ElasticsearchConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "api_key": null,     "authentication_method": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "password": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "ssl_certificate": null,     "url": null,     "use_anonymous_access": false,     "username": null,     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_index_body: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field document_type: str | None = None#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field file_action: FileAction | None = FileAction.append#
field file_name: str = None#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_mode: ReadMode | None = ReadMode.read_single#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field search_query_body: str | None = None (alias 'query_body')#
field search_query_string: str | None = None (alias 'query_string')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field unique: bool | None = None (alias 'part_unique')#
field write_mode: WriteMode | None = WriteMode.write#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/elasticsearch.svg'#
label: ClassVar[str] = 'Elasticsearch'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'elasticsearch'#

This module defines configuration or the Exasol stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.exasol.exasol#

Bases: BaseStage

Properties for the Exasol stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: ExasolConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "fingerprint": null,     "gateway_url": null,     "hostname_or_ip_address": null,     "jar_uris": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/exasol.svg'#
label: ClassVar[str] = 'Exasol'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'exasol'#

This module defines configuration or the FTP stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.ftp.ftp#

Bases: BaseStage

Properties for the FTP stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field allow_per_column_mapping: AllowPerColumnMapping | None = AllowPerColumnMapping.false (alias 'allow_column_mapping')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: FtpConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": null,     "cluster_access_token": null,     "cluster_user_name": null,     "connection_mode": null,     "defer_credentials": false,     "ftadv_strings": null,     "hostname_or_ip_address": null,     "key_passphrase": null,     "access_mvs_dataset": null,     "password": null,     "port": null,     "private_key": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": true,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "use_home_as_root": true,     "username": "anonymous",     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_seq (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field map_name: MapName | None = MapName.UTF_8 (alias 'nls_map_name')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field ssl_certificate_hostname: str | None = None (alias 'ssl_certificate_host')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/ftp.svg'#
label: ClassVar[str] = 'FTP'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'ftp'#

This module defines configuration or the Generic S3 stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.generics3.generics3#

Bases: BaseStage

Properties for the Generic S3 stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field bucket: str | None = None#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: Generics3Conn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "access_key": null,     "bucket": null,     "das_api_key": null,     "das_endpoint": null,     "das_instance_id": null,     "catalog_name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "disable_chunked_encoding": false,     "enable_global_bucket_access": true,     "enable_path_style_access": null,     "s3_list_objects_version": "v1",     "iceberg_rest_catalog_endpoint": null,     "region": null,     "secret_key": null,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "trust_all_ssl_certificates": false,     "url": null,     "use_watsonx_credential_provider": null,     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_bucket: bool | None = False#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field delete_bucket: bool | None = False#
field deltalake_pfk: str | None = None#
field deltalake_pfv: str | None = None#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field endpoint_folder: str | None = None (alias 'table_folder_name')#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_a_file_to_a_row: bool | None = False (alias 'read_file_to_row')#
field read_mode: ReadMode | None = ReadMode.read_single#
field read_part_size: int | None = None#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field table_action: TableAction | None = TableAction.append#
field table_data_file_compression_codec: TableDataFileCompressionCodec | None = None#
field table_data_file_format: TableDataFileFormat | None = TableDataFileFormat.avro#
field table_format: TableFormat | None = None#
field table_name: str | None = None#
field table_namespace: str | None = None#
field the_cache_expiration: str | None = None (alias 'table_partition_cache_expiration')#
field the_cache_size: int | None = None (alias 'table_partition_cache_size')#
field the_data_path: str | None = None (alias 'table_data_path')#
field the_partition_columns: str | None = None (alias 'table_partition_columns')#
field the_partition_paths: str | None = None (alias 'table_partition_path')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field trust_all_ssl_certificates: bool | None = False (alias 'trust_all_ssl_cert')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field write_part_size: int | None = None#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/generics3.svg'#
label: ClassVar[str] = 'Generic S3'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'generics3'#

This module defines configuration or the Google Cloud Storage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.google_cloud_storage.google_cloud_storage#

Bases: BaseStage

Properties for the Google Cloud Storage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field big_query_temp_bucket_name: str | None = None#
field bucket: str | None = None#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: GoogleCloudStorageConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "access_token": null,     "authentication_method": null,     "discover_data_assets": null,     "bucket": null,     "das_api_key": null,     "das_endpoint": null,     "das_instance_id": null,     "catalog_name": null,     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "credentials": null,     "credentials_file_path": null,     "defer_credentials": false,     "iceberg_rest_catalog_endpoint": null,     "project_id": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_protocol": "https",     "proxy_username": null,     "refresh_token": null,     "service_account_email": null,     "service_account_token_lifetime": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "security_token_service_audience": null,     "token_field_name": null,     "token_format": "text",     "token_type": "id_token",     "token_url": null,     "request_body": null,     "http_headers": null,     "http_method": "get",     "use_watsonx_credential_provider": null,     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_bucket: bool | None = False#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field database_name: str | None = None#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field delete_bucket: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field endpoint_folder: str | None = None (alias 'table_folder_name')#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field load_big_query_table: bool | None = False (alias 'create_bigquery_table')#
field location: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field node_count: int | None = 1#
field node_number: int | None = 0#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field read_part_size: int | None = None#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field storage_class: StorageClass | None = StorageClass.standard#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field table_action: TableAction | None = TableAction.append#
field table_data_file_compression_codec: TableDataFileCompressionCodec | None = None#
field table_data_file_format: TableDataFileFormat | None = TableDataFileFormat.avro#
field table_format: TableFormat | None = None#
field table_name: str | None = None#
field table_namespace: str | None = None#
field the_cache_expiration: str | None = None (alias 'table_partition_cache_expiration')#
field the_cache_size: int | None = None (alias 'table_partition_cache_size')#
field the_data_path: str | None = None (alias 'table_data_path')#
field the_partition_columns: str | None = None (alias 'table_partition_columns')#
field the_partition_paths: str | None = None (alias 'table_partition_path')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field write_part_size: int | None = None#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/GoogleCloudStoragePX.svg'#
label: ClassVar[str] = 'Google Cloud Storage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'GoogleCloudStoragePX'#

This module defines configuration or the Google Looker stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.google_looker.google_looker#

Bases: BaseStage

Properties for the Google Looker stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: GoogleLookerConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "port": 19999,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field worksheet_name: str | None = None (alias 'sheet_name')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/looker.svg'#
label: ClassVar[str] = 'Google Looker'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'looker'#

This module defines configuration or the Google Cloud Pub/Sub stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.google_pub_sub.google_pub_sub#

Bases: BaseStage

Properties for the Google Cloud Pub/Sub stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: GooglePubSubConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "access_token": null,     "authentication_method": null,     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_user_name": null,     "credentials": null,     "credentials_file_path": null,     "defer_credentials": false,     "project_id": null,     "refresh_token": null,     "service_account_email": null,     "service_account_token_lifetime": null,     "security_token_service_audience": null,     "token_field_name": null,     "token_format": "text",     "token_type": "id_token",     "token_url": null,     "request_body": null,     "http_headers": null,     "http_method": "get",     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field dummy: str | None = None (alias 'dummy_props')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field subscription_id: str | None = None#
field timeout_after_last_message: int | None = 300#
field topic_id: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/GooglePubSubPX.svg'#
label: ClassVar[str] = 'Google Cloud Pub/Sub'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'GooglePubSubPX'#

This module defines configuration or the Greenplum stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.greenplum.greenplum#

Bases: BaseStage

Properties for the Greenplum stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: GreenplumConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "login_timeout": null,     "password": null,     "port": null,     "prepare_statement_support": false,     "query_timeout": 300,     "retry_limit": 2,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field login_timeout: int | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/greenplum.svg'#
label: ClassVar[str] = 'Greenplum'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'greenplum'#

This module defines configuration or the Apache HDFS stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.hdfs_apache.hdfs_apache#

Bases: BaseStage

Properties for the Apache HDFS stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = CodecAvro.null#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = CodecOrc.none#
field codec_parquet: CodecParquet | None = CodecParquet.snappy#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: HdfsApacheConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": "password",     "cluster_access_token": null,     "cluster_user_name": null,     "connect_to_apache_hive": false,     "defer_credentials": false,     "hive_database": null,     "hive_host": null,     "hive_http_path": null,     "hive_keytab_file": null,     "hive_password": null,     "hive_port": null,     "hive_service_principal_name": null,     "enable_ssl_for_hive": true,     "hive_username": null,     "hive_user_principal_name": null,     "keytab_file": null,     "password": null,     "service_principal_name": null,     "ssl_certificate": null,     "url": null,     "use_home_as_root": true,     "user_principal_name": null,     "username": null,     "vaulted_properties": null,     "ds_host": null,     "ds_port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "file_system_type": null,     "hive_login_config_name": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_or_use_existing_hive_table: bool | None = False (alias 'create_hive_table')#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_cleanup: bool | None = True (alias '_cleanup')#
field ds_create_hive_table: bool | None = False (alias '_create_hive_table')#
field ds_create_hive_table_additional_driver_params: str | None = None (alias '_create_hive_table.additional_driver_params')#
field ds_create_hive_table_create_hive_schema: bool | None = False (alias '_create_hive_table.create_hive_schema')#
field ds_create_hive_table_drop_hive_table: bool | None = True (alias '_create_hive_table.drop_hive_table')#
field ds_create_hive_table_hive_kerberos: bool | None = False (alias '_create_hive_table.hive_kerberos')#
field ds_create_hive_table_hive_keytab: str = None (alias '_create_hive_table.hive_keytab')#
field ds_create_hive_table_hive_service_principal: str | None = None (alias '_create_hive_table.hive_service_principal')#
field ds_create_hive_table_hive_table: str = None (alias '_create_hive_table.hive_table')#
field ds_create_hive_table_hive_table_type: DSCreateHiveTableHiveTableType | None = DSCreateHiveTableHiveTableType.external (alias '_create_hive_table.hive_table_type')#
field ds_create_hive_table_hive_use_keytab: bool | None = False (alias '_create_hive_table.hive_use_keytab')#
field ds_create_hive_table_use_staging_table: bool | None = False (alias '_create_hive_table.use_staging_table')#
field ds_create_hive_table_use_staging_table_hive_target_table_properties_hive_drop_staging_table: bool | None = True (alias '_create_hive_table.use_staging_table.hive_target_table_properties.hive_drop_staging_table')#
field ds_create_hive_table_use_staging_table_hive_target_table_properties_hive_orc_compress: DSCreateHiveTableUseStagingTableHiveTargetTablePropertiesHiveOrcCompress | None = DSCreateHiveTableUseStagingTableHiveTargetTablePropertiesHiveOrcCompress.zlib (alias '_create_hive_table.use_staging_table.hive_target_table_properties.hive_orc_compress')#
field ds_create_hive_table_use_staging_table_hive_target_table_properties_hive_orc_stripe_size: int | None = 64 (alias '_create_hive_table.use_staging_table.hive_target_table_properties.hive_orc_stripe_size')#
field ds_create_hive_table_use_staging_table_hive_target_table_properties_hive_parquet_compress: DSCreateHiveTableUseStagingTableHiveTargetTablePropertiesHiveParquetCompress | None = DSCreateHiveTableUseStagingTableHiveTargetTablePropertiesHiveParquetCompress.snappy (alias '_create_hive_table.use_staging_table.hive_target_table_properties.hive_parquet_compress')#
field ds_create_hive_table_use_staging_table_hive_target_table_properties_hive_target_table_format: DSCreateHiveTableUseStagingTableHiveTargetTablePropertiesHiveTargetTableFormat | None = DSCreateHiveTableUseStagingTableHiveTargetTablePropertiesHiveTargetTableFormat.parquet (alias '_create_hive_table.use_staging_table.hive_target_table_properties.hive_target_table_format')#
field ds_create_hive_table_use_staging_table_hive_target_table_properties_hive_target_table_location: str | None = None (alias '_create_hive_table.use_staging_table.hive_target_table_properties.hive_target_table_location')#
field ds_create_hive_table_use_staging_table_hive_target_table_properties_hive_target_table_type: DSCreateHiveTableUseStagingTableHiveTargetTablePropertiesHiveTargetTableType | None = DSCreateHiveTableUseStagingTableHiveTargetTablePropertiesHiveTargetTableType.external (alias '_create_hive_table.use_staging_table.hive_target_table_properties.hive_target_table_type')#
field ds_create_hive_table_use_staging_table_load_existing_table_max_dynamic_partitions: int | None = 1000 (alias '_create_hive_table.use_staging_table.load_existing_table.max_dynamic_partitions')#
field ds_exclude_files: str | None = None (alias '_exclude_files')#
field ds_file_exists: DSFileExists | None = DSFileExists.overwrite_file (alias '_file_exists')#
field ds_file_format: DSFileFormat | None = DSFileFormat.delimited (alias '_file_format')#
field ds_file_format_avro_source_output_json: bool | None = False (alias '_file_format.avro_source.output_json')#
field ds_file_format_avro_target_avro_array_keys: str | None = None (alias '_file_format.avro_target.avro_array_keys')#
field ds_file_format_avro_target_avro_codec: DSFileFormatAvroTargetAvroCodec | None = DSFileFormatAvroTargetAvroCodec.none (alias '_file_format.avro_target.avro_codec')#
field ds_file_format_avro_target_avro_schema: str = None (alias '_file_format.avro_target.avro_schema')#
field ds_file_format_avro_target_input_json: bool | None = False (alias '_file_format.avro_target.input_json')#
field ds_file_format_delimited_syntax_encoding: str | None = None (alias '_file_format.delimited_syntax.encoding')#
field ds_file_format_delimited_syntax_encoding_output_bom: bool | None = False (alias '_file_format.delimited_syntax.encoding.output_bom')#
field ds_file_format_delimited_syntax_escape: str | None = None (alias '_file_format.delimited_syntax.escape')#
field ds_file_format_delimited_syntax_field_delimiter: str | None = ',' (alias '_file_format.delimited_syntax.field_delimiter')#
field ds_file_format_delimited_syntax_field_formats_date_format: str | None = None (alias '_file_format.delimited_syntax.field_formats.date_format')#
field ds_file_format_delimited_syntax_field_formats_decimal_format: str | None = None (alias '_file_format.delimited_syntax.field_formats.decimal_format')#
field ds_file_format_delimited_syntax_field_formats_time_format: str | None = None (alias '_file_format.delimited_syntax.field_formats.time_format')#
field ds_file_format_delimited_syntax_field_formats_timestamp_format: str | None = None (alias '_file_format.delimited_syntax.field_formats.timestamp_format')#
field ds_file_format_delimited_syntax_header: bool | None = False (alias '_file_format.delimited_syntax.header')#
field ds_file_format_delimited_syntax_header_include_types: bool | None = False (alias '_file_format.delimited_syntax.header.include_types')#
field ds_file_format_delimited_syntax_null_value: str | None = None (alias '_file_format.delimited_syntax.null_value')#
field ds_file_format_delimited_syntax_quotes: DSFileFormatDelimitedSyntaxQuotes | None = DSFileFormatDelimitedSyntaxQuotes.none (alias '_file_format.delimited_syntax.quotes')#
field ds_file_format_delimited_syntax_record_def: DSFileFormatDelimitedSyntaxRecordDef | None = DSFileFormatDelimitedSyntaxRecordDef.none (alias '_file_format.delimited_syntax.record_def')#
field ds_file_format_delimited_syntax_record_def_record_def_source: str | None = None (alias '_file_format.delimited_syntax.record_def.record_def_source')#
field ds_file_format_delimited_syntax_record_limit: int | None = None (alias '_file_format.delimited_syntax.record_limit')#
field ds_file_format_delimited_syntax_row_delimiter: str | None = '<NL>' (alias '_file_format.delimited_syntax.row_delimiter')#
field ds_file_format_impl_syntax_binary: DSFileFormatImplSyntaxBinary | None = DSFileFormatImplSyntaxBinary.binary (alias '_file_format.impl_syntax.binary')#
field ds_file_format_impl_syntax_encoding: str | None = None (alias '_file_format.impl_syntax.encoding')#
field ds_file_format_impl_syntax_encoding_output_bom: bool | None = False (alias '_file_format.impl_syntax.encoding.output_bom')#
field ds_file_format_impl_syntax_header: bool | None = False (alias '_file_format.impl_syntax.header')#
field ds_file_format_impl_syntax_header_include_types: bool | None = False (alias '_file_format.impl_syntax.header.include_types')#
field ds_file_format_impl_syntax_record_def: DSFileFormatImplSyntaxRecordDef | None = DSFileFormatImplSyntaxRecordDef.none (alias '_file_format.impl_syntax.record_def')#
field ds_file_format_impl_syntax_record_def_record_def_source: str | None = None (alias '_file_format.impl_syntax.record_def.record_def_source')#
field ds_file_format_impl_syntax_record_limit: int | None = None (alias '_file_format.impl_syntax.record_limit')#
field ds_file_format_orc_target_orc_buffer_size: int | None = 10000 (alias '_file_format.orc_target.orc_buffer_size')#
field ds_file_format_orc_target_orc_compress: DSFileFormatOrcTargetOrcCompress | None = DSFileFormatOrcTargetOrcCompress.snappy (alias '_file_format.orc_target.orc_compress')#
field ds_file_format_orc_target_orc_stripe_size: int | None = 100000 (alias '_file_format.orc_target.orc_stripe_size')#
field ds_file_format_parquet_target_parquet_block_size: int | None = 10000000 (alias '_file_format.parquet_target.parquet_block_size')#
field ds_file_format_parquet_target_parquet_compress: DSFileFormatParquetTargetParquetCompress | None = DSFileFormatParquetTargetParquetCompress.snappy (alias '_file_format.parquet_target.parquet_compress')#
field ds_file_format_parquet_target_parquet_page_size: int | None = 10000 (alias '_file_format.parquet_target.parquet_page_size')#
field ds_file_format_trace_file: str | None = None (alias '_file_format.trace_file')#
field ds_filename_column: str | None = None (alias '_filename_column')#
field ds_filename_source: str = None (alias '_filename_source')#
field ds_filename_target: str = None (alias '_filename_target')#
field ds_force_sequential: bool | None = False (alias '_force_sequential')#
field ds_java_heap_size: int | None = 256 (alias '_java.heap_size')#
field ds_max_file_size: int | None = 0 (alias '_max_file_size')#
field ds_read_mode: DSReadMode | None = DSReadMode.read_single_file (alias '_read_mode')#
field ds_reject_mode: DSRejectMode | None = DSRejectMode.cont (alias '_reject_mode')#
field ds_split_on_key: bool | None = False (alias '_split_on_key')#
field ds_split_on_key_case_sensitive: bool | None = False (alias '_split_on_key.case_sensitive')#
field ds_split_on_key_exclude_part_string: bool | None = False (alias '_split_on_key.exclude_part_string')#
field ds_split_on_key_key_column: str | None = None (alias '_split_on_key.key_column')#
field ds_split_on_key_key_in_filename: bool | None = False (alias '_split_on_key.key_in_filename')#
field ds_use_datastage: bool | None = True (alias '_use_datastage')#
field ds_user_class_name: str | None = 'com.ibm.iis.cc.filesystem.FileSystem' (alias '_user_class_name')#
field ds_wave_handling_append_uid: bool | None = False (alias '_wave_handling.append_uid')#
field ds_wave_handling_file_size_threshold: int | None = 1 (alias '_wave_handling.file_size_threshold')#
field ds_write_mode: DSWriteMode | None = DSWriteMode.write_single_file (alias '_write_mode')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field endpoint_folder: str | None = None (alias 'table_folder_name')#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field hive_table: str | None = None#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field is_reject: bool | None = False#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field node_count: int | None = 1#
field node_number: int | None = 0#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_avro_as_json: bool | None = None#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field table_action: TableAction | None = TableAction.append#
field table_data_file_compression_codec: TableDataFileCompressionCodec | None = None#
field table_data_file_format: TableDataFileFormat | None = TableDataFileFormat.avro#
field table_format: TableFormat | None = None#
field table_name: str | None = None#
field table_namespace: str | None = None#
field the_cache_expiration: str | None = None (alias 'table_partition_cache_expiration')#
field the_cache_size: int | None = None (alias 'table_partition_cache_size')#
field the_data_path: str | None = None (alias 'table_data_path')#
field the_partition_columns: str | None = None (alias 'table_partition_columns')#
field the_partition_paths: str | None = None (alias 'table_partition_path')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/hdfs-apache.svg'#
label: ClassVar[str] = 'Apache HDFS'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'hdfs-apache'#

This module defines configuration or the HTTP stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.http.http#

Bases: BaseStage

Properties for the HTTP stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: HttpConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "ssl_certificate": null,     "url": null,     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field ds_use_datastage: bool = False (alias '_use_datastage')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str | None = None#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/http.svg'#
label: ClassVar[str] = 'HTTP'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'http'#

This module defines configuration or the IBM MQ stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.ibm_mq.ibm_mq#

Bases: BaseStage

Properties for the IBM MQ stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field access_mode: AccessMode | None = AccessMode.as_in_queue_definition#
field alternate_user_id: str | None = None (alias 'other_queue_settings.alternate_user_id')#
field append_node_number: bool | None = True (alias 'work_queue.append_node_number')#
field binding_mode: OtherQueueSettingsClusterQueueBindingMode | None = OtherQueueSettingsClusterQueueBindingMode.as_in_queue_definition (alias 'other_queue_settings.cluster_queue.binding_mode')#
field blocking_transaction_processing: bool | None = False (alias 'transaction.end_of_day')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field cipher_spec: str | None = 'ECDHE_RSA_AES_128_CBC_SHA256' (alias 'ssl_cipher_spec')#
field cluster_queue: bool | None = False (alias 'other_queue_settings.cluster_queue')#
field cluster_queue_manager_name: str | None = None (alias 'other_queue_settings.cluster_queue.queue_manager_name')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: IbmMqConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "cas_lite_service_authorization_header": null,     "channel_name": null,     "connection_name": null,     "transport_type": "tcp",     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "password": null,     "queue_manager_name": null,     "cipher_spec": "ECDHE_RSA_AES_128_CBC_SHA256",     "client_ssl_certificate": null,     "client_ssl_key": null,     "ssl_connection": false,     "server_ssl_certificate": null,     "use_cas_lite_service": true,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field content_filter: str | None = None (alias 'pub_sub.content_filter')#
field context_mode: ContextMode | None = ContextMode.none#
field custom_value: str | None = None (alias 'pub_sub.publish.publication_format.custom_value')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field deregistration: bool | None = False (alias 'pub_sub.deregistration')#
field deregistration_correlation_id: str | None = None (alias 'pub_sub.deregistration.deregistration_correl_id')#
field deregistration_topic: str | None = None (alias 'pub_sub.deregistration.deregistration_topic')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field dynamic_queue_name: str = '*' (alias 'other_queue_settings.dynamic_queue.name')#
field dynamic_reply_queue: bool | None = False (alias 'pub_sub.pub_sub_dynamic_reply_to_queue')#
field dynamic_reply_queue_close_options: OtherQueueSettingsDynamicQueueCloseOptions | None = OtherQueueSettingsDynamicQueueCloseOptions.none (alias 'other_queue_settings.dynamic_queue.close_options')#
field dynamic_reply_queue_name: str = '*' (alias 'pub_sub.pub_sub_dynamic_reply_to_queue.name')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_payload_reference: bool | None = False (alias 'message_options.pass_by_reference')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field end_of_data: bool | None = True (alias 'transaction.end_of_wave.end_of_data')#
field end_of_data_message_type: int | None = None#
field end_of_wave: TransactionEndOfWave | None = TransactionEndOfWave.none (alias 'transaction.end_of_wave')#
field error_queue: bool | None = False#
field error_queue_context_mode: ErrorQueueContextMode | None = ErrorQueueContextMode.none (alias 'error_queue.context_mode')#
field error_queue_name: str = None (alias 'error_queue.name')#
field error_queue_queue_manager_name: str | None = '' (alias 'error_queue.queue_manager_name')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field extract_key: bool | None = False (alias 'message_options.extract_key')#
field extract_key_length: int = 0 (alias 'message_options.extract_key.key_length')#
field extract_key_offset: int = 0 (alias 'message_options.extract_key.key_offset')#
field filter_acceptable_value_correlation_id: str | None = None (alias 'header_fields_filter.correl_id.value')#
field filter_confirm_on_arrival: HeaderFieldsFilterReportValue | None = None (alias 'header_fields_filter.report.value')#
field filter_correlation_id_is_hex: bool | None = False (alias 'header_fields_filter.correl_id.hex')#
field filter_group_id_is_hex: bool | None = False (alias 'header_fields_filter.group_id.hex')#
field filter_group_id_use_wildcard: bool | None = False (alias 'header_fields_filter.group_id.use_wildcard')#
field filter_match_all_report: bool | None = False (alias 'header_fields_filter.report.must_match_all')#
field filter_message_flags_must_match_all: bool | None = False (alias 'header_fields_filter.msg_flags.must_match_all')#
field filter_message_id_is_hex: bool | None = False (alias 'header_fields_filter.msg_id.hex')#
field filter_messages: bool | None = False (alias 'header_fields_filter')#
field filter_source_acceptable_application_id_data: str | None = None (alias 'header_fields_filter.appl_identity_data')#
field filter_source_acceptable_backout_count: str | None = None (alias 'header_fields_filter.backout_count')#
field filter_source_acceptable_coded_character_set_identifer: str | None = None (alias 'header_fields_filter.coded_char_set_id')#
field filter_source_acceptable_encoding_value: str | None = None (alias 'header_fields_filter.encoding')#
field filter_source_acceptable_expiry_interval_value: str | None = None (alias 'header_fields_filter.expiry')#
field filter_source_acceptable_feedback_custom_value: str | None = None (alias 'header_fields_filter.feedback.custom_value')#
field filter_source_acceptable_feedback_system_value: HeaderFieldsFilterFeedbackSystemValue | None = None (alias 'header_fields_filter.feedback.system_value')#
field filter_source_acceptable_format_custom_value: str | None = None (alias 'header_fields_filter.format.custom_value')#
field filter_source_acceptable_format_system_value: HeaderFieldsFilterFormatSystemValue | None = None (alias 'header_fields_filter.format.system_value')#
field filter_source_acceptable_group_id: str | None = None (alias 'header_fields_filter.group_id.value')#
field filter_source_acceptable_message_flag_values: HeaderFieldsFilterMsgFlagsValue | None = None (alias 'header_fields_filter.msg_flags.value')#
field filter_source_acceptable_message_payload_size_value: str | None = None (alias 'header_fields_filter.msg_payload_size')#
field filter_source_acceptable_message_sequence_number_value: str | None = None (alias 'header_fields_filter.msg_seq_number')#
field filter_source_acceptable_message_type_custom_value: str | None = None (alias 'header_fields_filter.msg_type.custom_value')#
field filter_source_acceptable_message_type_system_value: HeaderFieldsFilterMsgTypeSystemValue | None = None (alias 'header_fields_filter.msg_type.system_value')#
field filter_source_acceptable_offset_value: str | None = None (alias 'header_fields_filter.offset')#
field filter_source_acceptable_original_length_value: str | None = None (alias 'header_fields_filter.original_length')#
field filter_source_acceptable_persistence_value: HeaderFieldsFilterPersistence | None = None (alias 'header_fields_filter.persistence')#
field filter_source_acceptable_priority_value: str | None = None (alias 'header_fields_filter.priority')#
field filter_source_acceptable_put_application_name_value: str | None = None (alias 'header_fields_filter.put_appl_name')#
field filter_source_acceptable_put_application_type_custom_value: str | None = None (alias 'header_fields_filter.put_appl_type.custom_value')#
field filter_source_acceptable_put_application_type_system_value: HeaderFieldsFilterPutApplTypeSystemValue | None = None (alias 'header_fields_filter.put_appl_type.system_value')#
field filter_source_acceptable_put_date_value: str | None = None (alias 'header_fields_filter.put_date')#
field filter_source_acceptable_put_time_value: str | None = None (alias 'header_fields_filter.put_time')#
field filter_source_acceptable_reply_to_queue_manager_value: str | None = None (alias 'header_fields_filter.reply_to_q_mgr')#
field filter_source_acceptable_reply_to_queue_value: str | None = None (alias 'header_fields_filter.reply_to_q')#
field filter_source_acceptable_user_id_value: str | None = None (alias 'header_fields_filter.user_identifier')#
field filter_source_accptable_accounting_token_value: str | None = None (alias 'header_fields_filter.accounting_token.value')#
field filter_source_application_origin_data: str | None = None (alias 'header_fields_filter.appl_origin_data')#
field filter_treat_accounting_token_as_hex: bool | None = False (alias 'header_fields_filter.accounting_token.hex')#
field filter_use_wildcard_accounting_token: bool | None = False (alias 'header_fields_filter.accounting_token.use_wildcard')#
field filter_use_wildcard_correlation_id: bool | None = False (alias 'header_fields_filter.correl_id.use_wildcard')#
field filter_use_wildcard_message_id: str | None = None (alias 'header_fields_filter.msg_id.value')#
field flow_dirty: str | None = 'false'#
field hex: bool | None = True (alias 'other_queue_settings.alternate_security_id.hex')#
field hide: bool | None = False#
field identity_options: PubSubRegistrationSubscriberIdentity | None = None (alias 'pub_sub.registration.subscriber_identity')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column: list | None = [] (alias 'record_ordering.key_column')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_depth: int = None (alias 'work_queue.monitor_queue_depth.max_queue_depth')#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field message_coded_character_set_id: int | None = 0 (alias 'message_options.message_conversion.coded_char_set_id')#
field message_content_descriptor: bool | None = False (alias 'pub_sub.publish.message_content_descriptor')#
field message_controlled: bool | None = False (alias 'transaction.message_controlled')#
field message_conversion_encoding: int | None = -1 (alias 'message_options.message_conversion.encoding')#
field message_options: bool | None = False#
field message_order_and_assembly: MessageOptionsMessageOrderAndAssembly | None = MessageOptionsMessageOrderAndAssembly.individual_ordered (alias 'message_options.message_order_and_assembly')#
field message_publication_options: PubSubPublishPublication | None = None (alias 'pub_sub.publish.publication')#
field message_quantity: int | None = -1#
field message_read_mode: MessageReadMode | None = MessageReadMode.delete_under_transaction#
field message_service_domain: PubSubPublishMessageContentDescriptorMessageServiceDomain = PubSubPublishMessageContentDescriptorMessageServiceDomain.mrm (alias 'pub_sub.publish.message_content_descriptor.message_service_domain')#
field message_set: str | None = '' (alias 'pub_sub.publish.message_content_descriptor.message_set')#
field message_type: str | None = '' (alias 'pub_sub.publish.message_content_descriptor.message_type')#
field message_write_mode: MessageWriteMode | None = MessageWriteMode.create_under_transaction#
field minimum_depth: int = None (alias 'work_queue.monitor_queue_depth.min_queue_depth')#
field monitor_queue_depth: bool | None = False (alias 'work_queue.monitor_queue_depth')#
field name: str = None (alias 'work_queue.name')#
field open_as_dynamic_queue: bool | None = False (alias 'other_queue_settings.dynamic_queue')#
field other_queue_settings: bool | None = False#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = None (alias 'outputcolProperties')#
field pad_message_payload: bool | None = False (alias 'message_options.message_padding')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field peform_message_conversion: bool | None = False (alias 'message_options.message_conversion')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field period: int | None = -1 (alias 'refresh.period')#
field persistence_options: PubSubRegistrationSubscriberPersistence | None = PubSubRegistrationSubscriberPersistence.persistent_as_publish (alias 'pub_sub.registration.subscriber_persistence')#
field physical_format: str | None = '' (alias 'pub_sub.publish.message_content_descriptor.mrm_physical_format')#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field process_end_of_data_message: bool | None = True (alias 'end_of_data_message_type.process_end_of_data_message')#
field publication_message_topic: str | None = None (alias 'pub_sub.publish.publish_topic')#
field publish_subscribe: bool | None = False (alias 'pub_sub')#
field publisher_general_deregistration_options: PubSubDeregistrationPublisher | None = None (alias 'pub_sub.deregistration.publisher')#
field publisher_general_registration_options: PubSubRegistrationPublisher | None = None (alias 'pub_sub.registration.publisher')#
field queue_name: str | None = ''#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field record_count: int | None = 0 (alias 'transaction.record_count')#
field record_ordering: RecordOrdering | None = RecordOrdering.zero#
field refresh: bool | None = False#
field registration: bool | None = False (alias 'pub_sub.registration')#
field registration_correlation_id: str | None = None (alias 'pub_sub.registration.registration_correl_id')#
field registration_options: PubSubPublishRegistration | None = None (alias 'pub_sub.publish.registration')#
field registration_topics: str = None (alias 'pub_sub.registration.registration_topic')#
field reject_condition_row_not_updated: bool | None = False#
field reject_condition_sql_error: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field remote_transmission_queue_name: str | None = None (alias 'other_queue_settings.transmission_queue_name')#
field remove_mqrfh2_header: bool | None = False (alias 'message_options.remove_mqrfh2header')#
field reply_queue: str | None = None (alias 'pub_sub.pub_sub_reply_to_queue')#
field row_buffer_count: int | None = 1 (alias 'message_options.row_buffer_count')#
field runtime_column_propagation: bool | None = None#
field seperate_message_into_segments: bool | None = False (alias 'message_options.create_segmented_message')#
field service_type: PubSubServiceType | None = PubSubServiceType.mqrfh (alias 'pub_sub.service_type')#
field set_header_fields: bool | None = False (alias 'header_fields_setter')#
field set_message_id_to_column_value: bool | None = False (alias 'message_options.set_message_id_column_value')#
field setter_acceptable_value_correlation_id: str | None = None (alias 'header_fields_setter.correl_id.value')#
field setter_correlation_id_is_hex: bool | None = False (alias 'header_fields_setter.correl_id.hex')#
field setter_destination_header_version: HeaderFieldsSetterVersion | None = HeaderFieldsSetterVersion.two (alias 'header_fields_setter.version')#
field setter_destination_message_flags: HeaderFieldsSetterMsgFlags | None = None (alias 'header_fields_setter.msg_flags')#
field setter_destination_report_value: HeaderFieldsSetterReport | None = None (alias 'header_fields_setter.report')#
field setter_group_id_is_hex: bool | None = False (alias 'header_fields_setter.group_id.hex')#
field setter_message_id_is_hex: bool | None = False (alias 'header_fields_setter.msg_id.hex')#
field setter_source_acceptable_application_id_data: str | None = None (alias 'header_fields_setter.appl_identity_data')#
field setter_source_acceptable_coded_character_set_identifer: int | None = 0 (alias 'header_fields_setter.coded_char_set_id')#
field setter_source_acceptable_encoding_value: int | None = -1 (alias 'header_fields_setter.encoding')#
field setter_source_acceptable_expiry_interval_value: int | None = -1 (alias 'header_fields_setter.expiry')#
field setter_source_acceptable_feedback_custom_value: int | None = None (alias 'header_fields_setter.feedback.custom_value')#
field setter_source_acceptable_feedback_system_value: HeaderFieldsSetterFeedbackSystemValue | None = HeaderFieldsSetterFeedbackSystemValue.none (alias 'header_fields_setter.feedback.system_value')#
field setter_source_acceptable_format_custom_value: str | None = None (alias 'header_fields_setter.format.custom_value')#
field setter_source_acceptable_format_system_value: HeaderFieldsSetterFormatSystemValue | None = HeaderFieldsSetterFormatSystemValue.mqstr (alias 'header_fields_setter.format.system_value')#
field setter_source_acceptable_group_id: str | None = None (alias 'header_fields_setter.group_id.value')#
field setter_source_acceptable_message_sequence_number_value: int | None = 1 (alias 'header_fields_setter.msg_seq_number')#
field setter_source_acceptable_message_type_custom_value: int | None = None (alias 'header_fields_setter.msg_type.custom_value')#
field setter_source_acceptable_message_type_system_value: HeaderFieldsSetterMsgTypeSystemValue | None = HeaderFieldsSetterMsgTypeSystemValue.datagram (alias 'header_fields_setter.msg_type.system_value')#
field setter_source_acceptable_offset_value: int | None = 0 (alias 'header_fields_setter.offset')#
field setter_source_acceptable_persistence_value: HeaderFieldsSetterPersistence | None = HeaderFieldsSetterPersistence.as_in_queue_definition (alias 'header_fields_setter.persistence')#
field setter_source_acceptable_priority_value: int | None = -1 (alias 'header_fields_setter.priority')#
field setter_source_acceptable_put_application_name_value: str | None = None (alias 'header_fields_setter.put_appl_name')#
field setter_source_acceptable_put_application_type_custom_value: int | None = None (alias 'header_fields_setter.put_appl_type.custom_value')#
field setter_source_acceptable_put_application_type_system_value: HeaderFieldsSetterPutApplTypeSystemValue | None = HeaderFieldsSetterPutApplTypeSystemValue.nocontext (alias 'header_fields_setter.put_appl_type.system_value')#
field setter_source_acceptable_put_date_value: str | None = None (alias 'header_fields_setter.put_date')#
field setter_source_acceptable_put_time_value: str | None = None (alias 'header_fields_setter.put_time')#
field setter_source_acceptable_reply_to_queue_manager_value: str | None = None (alias 'header_fields_setter.reply_to_q_mgr')#
field setter_source_acceptable_reply_to_queue_value: str | None = None (alias 'header_fields_setter.reply_to_q')#
field setter_source_acceptable_user_id_value: str | None = None (alias 'header_fields_setter.user_identifier')#
field setter_source_accptable_accounting_token_value: str | None = None (alias 'header_fields_setter.accounting_token.value')#
field setter_source_application_origin_data: str | None = None (alias 'header_fields_setter.appl_origin_data')#
field setter_treat_accounting_token_as_hex: bool | None = False (alias 'header_fields_setter.accounting_token.hex')#
field setter_use_wildcard_message_id: str | None = None (alias 'header_fields_setter.msg_id.value')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field size_of_message_segments: int = 1024 (alias 'message_options.create_segmented_message.segment_size')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field sql_select_statement: str | None = None (alias 'sql.select_statement')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field start_value: int | None = 1 (alias 'pub_sub.publish.msg_seq_number.start_value')#
field stream_name: str | None = 'SYSTEM.BROKER.DEFAULT.STREAM' (alias 'pub_sub.stream_name')#
field subscriber_general_deregistration_options: PubSubDeregistrationSubscriber | None = None (alias 'pub_sub.deregistration.subscriber')#
field subscriber_general_registration_options: PubSubRegistrationSubscriberGeneral | None = None (alias 'pub_sub.registration.subscriber_general')#
field subscription_identity: str | None = None (alias 'pub_sub.sub_identity')#
field subscription_name: str | None = None (alias 'pub_sub.sub_name')#
field subscription_point: str | None = None (alias 'pub_sub.sub_point')#
field system_value: PubSubPublishPublicationFormatSystemValue | None = PubSubPublishPublicationFormatSystemValue.mqstr (alias 'pub_sub.publish.publication_format.system_value')#
field time_interval: int | None = 0 (alias 'transaction.time_interval')#
field timeout: int | None = -1 (alias 'transaction.end_of_day.timeout')#
field timestamp: bool | None = False (alias 'pub_sub.publish.timestamp')#
field transaction_end_of_day_method_name: str = '' (alias 'transaction.end_of_day.method_name')#
field transaction_end_of_day_module_name: str = '' (alias 'transaction.end_of_day.module_name')#
field transaction_message_controlled_method_name: str = '' (alias 'transaction.message_controlled.method_name')#
field transaction_message_controlled_module_name: str = '' (alias 'transaction.message_controlled.module_name')#
field transmission_queue: str | None = None (alias 'error_queue.tranmission_queue_name')#
field treat_eol_as_row_terminator: bool | None = False (alias 'message_options.treat_eol_as_row_terminator')#
field truncate_message: bool | None = True (alias 'message_options.message_truncation')#
field unique: bool | None = None (alias 'part_unique')#
field update_message_sequence_number: bool | None = False (alias 'pub_sub.publish.msg_seq_number')#
field use_cas_lite_service: bool | None = True (alias 'use_cas_lite')#
field value: str | None = None (alias 'other_queue_settings.alternate_security_id.value')#
field wait_time: int | None = -1#
field work_queue_context_mode: WorkQueueContextMode | None = WorkQueueContextMode.set_all (alias 'work_queue.context_mode')#
image: ClassVar[str] = '/data-intg/flows/graphics/alternateCanvasSVG/WebSphereMQConnectorPX_Alt.svg'#
label: ClassVar[str] = 'IBM MQ'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'WebSphereMQConnectorPX'#

This module defines configuration or the Apache Impala stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.impala.impala#

Bases: BaseStage

Properties for the Apache Impala stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: ImpalaConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": "password",     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "impersonate_user": null,     "kerberos_sso": null,     "kerberos_sso_keytab": null,     "kerberos_sso_principal": null,     "keytab_file": null,     "password": null,     "port": null,     "service_principal_name": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "user_principal_name": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "login_config_name": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_before_after: bool | None = False (alias '_before_after')#
field ds_before_after_after_sql: str | None = None (alias '_before_after._after_sql')#
field ds_before_after_after_sql_fail_on_error: bool | None = True (alias '_before_after._after_sql._fail_on_error')#
field ds_before_after_after_sql_node: str | None = None (alias '_before_after._after_sql_node')#
field ds_before_after_after_sql_node_fail_on_error: bool | None = True (alias '_before_after._after_sql_node._fail_on_error')#
field ds_before_after_after_sql_node_read_from_file_after_sql_node: bool | None = False (alias '_before_after._after_sql_node._read_from_file_after_sql_node')#
field ds_before_after_after_sql_read_from_file_after_sql: bool | None = False (alias '_before_after._after_sql._read_from_file_after_sql')#
field ds_before_after_before_sql: str | None = None (alias '_before_after._before_sql')#
field ds_before_after_before_sql_fail_on_error: bool | None = True (alias '_before_after._before_sql._fail_on_error')#
field ds_before_after_before_sql_node: str | None = None (alias '_before_after._before_sql_node')#
field ds_before_after_before_sql_node_fail_on_error: bool | None = True (alias '_before_after._before_sql_node._fail_on_error')#
field ds_before_after_before_sql_node_read_from_file_before_sql_node: bool | None = False (alias '_before_after._before_sql_node._read_from_file_before_sql_node')#
field ds_before_after_before_sql_read_from_file_before_sql: bool | None = False (alias '_before_after._before_sql._read_from_file_before_sql')#
field ds_custom_statements: str | None = None (alias '_custom_statements')#
field ds_custom_statements_read_from_file_custom: bool | None = False (alias '_custom_statements._read_from_file_custom')#
field ds_delete_statement: str = None (alias '_delete_statement')#
field ds_delete_statement_read_from_file_delete: bool | None = False (alias '_delete_statement._read_from_file_delete')#
field ds_enable_partitioned_reads: bool | None = False (alias '_enable_partitioned_reads')#
field ds_enable_partitioned_reads_column_name: str | None = None (alias '_enable_partitioned_reads._column_name')#
field ds_enable_partitioned_reads_partition_method: DSEnablePartitionedReadsPartitionMethod | None = DSEnablePartitionedReadsPartitionMethod._hive_partition (alias '_enable_partitioned_reads._partition_method')#
field ds_enable_partitioned_reads_table_name: str | None = None (alias '_enable_partitioned_reads._table_name')#
field ds_enable_partitioned_write: bool | None = False (alias '_enable_partitioned_write')#
field ds_enable_quoted_ids: bool | None = False (alias '_enable_quoted_ids')#
field ds_generate_sql: bool | None = True (alias '_generate_sql')#
field ds_hive_parameters: str | None = None (alias '_hive_parameters')#
field ds_hive_parameters_fail_on_error: bool | None = False (alias '_hive_parameters._fail_on_error')#
field ds_insert_statement: str = None (alias '_insert_statement')#
field ds_insert_statement_read_from_file_insert: bool | None = False (alias '_insert_statement._read_from_file_insert')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field ds_limit_rows_limit: int | None = None (alias '_limit_rows._limit')#
field ds_read_mode: DSReadMode | None = DSReadMode._select (alias '_read_mode')#
field ds_record_ordering: DSRecordOrdering | None = DSRecordOrdering.zero (alias '_record_ordering')#
field ds_record_ordering_key_column: list | None = [] (alias '_record_ordering._key_column')#
field ds_select_statement: str = None (alias '_select_statement')#
field ds_select_statement_other_clause: str | None = None (alias '_select_statement._other_clause')#
field ds_select_statement_read_from_file_select: bool | None = False (alias '_select_statement._read_from_file_select')#
field ds_select_statement_where_clause: str | None = None (alias '_select_statement._where_clause')#
field ds_session_array_size: int | None = 1 (alias '_session._array_size')#
field ds_session_batch_size: int | None = 2000 (alias '_session._batch_size')#
field ds_session_character_set_for_non_unicode_columns: DSSessionCharacterSetForNonUnicodeColumns | None = DSSessionCharacterSetForNonUnicodeColumns._default (alias '_session._character_set_for_non_unicode_columns')#
field ds_session_character_set_for_non_unicode_columns_character_set_name: str = None (alias '_session._character_set_for_non_unicode_columns._character_set_name')#
field ds_session_default_length_for_columns: int | None = 200 (alias '_session._default_length_for_columns')#
field ds_session_default_length_for_long_columns: int | None = 20000 (alias '_session._default_length_for_long_columns')#
field ds_session_drop_unmatched_fields: bool | None = False (alias '_session._drop_unmatched_fields')#
field ds_session_fail_on_truncation: bool | None = True (alias '_session._fail_on_truncation')#
field ds_session_fetch_size: int | None = 0 (alias '_session._fetch_size')#
field ds_session_generate_all_columns_as_unicode: bool | None = False (alias '_session._generate_all_columns_as_unicode')#
field ds_session_keep_conductor_connection_alive: bool | None = True (alias '_session._keep_conductor_connection_alive')#
field ds_session_report_schema_mismatch: bool | None = False (alias '_session._report_schema_mismatch')#
field ds_table_action: DSTableAction = DSTableAction._append (alias '_table_action')#
field ds_table_action_generate_create_statement: bool | None = True (alias '_table_action._generate_create_statement')#
field ds_table_action_generate_create_statement_create_statement: str = None (alias '_table_action._generate_create_statement._create_statement')#
field ds_table_action_generate_create_statement_fail_on_error: bool | None = True (alias '_table_action._generate_create_statement._fail_on_error')#
field ds_table_action_generate_create_statement_row_format: DSTableActionGenerateCreateStatementRowFormat | None = DSTableActionGenerateCreateStatementRowFormat._storage_format (alias '_table_action._generate_create_statement._row_format')#
field ds_table_action_generate_create_statement_row_format_field_terminator: str | None = None (alias '_table_action._generate_create_statement._row_format._field_terminator')#
field ds_table_action_generate_create_statement_row_format_line_terminator: str | None = None (alias '_table_action._generate_create_statement._row_format._line_terminator')#
field ds_table_action_generate_create_statement_row_format_serde_library: str = None (alias '_table_action._generate_create_statement._row_format._serde_library')#
field ds_table_action_generate_create_statement_storage_format: DSTableActionGenerateCreateStatementStorageFormat | None = DSTableActionGenerateCreateStatementStorageFormat._text_file (alias '_table_action._generate_create_statement._storage_format')#
field ds_table_action_generate_create_statement_table_location: str | None = None (alias '_table_action._generate_create_statement._table_location')#
field ds_table_action_generate_drop_statement: bool | None = True (alias '_table_action._generate_drop_statement')#
field ds_table_action_generate_drop_statement_drop_statement: str = None (alias '_table_action._generate_drop_statement._drop_statement')#
field ds_table_action_generate_drop_statement_fail_on_error: bool | None = False (alias '_table_action._generate_drop_statement._fail_on_error')#
field ds_table_action_generate_truncate_statement: bool | None = True (alias '_table_action._generate_truncate_statement')#
field ds_table_action_generate_truncate_statement_fail_on_error: bool | None = True (alias '_table_action._generate_truncate_statement._fail_on_error')#
field ds_table_action_generate_truncate_statement_truncate_statement: str = None (alias '_table_action._generate_truncate_statement._truncate_statement')#
field ds_table_action_table_action_first: bool | None = True (alias '_table_action._table_action_first')#
field ds_table_name: str = None (alias '_table_name')#
field ds_transaction_auto_commit_mode: DSTransactionAutoCommitMode | None = DSTransactionAutoCommitMode._disable (alias '_transaction._auto_commit_mode')#
field ds_transaction_begin_end: bool | None = False (alias '_transaction._begin_end')#
field ds_transaction_begin_end_begin_sql: str | None = None (alias '_transaction._begin_end._begin_sql')#
field ds_transaction_begin_end_end_sql: str | None = None (alias '_transaction._begin_end._end_sql')#
field ds_transaction_begin_end_run_end_sql_if_no_records_processed: bool | None = False (alias '_transaction._begin_end._run_end_sql_if_no_records_processed')#
field ds_transaction_end_of_wave: DSTransactionEndOfWave | None = DSTransactionEndOfWave._no (alias '_transaction._end_of_wave')#
field ds_transaction_isolation_level: DSTransactionIsolationLevel | None = DSTransactionIsolationLevel._default (alias '_transaction._isolation_level')#
field ds_transaction_record_count: int | None = 2000 (alias '_transaction._record_count')#
field ds_update_statement: str = None (alias '_update_statement')#
field ds_update_statement_read_from_file_update: bool | None = False (alias '_update_statement._read_from_file_update')#
field ds_use_datastage: bool | None = True (alias '_use_datastage')#
field ds_write_mode: DSWriteMode | None = DSWriteMode._insert (alias '_write_mode')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/impala.svg'#
label: ClassVar[str] = 'Apache Impala'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'impala'#

This module defines configuration or the IBM Informix stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.informix.informix#

Bases: BaseStage

Properties for the IBM Informix stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field allow_per_column_mapping: AllowPerColumnMapping | None = AllowPerColumnMapping.false (alias 'allow_column_mapping')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field catalog_name: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: InformixConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "database_locale": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "server": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field map_name: MapName | None = MapName.UTF_8 (alias 'nls_map_name')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/informix.svg'#
label: ClassVar[str] = 'IBM Informix'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'informix'#

This module defines configuration or the Generic JDBC stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.jdbc.jdbc#

Bases: BaseStage

Properties for the Generic JDBC stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field catalog_name: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: JdbcConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "allow_filter_pushdown": null,     "batch_size": null,     "case_sensitive_ids": null,     "cluster_access_token": null,     "cluster_user_name": null,     "fetch_size": null,     "gateway_url": null,     "jdbc_driver_files": null,     "jdbc_driver_class": null,     "jdbc_properties": null,     "jdbc_url": null,     "password": null,     "row_limit_prefix": null,     "row_limit_suffix": null,     "row_limit_support": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "table_types": null,     "trust_all_ssl_certificates": false,     "username": null,     "vaulted_properties": null,     "auto_commit_in_metadata_discovery": null,     "connector_uses_catalog_structure": false,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "use_column_name_in_the_statements": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_statement: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_before_after: bool | None = False (alias '_before_after')#
field ds_before_after_after_sql: str | None = '' (alias '_before_after.after_sql')#
field ds_before_after_after_sql_fail_on_error: bool | None = True (alias '_before_after.after_sql.fail_on_error')#
field ds_before_after_after_sql_node: str | None = '' (alias '_before_after.after_sql_node')#
field ds_before_after_after_sql_node_fail_on_error: bool | None = True (alias '_before_after.after_sql_node.fail_on_error')#
field ds_before_after_after_sql_node_read_from_file_after_sql_node: bool | None = False (alias '_before_after.after_sql_node.read_from_file_after_sql_node')#
field ds_before_after_after_sql_read_from_file_after_sql: bool | None = False (alias '_before_after.after_sql.read_from_file_after_sql')#
field ds_before_after_before_sql: str | None = '' (alias '_before_after.before_sql')#
field ds_before_after_before_sql_fail_on_error: bool | None = True (alias '_before_after.before_sql.fail_on_error')#
field ds_before_after_before_sql_node: str | None = '' (alias '_before_after.before_sql_node')#
field ds_before_after_before_sql_node_fail_on_error: bool | None = True (alias '_before_after.before_sql_node.fail_on_error')#
field ds_before_after_before_sql_node_read_from_file_before_sql_node: bool | None = False (alias '_before_after.before_sql_node.read_from_file_before_sql_node')#
field ds_before_after_before_sql_read_from_file_before_sql: bool | None = False (alias '_before_after.before_sql.read_from_file_before_sql')#
field ds_enable_quoted_i_ds: bool | None = False (alias '_enable_quoted_i_ds')#
field ds_generate_sql: bool | None = True (alias '_generate_sql')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field ds_limit_rows: bool | None = False (alias '_limit_rows')#
field ds_limit_rows_limit: int | None = 1000 (alias '_limit_rows.limit')#
field ds_read_mode: DSReadMode | None = DSReadMode.select (alias '_read_mode')#
field ds_record_ordering: DSRecordOrdering | None = DSRecordOrdering.zero (alias '_record_ordering')#
field ds_record_ordering_properties: list | None = [] (alias '_record_ordering_properties')#
field ds_session_array_size: int | None = 1 (alias '_session.array_size')#
field ds_session_batch_size: int | None = 2000 (alias '_session.batch_size')#
field ds_session_character_set_for_non_unicode_columns: DSSessionCharacterSetForNonUnicodeColumns | None = DSSessionCharacterSetForNonUnicodeColumns.default (alias '_session.character_set_for_non_unicode_columns')#
field ds_session_character_set_for_non_unicode_columns_character_set_name: str = '' (alias '_session.character_set_for_non_unicode_columns.character_set_name')#
field ds_session_default_length_for_columns: int | None = 200 (alias '_session.default_length_for_columns')#
field ds_session_default_length_for_long_columns: int | None = 20000 (alias '_session.default_length_for_long_columns')#
field ds_session_drop_unmatched_fields: bool | None = False (alias '_session.drop_unmatched_fields')#
field ds_session_fail_on_truncation: bool | None = True (alias '_session.fail_on_truncation')#
field ds_session_fetch_size: int | None = 0 (alias '_session.fetch_size')#
field ds_session_generate_all_columns_as_unicode: bool | None = False (alias '_session.generate_all_columns_as_unicode')#
field ds_session_keep_conductor_connection_alive: bool | None = True (alias '_session.keep_conductor_connection_alive')#
field ds_session_report_schema_mismatch: bool | None = False (alias '_session.report_schema_mismatch')#
field ds_sql_custom_statements: str | None = None (alias '_sql.custom_statements')#
field ds_sql_custom_statements_read_from_file_custom: bool | None = False (alias '_sql.custom_statements.read_from_file_custom')#
field ds_sql_delete_statement: str = None (alias '_sql.delete_statement')#
field ds_sql_delete_statement_read_from_file_delete: bool | None = False (alias '_sql.delete_statement.read_from_file_delete')#
field ds_sql_enable_partitioned_reads: bool | None = False (alias '_sql.enable_partitioned_reads')#
field ds_sql_insert_statement: str = None (alias '_sql.insert_statement')#
field ds_sql_insert_statement_read_from_file_insert: bool | None = False (alias '_sql.insert_statement.read_from_file_insert')#
field ds_sql_select_statement: str = None (alias '_sql.select_statement')#
field ds_sql_select_statement_other_clause: str | None = None (alias '_sql.select_statement.other_clause')#
field ds_sql_select_statement_read_from_file_select: bool | None = False (alias '_sql.select_statement.read_from_file_select')#
field ds_sql_select_statement_where_clause: str | None = None (alias '_sql.select_statement.where_clause')#
field ds_sql_update_statement: str = None (alias '_sql.update_statement')#
field ds_sql_update_statement_read_from_file_update: bool | None = False (alias '_sql.update_statement.read_from_file_update')#
field ds_table_action: DSTableAction = DSTableAction.append (alias '_table_action')#
field ds_table_action_generate_create_statement: bool | None = True (alias '_table_action.generate_create_statement')#
field ds_table_action_generate_create_statement_create_statement: str = '' (alias '_table_action.generate_create_statement.create_statement')#
field ds_table_action_generate_create_statement_fail_on_error: bool | None = True (alias '_table_action.generate_create_statement.fail_on_error')#
field ds_table_action_generate_drop_statement: bool | None = True (alias '_table_action.generate_drop_statement')#
field ds_table_action_generate_drop_statement_drop_statement: str = '' (alias '_table_action.generate_drop_statement.drop_statement')#
field ds_table_action_generate_drop_statement_fail_on_error: bool | None = False (alias '_table_action.generate_drop_statement.fail_on_error')#
field ds_table_action_generate_truncate_statement: bool | None = True (alias '_table_action.generate_truncate_statement')#
field ds_table_action_generate_truncate_statement_fail_on_error: bool | None = True (alias '_table_action.generate_truncate_statement.fail_on_error')#
field ds_table_action_generate_truncate_statement_truncate_statement: str = '' (alias '_table_action.generate_truncate_statement.truncate_statement')#
field ds_table_action_table_action_first: bool | None = True (alias '_table_action.table_action_first')#
field ds_table_name: str = None (alias '_table_name')#
field ds_transaction_autocommit_mode: DSTransactionAutocommitMode | None = DSTransactionAutocommitMode.disable (alias '_transaction.autocommit_mode')#
field ds_transaction_begin_end: bool | None = False (alias '_transaction.begin_end')#
field ds_transaction_begin_end_begin_sql: str | None = None (alias '_transaction.begin_end.begin_sql')#
field ds_transaction_begin_end_end_sql: str | None = None (alias '_transaction.begin_end.end_sql')#
field ds_transaction_begin_end_run_end_sql_if_no_records_processed: bool | None = False (alias '_transaction.begin_end.run_end_sql_if_no_records_processed')#
field ds_transaction_end_of_wave: DSTransactionEndOfWave | None = DSTransactionEndOfWave.no (alias '_transaction.end_of_wave')#
field ds_transaction_isolation_level: DSTransactionIsolationLevel | None = DSTransactionIsolationLevel.default (alias '_transaction.isolation_level')#
field ds_transaction_record_count: int | None = 2000 (alias '_transaction.record_count')#
field ds_use_datastage: bool | None = True (alias '_use_datastage')#
field ds_write_mode: DSWriteMode | None = DSWriteMode.insert (alias '_write_mode')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field heap_size: int | None = 256#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field is_reject_output: bool | None = False#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_not_deleted: bool | None = False#
field reject_condition_row_not_inserted: bool | None = False#
field reject_condition_row_not_updated: bool | None = False#
field reject_condition_sql_error: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field truncate_statement: str | None = None#
field trust_all_ssl_certificates: bool | None = False (alias 'trust_all_ssl_cert')#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field use_column_name_in_the_statements: bool | None = None (alias 'use_column_name')#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/JDBCConnectorPX.svg'#
label: ClassVar[str] = 'Generic JDBC'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'JDBCConnectorPX'#

This module defines configuration or the MariaDB stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.mariadb.mariadb#

Bases: BaseStage

Properties for the MariaDB stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: MariadbConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/mariadb.svg'#
label: ClassVar[str] = 'MariaDB'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'mariadb'#

This module defines configuration or the IBM Match 360 stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.match360.match360#

Bases: BaseStage

Properties for the IBM Match 360 stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field columns: str | None = None (alias 'record_sub_type_columns')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: Match360Conn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "api_key": null,     "cluster_access_token": null,     "cluster_user_name": null,     "cpd_user": null,     "crn": null,     "defer_credentials": false,     "gateway_url": null,     "iam_url": null,     "mdm_instance_id": null,     "route_host": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field data_category: DataCategory = DataCategory.records (alias 'model_type_name')#
field data_stage_folder: str | None = None (alias 'ds-folder-name')#
field data_subtype: str | None = None (alias 'record_sub_type_name')#
field data_type: str = None (alias 'model_sub_type_name')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field export_job_id: str | None = None#
field filter_rule: str | None = None#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field job_id: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field unique: bool | None = None (alias 'part_unique')#
field write_method: WriteMethod = None#
field zip_downloaded: bool | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/match360.svg'#
label: ClassVar[str] = 'IBM Match 360'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'match360'#

This module defines configuration or the MinIO stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.minio.minio#

Bases: BaseStage

Properties for the MinIO stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field bucket: str | None = None#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: MinioConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "access_key": null,     "bucket": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "secret_key": null,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "url": null,     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_bucket: bool | None = False#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field defer_credentials: bool | None = False#
field delete_bucket: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field endpoint_folder: str | None = None (alias 'table_folder_name')#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_seq (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field read_part_size: int | None = None#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field table_action: TableAction | None = TableAction.append#
field table_data_file_compression_codec: TableDataFileCompressionCodec | None = None#
field table_data_file_format: TableDataFileFormat | None = TableDataFileFormat.avro#
field table_format: TableFormat | None = None#
field table_name: str | None = None#
field table_namespace: str | None = None#
field the_cache_expiration: str | None = None (alias 'table_partition_cache_expiration')#
field the_cache_size: int | None = None (alias 'table_partition_cache_size')#
field the_data_path: str | None = None (alias 'table_data_path')#
field the_partition_columns: str | None = None (alias 'table_partition_columns')#
field the_partition_paths: str | None = None (alias 'table_partition_path')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field write_part_size: int | None = None#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/minio.svg'#
label: ClassVar[str] = 'MinIO'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'minio'#

This module defines configuration or the MongoDB stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.mongodb.mongodb#

Bases: BaseStage

Properties for the MongoDB stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auth_method: AuthMethod | None = None#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: MongodbConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_database": null,     "cluster_access_token": null,     "cluster_user_name": null,     "column_discovery_sample_size": 1000,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "schema_filter": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "special_character_behavior": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/mongodb.svg'#
label: ClassVar[str] = 'MongoDB'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'mongodb'#

This module defines configuration or the IBM Cloud Databases for MongoDB stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.mongodb_ibmcloud.mongodb_ibmcloud#

Bases: BaseStage

Properties for the IBM Cloud Databases for MongoDB stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auth_method: AuthMethod | None = None#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: MongodbIbmcloudConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_database": null,     "cluster_access_token": null,     "cluster_user_name": null,     "column_discovery_sample_size": 1000,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "schema_filter": null,     "special_character_behavior": null,     "port_is_ssl_enabled": true,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/mongodb-ibmcloud.svg'#
label: ClassVar[str] = 'IBM Cloud Databases for MongoDB'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'mongodb-ibmcloud'#

This module defines configuration or the MySQL stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.mysql.mysql#

Bases: BaseStage

Properties for the MySQL stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: MysqlConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "encoding": null,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_username": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "zero_date_time_conversion": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_message: bool | None = False (alias 'no_procmessage_column')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/mysql.svg'#
label: ClassVar[str] = 'MySQL'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'mysql'#

This module defines configuration or the Amazon RDS for MySQL stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.mysql_amazon.mysql_amazon#

Bases: BaseStage

Properties for the Amazon RDS for MySQL stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field abort_after: int | None = 0 (alias 'abort_after_percent')#
field abort_after_rows: int | None = 0 (alias 'reject_number')#
field abort_when: AbortWhen | None = AbortWhen.rows (alias 'reject_uses')#
field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field add_to_reject_row: RejectRowsPropertiesSet | None = None (alias 'rejectRowsPropertiesSet')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: MysqlAmazonConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "port_is_ssl_enabled": true,     "ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "zero_date_time_conversion": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field is_reject_output: bool | None = False#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_message: bool | None = False (alias 'no_procmessage_column')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_rows_based_on_selected_condition: RejectConditionPropertiesOption | None = None (alias 'rejectConditionProperties_option')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field start_count_after_rows: int | None = 1 (alias 'reject_threshold')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/mysql-amazon.svg'#
label: ClassVar[str] = 'Amazon RDS for MySQL'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'mysql-amazon'#

This module defines configuration or the IBM Cloud Databases for MySQL stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.mysql_compose.mysql_compose#

Bases: BaseStage

Properties for the IBM Cloud Databases for MySQL stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: MysqlComposeConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "zero_date_time_conversion": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_message: bool | None = False (alias 'no_procmessage_column')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/mysql-compose.svg'#
label: ClassVar[str] = 'IBM Cloud Databases for MySQL'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'mysql-compose'#

This module defines configuration or the IBM Netezza Performance Server stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.netezza.netezza#

Bases: BaseStage

Properties for the IBM Netezza Performance Server stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: NetezzaConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "def_schema": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/netezza.svg'#
label: ClassVar[str] = 'IBM Netezza Performance Server'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'netezza'#

This module defines configuration or the IBM Netezza Performance Server for DataStage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.netezza_optimized.netezza_optimized#

Bases: BaseStage

Properties for the IBM Netezza Performance Server for DataStage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field action_column: str = None (alias 'sql.action_column')#
field after_sql: str | None = '' (alias 'before_after_sql.after_sql')#
field after_sql_node: str | None = '' (alias 'before_after_sql.after_sql_node')#
field array_size: int | None = 2000 (alias 'session.array_size')#
field atomic_mode: bool | None = True (alias 'sql.atomic_mode')#
field atomic_mode_after_sql: bool | None = True (alias 'before_after_sql.after_sql.fail_on_error.atomic_mode')#
field atomic_mode_after_sql_node: bool | None = True (alias 'before_after_sql.after_sql_node.fail_on_error.atomic_mode')#
field atomic_mode_before_sql: bool | None = True (alias 'before_after_sql.before_sql.fail_on_error.atomic_mode')#
field atomic_mode_before_sql_node: bool | None = True (alias 'before_after_sql.before_sql_node.fail_on_error.atomic_mode')#
field before_after_sql_after_sql_fail_on_error_log_level_for_after_sql: BeforeAfterSqlAfterSqlFailOnErrorLogLevelForAfterSql | None = BeforeAfterSqlAfterSqlFailOnErrorLogLevelForAfterSql.warning (alias 'before_after_sql.after_sql.fail_on_error.log_level_for_after_sql')#
field before_after_sql_after_sql_node_fail_on_error_log_level_for_after_sql_node: BeforeAfterSqlAfterSqlNodeFailOnErrorLogLevelForAfterSqlNode | None = BeforeAfterSqlAfterSqlNodeFailOnErrorLogLevelForAfterSqlNode.warning (alias 'before_after_sql.after_sql_node.fail_on_error.log_level_for_after_sql_node')#
field before_after_sql_after_sql_node_read_from_file: bool | None = False (alias 'before_after_sql.after_sql_node.read_from_file')#
field before_after_sql_after_sql_read_from_file: bool | None = False (alias 'before_after_sql.after_sql.read_from_file')#
field before_after_sql_before_sql_fail_on_error_log_level_for_before_sql: BeforeAfterSqlBeforeSqlFailOnErrorLogLevelForBeforeSql | None = BeforeAfterSqlBeforeSqlFailOnErrorLogLevelForBeforeSql.warning (alias 'before_after_sql.before_sql.fail_on_error.log_level_for_before_sql')#
field before_after_sql_before_sql_node_fail_on_error_log_level_for_before_sql_node: BeforeAfterSqlBeforeSqlNodeFailOnErrorLogLevelForBeforeSqlNode | None = BeforeAfterSqlBeforeSqlNodeFailOnErrorLogLevelForBeforeSqlNode.warning (alias 'before_after_sql.before_sql_node.fail_on_error.log_level_for_before_sql_node')#
field before_after_sql_before_sql_node_read_from_file: bool | None = False (alias 'before_after_sql.before_sql_node.read_from_file')#
field before_after_sql_before_sql_read_from_file: bool | None = False (alias 'before_after_sql.before_sql.read_from_file')#
field before_sql: str | None = '' (alias 'before_after_sql.before_sql')#
field before_sql_node: str | None = '' (alias 'before_after_sql.before_sql_node')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field check_duplicate_rows: bool | None = False (alias 'sql.check_duplicate_rows')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field column_name: str | None = None (alias 'sql.enable_record_ordering.order_key.column_name')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: NetezzaOptimizedConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cas_lite_service_authorization_header": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname": null,     "password": null,     "port": 5480,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "original_hostname_of_the_resource": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "ssl_certificate_pem": null,     "ssl_connection": false,     "use_cas_lite_service": true,     "use_separate_connection_for_twt": false,     "twt_separate_connection_database_name": null,     "twt_separate_connection_password": null,     "user_name": null,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_statement: str | None = '' (alias 'session.temporary_work_table.create_statement')#
field create_table_statement: str = '' (alias 'table_action.generate_create_statement.create_statement')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field direct_insert: bool | None = False (alias 'sql.direct_insert')#
field directory_for_log_files: str | None = None (alias 'session.load_options.directory_for_log_files')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field distribution_key: TableActionGenerateCreateStatementDistributionKey | None = TableActionGenerateCreateStatementDistributionKey.random (alias 'table_action.generate_create_statement.distribution_key')#
field drop_table: bool | None = True (alias 'session.temporary_work_table.drop_table')#
field drop_table_statement: str = '' (alias 'table_action.generate_drop_statement.drop_statement')#
field duplicate_row_action: SqlCheckDuplicateRowsDuplicateRowAction | None = SqlCheckDuplicateRowsDuplicateRowAction.filter (alias 'sql.check_duplicate_rows.duplicate_row_action')#
field enable_before_after_sql_for_child_element: bool | None = False (alias 'before_after_sql')#
field enable_case_sensitive_identifiers: bool | None = True (alias 'enable_case_sensitive_i_ds')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_partitioned_reads: bool | None = False (alias 'sql.enable_partitioned_reads')#
field enable_record_ordering: bool | None = False (alias 'sql.enable_record_ordering')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field enable_use_of_merge_join_plan_type: SessionTemporaryWorkTableEnableMergeJoin | None = SessionTemporaryWorkTableEnableMergeJoin.database_default (alias 'session.temporary_work_table.enable_merge_join')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after_sql.after_sql.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after_sql.after_sql_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after_sql.before_sql.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after_sql.before_sql_node.fail_on_error')#
field fail_on_error_create_statement: bool | None = True (alias 'table_action.generate_create_statement.fail_on_error')#
field fail_on_error_drop_statement: bool | None = False (alias 'table_action.generate_drop_statement.fail_on_error')#
field fail_on_error_truncate_statement: bool | None = True (alias 'table_action.generate_truncate_statement.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_create_statement_at_runtime: bool | None = True (alias 'table_action.generate_create_statement')#
field generate_create_statement_distribution_key_column_names: str = '' (alias 'table_action.generate_create_statement.distribution_key.key_columns')#
field generate_drop_statement_at_runtime: bool | None = True (alias 'table_action.generate_drop_statement')#
field generate_sql_at_runtime: bool | None = True (alias 'generate_sql')#
field generate_statistics: bool | None = False (alias 'session.load_options.generate_statistics')#
field generate_statistics_mode: SessionLoadOptionsGenerateStatisticsGenerateStatisticsMode | None = SessionLoadOptionsGenerateStatisticsGenerateStatisticsMode.table (alias 'session.load_options.generate_statistics.generate_statistics_mode')#
field generate_statistics_on_columns: str | None = None (alias 'session.load_options.generate_statistics.generate_statistics_columns')#
field generate_truncate_statement_at_runtime: bool | None = True (alias 'table_action.generate_truncate_statement')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_columns: str = None (alias 'sql.key_columns')#
field limit: int | None = 1000 (alias 'limit_rows.limit')#
field limit_number_of_returned_rows: bool | None = False (alias 'limit_rows')#
field lookup_type: LookupType | None = LookupType.empty#
field mark_end_of_wave: bool | None = False (alias 'transaction.mark_end_of_wave')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field maximum_reject_count: int | None = 1 (alias 'session.load_options.max_reject_count')#
field message_type_for_create_statement_errors: TableActionGenerateCreateStatementFailOnErrorLogLevelForCreateStatement | None = TableActionGenerateCreateStatementFailOnErrorLogLevelForCreateStatement.warning (alias 'table_action.generate_create_statement.fail_on_error.log_level_for_create_statement')#
field message_type_for_drop_statement_errors: TableActionGenerateDropStatementFailOnErrorLogLevelForDropStatement | None = TableActionGenerateDropStatementFailOnErrorLogLevelForDropStatement.warning (alias 'table_action.generate_drop_statement.fail_on_error.log_level_for_drop_statement')#
field message_type_for_truncate_statement_errors: TableActionGenerateTruncateStatementFailOnErrorLogLevelForTruncateStatement | None = TableActionGenerateTruncateStatementFailOnErrorLogLevelForTruncateStatement.warning (alias 'table_action.generate_truncate_statement.fail_on_error.log_level_for_truncate_statement')#
field other_options: str | None = None (alias 'session.load_options.other_options')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_create_statement_from_file: bool | None = False (alias 'table_action.generate_create_statement.read_create_statement_from_file')#
field read_drop_statement_from_file: bool | None = False (alias 'table_action.generate_drop_statement.read_drop_statement_from_file')#
field read_truncate_statement_from_file: bool | None = False (alias 'table_action.generate_truncate_statement.read_truncate_statement_from_file')#
field reconciliation_input_mismatch_column_action: SessionSchemaReconciliationUnmatchedLinkColumnAction | None = SessionSchemaReconciliationUnmatchedLinkColumnAction.drop (alias 'session.schema_reconciliation.unmatched_link_column_action')#
field reconciliation_mismatch_message: SessionSchemaReconciliationMismatchReportingAction | None = SessionSchemaReconciliationMismatchReportingAction.warning (alias 'session.schema_reconciliation.mismatch_reporting_action')#
field reconciliation_source_mismatch_message: SessionSchemaReconciliationMismatchReportingActionSource | None = SessionSchemaReconciliationMismatchReportingActionSource.warning (alias 'session.schema_reconciliation.mismatch_reporting_action_source')#
field reconciliation_source_table_or_query_column_mismatch_action: SessionSchemaReconciliationUnmatchedTableOrQueryColumnActionSource | None = SessionSchemaReconciliationUnmatchedTableOrQueryColumnActionSource.ignore (alias 'session.schema_reconciliation.unmatched_table_or_query_column_action_source')#
field reconciliation_source_type_mismatch_action: SessionSchemaReconciliationTypeMismatchActionSource | None = SessionSchemaReconciliationTypeMismatchActionSource.drop (alias 'session.schema_reconciliation.type_mismatch_action_source')#
field reconciliation_table_or_query_column_mismatch_action: SessionSchemaReconciliationUnmatchedTableOrQueryColumnActionRequest | None = SessionSchemaReconciliationUnmatchedTableOrQueryColumnActionRequest.ignore (alias 'session.schema_reconciliation.unmatched_table_or_query_column_action_request')#
field reconciliation_type_mismatch_action: SessionSchemaReconciliationTypeMismatchAction | None = SessionSchemaReconciliationTypeMismatchAction.drop (alias 'session.schema_reconciliation.type_mismatch_action')#
field record_count: int | None = 2000 (alias 'transaction.record_count')#
field reject_table_name: str | None = None (alias 'session.load_options.validate_p_ks.reject_table_name')#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None (alias 'sql.select_statement')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field sql_read_user_defined_sql_from_file: bool | None = False (alias 'sql.user_defined_sql.read_user_defined_sql_from_file')#
field sql_select_statement_read_user_defined_sql_from_file: bool | None = False (alias 'sql.select_statement.read_user_defined_sql_from_file')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field table_action: TableAction = TableAction.append#
field table_name: str = None#
field temporary_work_table_mode: SessionTemporaryWorkTable | None = SessionTemporaryWorkTable.automatic (alias 'session.temporary_work_table')#
field temporary_work_table_name: str = None (alias 'session.temporary_work_table.table_name')#
field truncate_column_names: bool | None = False#
field truncate_length: int = 128 (alias 'truncate_column_names.truncate_length')#
field truncate_table: bool | None = False (alias 'session.temporary_work_table.truncate_table')#
field truncate_table_statement: str = '' (alias 'table_action.generate_truncate_statement.truncate_statement')#
field unique: bool | None = None (alias 'part_unique')#
field unique_key_column: str = None (alias 'sql.use_unique_key_column.unique_key_column')#
field unix_named_pipe_directory_for_load: str | None = None (alias 'session.load_options.directory_for_named_pipe')#
field unix_named_pipe_directory_for_unload: str | None = None (alias 'session.unload_options.directory_for_named_pipe')#
field unmatched_table_column_action: SessionSchemaReconciliationUnmatchedTableColumnAction | None = SessionSchemaReconciliationUnmatchedTableColumnAction.ignore_nullable (alias 'session.schema_reconciliation.unmatched_table_column_action')#
field update_columns: str | None = None (alias 'sql.update_columns')#
field use_cas_lite_service: bool | None = True (alias 'use_cas_lite')#
field use_unique_key_column: bool | None = False (alias 'sql.use_unique_key_column')#
field user_defined_sql: str = None (alias 'sql.user_defined_sql')#
field validate_primary_keys: bool | None = False (alias 'session.load_options.validate_p_ks')#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/NetezzaConnectorPX.svg'#
label: ClassVar[str] = 'IBM Netezza Performance Server for DataStage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'NetezzaConnectorPX'#

This module defines configuration or the ODBC stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.odbc.odbc#

Bases: BaseStage

Properties for the ODBC stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field array_size: int | None = 2000 (alias 'session.array_size')#
field autocommit_mode: SessionAutocommitMode | None = SessionAutocommitMode.off (alias 'session.autocommit_mode')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field character_set: str | None = None (alias 'sql.user_defined_sql.file.character_set')#
field code_page: SessionCodePage | None = SessionCodePage.default (alias 'session.code_page')#
field code_page_name: str = None (alias 'session.code_page.code_page_name')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_delimiter: LoggingLogColumnValuesDelimiter | None = LoggingLogColumnValuesDelimiter.space (alias 'logging.log_column_values.delimiter')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field columns: str = '' (alias 'session.pass_lob_locator.column')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: OdbcConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "access_token": null,     "additional_properties": null,     "authentication_method": "oauth2",     "cas_lite_service_authorization_header": null,     "client_id": null,     "client_secret": null,     "cluster_access_token": null,     "cluster_nodes": null,     "cluster_user_name": null,     "database": null,     "dataset": null,     "default_port": 2638,     "defer_credentials": false,     "data_source_name": null,     "data_source_type": "DB2",     "delimiter": ",",     "hostname_or_ip_address": null,     "keyspace": null,     "log_on_mech": "td2",     "network_address": null,     "password": null,     "port": null,     "project": null,     "refresh_token": null,     "service_account_email": null,     "service_account_key_content": null,     "input_method_for_service_account_key": "keycontent",     "service_account_private_key_file_path": null,     "service_name": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "original_hostname_of_the_resource": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "use_cas_lite_service": true,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_table_statement: str = '' (alias 'table_action.generate_create_statement.create_statement')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field default_port: int | None = 2638#
field defer_credentials: bool | None = False#
field delete_statement: str = None (alias 'sql.delete_statement')#
field delimiter: str | None = ',' (alias 'host_port_separator')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field drop_table_statement: str = '' (alias 'table_action.generate_drop_statement.drop_statement')#
field drop_unmatched_fields: bool | None = True (alias 'session.schema_reconciliation.drop_unmatched_fields')#
field ds_record_ordering: DSRecordOrdering | None = DSRecordOrdering.zero (alias '_record_ordering')#
field ds_record_ordering_properties: list | None = [] (alias '_record_ordering_properties')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_and_after_sql: bool | None = False (alias 'before_after')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_lob_references: bool | None = False (alias 'session.pass_lob_locator')#
field enable_partitioned_reads: bool | None = False (alias 'sql.enable_partitioning')#
field enable_quoted_identifiers: bool | None = True (alias 'enable_quoted_i_ds')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field end_of_data: bool | None = False (alias 'transaction.end_of_wave.end_of_data')#
field end_of_wave: TransactionEndOfWave | None = TransactionEndOfWave.none (alias 'transaction.end_of_wave')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_code_page_mismatch: bool | None = False (alias 'session.schema_reconciliation.fail_on_code_page_mismatch')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fail_on_error_for_after_sql_node_statements: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_row_error: bool | None = True (alias 'session.fail_on_row_error_px')#
field fail_on_size_mismatch: bool | None = True (alias 'session.schema_reconciliation.fail_on_size_mismatch')#
field fail_on_type_mismatch: bool | None = True (alias 'session.schema_reconciliation.fail_on_type_mismatch')#
field flow_dirty: str | None = 'false'#
field generate_create_statement_at_run_time: bool | None = True (alias 'table_action.generate_create_statement')#
field generate_drop_statement_at_run_time: bool | None = True (alias 'table_action.generate_drop_statement')#
field generate_sql_at_runtime: bool | None = False (alias 'generate_sql')#
field generate_truncate_statement_at_runtime: bool | None = True (alias 'table_action.generate_truncate_statement')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field insert_statement: str = None (alias 'sql.insert_statement')#
field is_reject_output: bool | None = False#
field isolation_level: SessionIsolationLevel | None = SessionIsolationLevel.read_uncommitted (alias 'session.isolation_level')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field limit: int | None = 1000 (alias 'limit_rows.limit')#
field limit_number_of_returned_rows: bool | None = False (alias 'limit_rows')#
field log_column_values_on_first_row_error: bool | None = False (alias 'logging.log_column_values')#
field log_key_values_only: bool | None = False (alias 'logging.log_column_values.log_keys_only')#
field log_on_mech: LogOnMech = LogOnMech.td2#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned_reads_column_name: str = None (alias 'sql.enable_partitioning.partitioning_method.key_field')#
field partitioned_reads_method: SqlEnablePartitioningPartitioningMethod | None = SqlEnablePartitioningPartitioningMethod.minimum_and_maximum_range (alias 'sql.enable_partitioning.partitioning_method')#
field partitioned_reads_table_name: str = None (alias 'sql.enable_partitioning.partitioning_method.table_name')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_after_sql_node_statements_from_file: bool | None = False (alias 'before_after.after_node.read_from_file_after_sql_node')#
field read_after_sql_statements_from_file: bool | None = False (alias 'before_after.after.read_from_file_after_sql')#
field read_before_sql_node_statement_from_file: bool | None = False (alias 'before_after.before_node.read_from_file_before_sql_node')#
field read_before_sql_statements_from_file: bool | None = False (alias 'before_after.before.read_from_file_before_sql')#
field read_select_statement_from_file: bool | None = False (alias 'sql.select_statement.read_statement_from_file')#
field record_count: int | None = 2000 (alias 'transaction.record_count')#
field reject_condition_row_not_updated: bool | None = False#
field reject_condition_sql_error: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None (alias 'sql.select_statement')#
field select_statement_column: str | None = None (alias 'sql.select_statement.columns.column')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field sql_delete_statement: str | None = None (alias 'sql.delete_statement.tables.table')#
field sql_delete_statement_parameters: str | None = None (alias 'sql.delete_statement.parameters.parameter')#
field sql_delete_statement_where_clause: str | None = None (alias 'sql.delete_statement.where_clause')#
field sql_insert_statement: str | None = None (alias 'sql.insert_statement.tables.table')#
field sql_insert_statement_parameters: str | None = None (alias 'sql.insert_statement.parameters.parameter')#
field sql_insert_statement_where_clause: str | None = None (alias 'sql.insert_statement.where_clause')#
field sql_other_clause: str | None = None (alias 'sql.other_clause')#
field sql_select_statement_other_clause: str | None = None (alias 'sql.select_statement.other_clause')#
field sql_select_statement_parameters: str | None = None (alias 'sql.select_statement.parameters.parameter')#
field sql_select_statement_table_name: str | None = None (alias 'sql.select_statement.tables.table')#
field sql_select_statement_where_clause: str | None = None (alias 'sql.select_statement.where_clause')#
field sql_update_statement: str | None = None (alias 'sql.update_statement.tables.table')#
field sql_update_statement_parameters: str | None = None (alias 'sql.update_statement.parameters.parameter')#
field sql_update_statement_where_clause: str | None = None (alias 'sql.update_statement.where_clause')#
field sql_user_defined_sql_fail_on_error: bool | None = True (alias 'sql.user_defined_sql.fail_on_error')#
field sql_where_clause: str | None = None (alias 'sql.where_clause')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field table_action: TableAction = TableAction.append#
field table_action_generate_create_statement_fail_on_error: bool | None = True (alias 'table_action.generate_create_statement.fail_on_error')#
field table_action_generate_drop_statement_fail_on_error: bool | None = False (alias 'table_action.generate_drop_statement.fail_on_error')#
field table_action_generate_truncate_statement_fail_on_error: bool | None = True (alias 'table_action.generate_truncate_statement.fail_on_error')#
field table_name: str = None#
field truncate_table_statement: str = '' (alias 'table_action.generate_truncate_statement.truncate_statement')#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str = None (alias 'sql.update_statement')#
field use_cas_lite_service: bool | None = True (alias 'use_cas_lite')#
field user_defined_sql: SqlUserDefinedSql = SqlUserDefinedSql.statements (alias 'sql.user_defined_sql')#
field user_defined_sql_file_name: str = None (alias 'sql.user_defined_sql.file')#
field user_defined_sql_statements: str = None (alias 'sql.user_defined_sql.statements')#
field write_mode: WriteMode = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/ODBCConnectorPX.svg'#
label: ClassVar[str] = 'ODBC'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'ODBCConnectorPX'#

This module defines configuration or the Oracle stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.oracle.oracle#

Bases: BaseStage

Properties for the Oracle stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field conn_impersonate_user: str | None = None#
field conn_impersonate_user_password: str | None = None#
field connection: OracleConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "alternate_servers": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "conn_impersonate_user": null,     "conn_impersonate_user_password": null,     "connection_mode": null,     "defer_credentials": false,     "failover_mode": null,     "hostname_or_ip_address": null,     "impersonate_user": null,     "include_public_synonyms": false,     "kerberos_sso": null,     "kerberos_sso_keytab": null,     "kerberos_sso_principal": null,     "load_balancing": null,     "metadata_discovery": "no_remarks",     "number_type": null,     "password": null,     "port": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_username": null,     "retry_limit": 2,     "service_name": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "database_sid": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "use_dba_catalog_views": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field include_public_synonyms: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field input_timestamp_parameters: str | None = 'false'#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field number_type: NumberType | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field use_dba_catalog_views: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/oracle.svg'#
label: ClassVar[str] = 'Oracle'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'oracle'#

This module defines configuration or the Oracle Database for DataStage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.oracle_datastage.oracle_datastage#

Bases: BaseStage

Properties for the Oracle Database for DataStage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field abort_when_create_table_statement_fails: bool | None = True (alias 'generate_create_statement.fail_on_error')#
field abort_when_drop_table_statement_fails: bool | None = True (alias 'generate_drop_statement.fail_on_error')#
field abort_when_truncate_table_statement_fails: bool | None = True (alias 'generate_truncate_statement.fail_on_error')#
field after_sql_node_statement: str | None = None (alias 'after_sql_node')#
field after_sql_statement: str | None = None (alias 'after_sql')#
field allow_concurrent_load_sessions: bool | None = True (alias 'enable_parallel_load_sessions')#
field array_size: int | None = 2000#
field before_load: bool | None = False#
field before_load_disable_constraints: bool | None = False (alias 'before_load.disable_constraints')#
field before_load_disable_triggers: bool | None = False (alias 'before_load.disable_triggers')#
field before_sql_node_statement: str | None = None (alias 'before_sql_node')#
field before_sql_statement: str | None = None (alias 'before_sql')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffer_size: int | None = 1024 (alias 'buffer_size_in_kilobytes')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field cache_size: int | None = 1000#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_delimiter: ColumnDelimiter | None = ColumnDelimiter.space (alias 'delimiter')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field column_name_for_partitioned_reads: str = None (alias 'read_strategy_column_name')#
field columns_for_lob_references: str = None (alias 'pass_lob_locator.column')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: OracleDatastageConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cas_lite_service_authorization_header": null,     "cluster_access_token": null,     "cluster_user_name": null,     "connection_string": null,     "connection_type": "tcp",     "defer_credentials": false,     "gateway_url": null,     "alternate_servers": null,     "hostname": null,     "port": null,     "servicename": null,     "password": null,     "rac_server_name": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "original_hostname_of_the_resource": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "use_cas_lite_service": true,     "use_connection_string": false,     "username": null,     "vaulted_properties": null,     "oracle_client_version": null,     "xa_database_name": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field control_file_name: str | None = None (alias 'cont_file')#
field create_table_statement: str = None (alias 'create_statement')#
field data_file_name: str | None = None (alias 'data_file')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field degree_of_parallelism: int | None = None#
field delete_statement: str = None#
field directory_for_data_and_control_files: str | None = None (alias 'directory_cont_file')#
field disable_cache_when_full: bool | None = False (alias 'disable_when_full')#
field disable_logging: bool | None = False (alias 'disable_redo_log')#
field disconnect: Disconnect | None = Disconnect.never#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field drop_table_statement: str = None (alias 'drop_statement')#
field drop_unmatched_fields: bool | None = False#
field ds_record_ordering: DSRecordOrdering | None = DSRecordOrdering.zero (alias '_record_ordering')#
field ds_record_ordering_key_column: list | None = [] (alias '_record_ordering._key_column')#
field enable_before_and_after_sql: bool | None = False (alias 'before_after')#
field enable_constraints: bool | None = False#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_lob_references: bool | None = False (alias 'pass_lob_locator')#
field enable_partitioned_reads: bool | None = False#
field enable_quoted_identifiers: bool | None = True (alias 'enable_quoted_ids')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field enable_triggers: bool | None = False#
field exceptions_table_name: str | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_for_data_truncation: bool | None = True (alias 'treat_fetch_truncate_as_error')#
field fail_if_no_rows_are_deleted: bool | None = False (alias 'fail_if_no_rows_deleted')#
field fail_if_no_rows_are_updated: bool | None = False (alias 'fail_if_no_rows_updated')#
field fail_on_error_for_after_sql_node_statement: bool | None = True (alias 'after_sql_node.fail_on_error')#
field fail_on_error_for_after_sql_statement: bool | None = True (alias 'after_sql.fail_on_error')#
field fail_on_error_for_before_sql_node_statement: bool | None = True (alias 'before_sql_node.fail_on_error')#
field fail_on_error_for_before_sql_statement: bool | None = True (alias 'before_sql.fail_on_error')#
field fail_on_error_for_index_rebuilding: bool | None = False (alias 'fail_on_rebuild_index')#
field fail_on_row_error_px: bool | None = True#
field fail_on_row_error_se: bool | None = False#
field flow_dirty: str | None = 'false'#
field generate_create_table_statement_at_runtime: bool | None = False (alias 'generate_create_statement')#
field generate_drop_table_statement_at_runtime: bool | None = False (alias 'generate_drop_statement')#
field generate_sql_at_runtime: bool | None = False (alias 'generate_sql')#
field generate_truncate_table_statement_at_runtime: bool | None = False (alias 'generate_truncate_statement')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field inactivity_period: int = 300#
field index_maintenance_option: IndexMaintenanceOption | None = IndexMaintenanceOption.do_not_skip_unusable (alias 'skip_indexes')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field insert_statement: str = None#
field interval_between_retries: int = 10 (alias 'retry_interval')#
field isolation_level: IsolationLevel | None = IsolationLevel.read_committed#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field limit: int | None = None#
field limit_number_of_returned_rows: bool | None = False (alias 'limit_rows')#
field load_options: str | None = None (alias 'load_opt')#
field log_column_values_on_first_row_error: bool | None = False (alias 'log_column_values')#
field log_key_values_only: bool | None = False (alias 'log_keys_only')#
field logging_clause: LoggingClause | None = LoggingClause.logging#
field lookup_type: LookupType | None = LookupType.empty#
field manage_application_failover: bool | None = False (alias 'application_failover_control')#
field manual_mode: bool | None = None#
field mark_end_of_wave: MarkEndOfWave | None = MarkEndOfWave.no (alias 'end_of_wave')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field number_of_retries: int | None = 10#
field other_clause: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field parallel_clause: ParallelClause | None = ParallelClause.do_not_include_parallel_clause#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name: str = None#
field partition_or_subpartition_name_for_partitioned_reads: str | None = None (alias 'read_strategy_partition_name')#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned_reads_method: PartitionedReadsMethod | None = PartitionedReadsMethod.rowid_range (alias 'partitioned_reads_strategy')#
field perform_operations_after_bulk_load: bool | None = False (alias 'after_load')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field perform_table_action_first: bool | None = True (alias 'table_action_first')#
field pl_sql_block: str = None (alias 'pl_sql_statement')#
field prefetch_buffer_size: int | None = 0 (alias 'prefetch_memory_size')#
field prefetch_row_count: int | None = 1#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field preserve_trailing_blanks: bool | None = True#
field process_exception_rows: bool | None = False#
field process_warning_messages_as_fatal_errors: bool | None = False (alias 'treat_warnings_as_errors')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_after_sql_node_statement_from_file: bool | None = False (alias 'after_sql_node.read_from_file')#
field read_after_sql_statement_from_file: bool | None = False (alias 'after_sql.read_from_file')#
field read_before_sql_node_statement_from_file: bool | None = False (alias 'before_sql_node.read_from_file')#
field read_before_sql_statement_from_file: bool | None = False (alias 'before_sql.read_from_file')#
field read_delete_statement_from_file: bool | None = False (alias 'read_from_file_delete')#
field read_insert_statement_from_file: bool | None = False (alias 'read_from_file_insert')#
field read_mode: ReadMode | None = ReadMode.select#
field read_pl_sql_block_from_file: bool | None = False (alias 'read_from_file_pl_sql_block')#
field read_select_statement_from_file: bool | None = False (alias 'read_from_file_select')#
field read_update_statement_from_file: bool | None = False (alias 'read_from_file_update')#
field rebuild_indexes: bool | None = False#
field reconnect: bool | None = False#
field record_count: int | None = 2000#
field reject_condition_row_not_deleted_delete_mode: bool | None = False#
field reject_condition_row_not_updated_insert_then_update_mode: bool | None = False#
field reject_condition_row_not_updated_update_mode: bool | None = False#
field reject_condition_sql_error_character_set_conversion: bool | None = False#
field reject_condition_sql_error_constraint_violation: bool | None = False#
field reject_condition_sql_error_data_truncation: bool | None = False#
field reject_condition_sql_error_data_type_conversion: bool | None = False#
field reject_condition_sql_error_other: bool | None = False#
field reject_condition_sql_error_partitioning: bool | None = False#
field reject_condition_sql_error_xml_processing: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field replay_before_sql_node_statement: bool | None = False (alias 'replay_before_sql_node')#
field replay_before_sql_statement: bool | None = False (alias 'replay_before_sql')#
field resume_write: bool | None = False#
field retry_count: int = 3#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field subpartition_name: str = None#
field table_action: TableAction = TableAction.append#
field table_name: str = None#
field table_name_for_partitioned_reads: str | None = None (alias 'read_strategy_table_name')#
field table_name_for_partitioned_writes: str | None = None (alias 'write_strategy_table_name')#
field table_scope: TableScope | None = TableScope.entire_table#
field time_between_retries: int | None = 10 (alias 'wait_time')#
field transfer_bfile_contents: bool | None = False (alias 'transfer_bfile_cont')#
field truncate_table_statement: str = None (alias 'truncate_statement')#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str = None#
field use_cas_lite_service: bool | None = True (alias 'use_cas_lite')#
field use_kerberos: bool | None = False#
field use_oracle_date_cache: bool | None = False (alias 'use_date_cache')#
field where_clause: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/OracleConnectorPX.svg'#
label: ClassVar[str] = 'Oracle Database for DataStage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'OracleConnectorPX'#

This module defines configuration or the IBM Planning Analytics stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.planning_analytics.planning_analytics#

Bases: BaseStage

Properties for the IBM Planning Analytics stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: PlanningAnalyticsConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "access_token": null,     "authentication_type": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "gateway_url": null,     "use_my_platform_login_credentials": false,     "namespace": null,     "password": null,     "tm1_server_api_root_url": null,     "sl_client_cert": null,     "sl_client_private_key": null,     "sl_connector_id": null,     "sl_endpoint_host": null,     "sl_endpoint_name": null,     "sl_endpoint_port": null,     "sl_host_original": null,     "sl_http_proxy": null,     "sl_location_id": null,     "sl_service_url": null,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field connection_mode: ConnectionMode | None = ConnectionMode.cube_name#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field creation_order: bool | None = False (alias 'use_creation_order')#
field cube_name: str = None#
field custom_dimension_mapping_information: str | None = None (alias 'custom_mapping_info')#
field custom_mapping: bool | None = None (alias 'use_custom_mapping')#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_seq (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field mdx_statement: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sl_client_cert: str | None = None#
field sl_client_private_key: str | None = None#
field sl_connector_id: str | None = None#
field sl_endpoint_host: str | None = None#
field sl_endpoint_name: str | None = None#
field sl_endpoint_port: int | None = None#
field sl_host_original: str | None = None#
field sl_http_proxy: bool | None = None#
field sl_location_id: str | None = None#
field sl_service_url: str | None = None#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field ssl_certificate_hostname: str | None = None (alias 'ssl_certificate_host')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field unique: bool | None = None (alias 'part_unique')#
field validate_ssl_certificate: bool | None = None (alias 'ssl_certificate_validation')#
field view: str = None (alias 'view_name')#
field view_group: str = None#
field write_to_consolidation: bool | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/tm1odata.svg'#
label: ClassVar[str] = 'IBM Planning Analytics'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'tm1odata'#

This module defines configuration or the PostgreSQL stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.postgresql.postgresql#

Bases: BaseStage

Properties for the PostgreSQL stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: PostgresqlConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "the_certificate_for_a_key": null,     "login_timeout": null,     "password": null,     "port": null,     "private_key": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_username": null,     "query_timeout": 300,     "retry_limit": 2,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_ref_output: bool | None = False#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field login_timeout: int | None = None#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_after_sql_node_statements_from_file: bool | None = False (alias 'before_after.after_node.read_from_file_after_sql_node')#
field read_after_sql_statements_from_file: bool | None = False (alias 'before_after.after.read_from_file_after_sql')#
field read_before_sql_node_statement_from_file: bool | None = False (alias 'before_after.before_node.read_from_file_before_sql_node')#
field read_before_sql_statements_from_file: bool | None = False (alias 'before_after.before.read_from_file_before_sql')#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field read_select_statement_from_file: bool | None = False (alias 'select_statement.read_from_file_select')#
field read_update_statement_from_file: bool | None = False (alias 'update_statement.read_from_file_update')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/postgresql.svg'#
label: ClassVar[str] = 'PostgreSQL'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'postgresql'#

This module defines configuration or the IBM Cloud Databases for PostgreSQL stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.postgresql_ibmcloud.postgresql_ibmcloud#

Bases: BaseStage

Properties for the IBM Cloud Databases for PostgreSQL stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: PostgresqlIbmcloudConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "login_timeout": null,     "password": null,     "port": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_username": null,     "query_timeout": 300,     "retry_limit": 2,     "ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_ref_output: bool | None = False#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field login_timeout: int | None = None#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_after_sql_node_statements_from_file: bool | None = False (alias 'before_after.after_node.read_from_file_after_sql_node')#
field read_after_sql_statements_from_file: bool | None = False (alias 'before_after.after.read_from_file_after_sql')#
field read_before_sql_node_statement_from_file: bool | None = False (alias 'before_after.before_node.read_from_file_before_sql_node')#
field read_before_sql_statements_from_file: bool | None = False (alias 'before_after.before.read_from_file_before_sql')#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field read_select_statement_from_file: bool | None = False (alias 'select_statement.read_from_file_select')#
field read_update_statement_from_file: bool | None = False (alias 'update_statement.read_from_file_update')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/postgresql-ibmcloud.svg'#
label: ClassVar[str] = 'IBM Cloud Databases for PostgreSQL'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'postgresql-ibmcloud'#

This module defines configuration or the Presto stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.presto.presto#

Bases: BaseStage

Properties for the Presto stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field catalog_name: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: PrestoConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "use_source_to_source_ssl_certificate": false,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str | None = None#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field use_source_to_source_ssl_certificate: bool | None = False (alias 'use_s2s_ssl_certificate')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/presto.svg'#
label: ClassVar[str] = 'Presto'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'presto'#

This module defines configuration or the Salesforce.com stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.salesforce.salesforce#

Bases: BaseStage

Properties for the Salesforce.com stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: SalesforceConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "password": null,     "server_name": "login.salesforce.com",     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_statement: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/salesforce.svg'#
label: ClassVar[str] = 'Salesforce.com'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'salesforce'#

This module defines configuration or the Salesforce API for DataStage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.salesforceapi.salesforceapi#

Bases: BaseStage

Properties for the Salesforce API for DataStage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field access_method: AccessMethod | None = AccessMethod.real_time_mode#
field batch_size: int | None = 200#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field business_object: str = None (alias 'salesforce_object_name')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: SalesforceapiConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "authentication_type": "username_and_password",     "cas_lite_service_authorization_header": null,     "cluster_access_token": null,     "cluster_user_name": null,     "consumer_key": null,     "consumer_secret_key": null,     "defer_credentials": false,     "password": null,     "proxy_server": false,     "proxy_server_hostname_or_ip_address": null,     "proxy_server_password": null,     "proxy_server_port": null,     "proxy_server_username": null,     "schema_name": null,     "server_certificate_key": null,     "token_expiry_time": null,     "url": null,     "use_cas_lite_service": true,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field delta_end_time: str = 'CurrentTime' (alias 'end_time')#
field delta_extract_id: str = ''#
field delta_start_time: str = 'LastExtractTime' (alias 'start_time')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field empty_recycle_bin: bool | None = False (alias 'hard_delete_property')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_load_or_extract_large_object_via_flat_file: bool | None = False (alias 'enable_flat_file')#
field enable_pk_chunking: bool | None = True (alias 'pk_chunking')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flat_file_column_name: str = None#
field flat_file_content_name: str = None#
field flat_file_folder_location: str = None#
field flat_file_overwrite: bool | None = True#
field flow_dirty: str | None = 'false'#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field hidden_job_id: str | None = '0'#
field hidden_total_record_count: str | None = '0'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field is_reject_output: bool | None = False#
field job_id: str = None#
field job_id_file_name: str = None (alias 'file_path_job_id')#
field job_id_in_file: bool | None = False (alias 'sf_job_id_in_file')#
field keep_temporary_files: bool | None = False (alias 'keep_temp_file')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field program_generated_reference_soql_query: str | None = '' (alias 'reference_soql')#
field public_datasource_field_map: str | None = None (alias 'public_data_source_field_map')#
field public_datasource_locator: str | None = None (alias 'public_data_source_locator')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_operation: ReadOperation | None = ReadOperation.query (alias 'read_mode')#
field reject_condition_write_error_row_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field runtime_column_propagation: bool | None = None#
field salesforce_concurrency_mode: SalesforceConcurrencyMode | None = SalesforceConcurrencyMode.parallel (alias 'backend_load_method')#
field serialize_modified_properties: bool | None = True#
field sf_v2_job_id_in_file: bool | None = False#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sleep: int | None = 60#
field soql_query_to_salesforce: str = '' (alias 'soql_string')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field table_name: str | None = None#
field tenacity: int | None = 1800#
field unique: bool | None = None (alias 'part_unique')#
field use_cas_lite_service: bool | None = True (alias 'use_cas_lite')#
field v2_file_path_job_id: str = None#
field v2_hard_delete_property: bool | None = False#
field v2_hidden_job_id: str | None = '0'#
field v2_hidden_total_record_count: str | None = '0'#
field v2_keep_temp_file: bool | None = False#
field write_operation: WriteOperation | None = WriteOperation.upsert (alias 'write_mode')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/SALESFORCEJCConnectorPX.svg'#
label: ClassVar[str] = 'Salesforce API for DataStage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'SALESFORCEJCConnectorPX'#

This module defines configuration or the SAP BAPI stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sapbapi.sapbapi#

Bases: BaseStage

Properties for the SAP BAPI stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field bapi_type_name: str = None#
field bapi_type_parameter: str | None = None#
field bapiexportinterface: str | None = None#
field bapiimportinterface: str | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field commitpackagesize: str | None = None#
field committracefrequency: str | None = None#
field connection: SapbapiConn = None#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field export_paramcol_properties: list | None = [] (alias 'exportParamcolProperties')#
field exportcolbindings: str | None = None#
field flow_dirty: str | None = 'false'#
field force: Force | None = Force.false#
field generate_unicode_columns: bool | None = False#
field hide: bool | None = False#
field import_paramcol_properties: list | None = [] (alias 'importParamcolProperties')#
field importcolbindings: str | None = None#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field table_paramcol_properties: list | None = [] (alias 'tableParamcolProperties')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxBapi.svg'#
label: ClassVar[str] = 'SAP BAPI'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'PxBapi'#

This module defines configuration or the SAP Bulk Extract stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sapbulkextract.sapbulkextract#

Bases: BaseStage

Properties for the SAP Bulk Extract stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field condition_clauses: str | None = None (alias 'condition_clause')#
field connection: SapbulkextractConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "client_number": null,     "cluster_access_token": null,     "cluster_user_name": null,     "connection_type": "application_server",     "defer_credentials": false,     "gateway_url": null,     "group": null,     "enable_x_509_certificate": false,     "jar_uris": null,     "language": null,     "message_server": null,     "sap_password": null,     "application_server": null,     "sap_application_system_number": null,     "sap_router": null,     "snc_name": null,     "snc_partner_name": null,     "snc_qop": "3",     "system_id": null,     "system_number": null,     "use_system_number": true,     "username": null,     "vaulted_properties": null,     "x_509_certificate": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field connection_count: int | None = 1#
field create_rfc_destination: bool | None = True#
field create_rfc_destination_name: bool | None = True#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field extract_data_in_foreground: bool | None = False (alias 'extract_in_foreground')#
field flow_dirty: str | None = 'false'#
field gateway_host: str | None = None#
field generate_sql_query: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field job: str | None = None#
field job_timeout_in_seconds: int | None = 600 (alias 'timeout')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field metadata_table: str | None = None#
field node_count: int | None = 1#
field node_number: int | None = 0#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field package_name: str | None = None#
field packet_size: int | None = 50000#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field program_id: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field rfc_destination_name: str | None = None (alias 'rfc_destination')#
field row_limit: str | None = None#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field sql_query: str | None = None#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field system_number_gateway_service: str | None = None (alias 'gateway_service')#
field table_name: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_secured_port: bool | None = False#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/sapbulkextract.svg'#
label: ClassVar[str] = 'SAP Bulk Extract'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'sapbulkextract'#

This module defines configuration or the SAP Delta Extract stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sapdeltaextract.sapdeltaextract#

Bases: BaseStage

Properties for the SAP Delta Extract stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: SapdeltaextractConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "client_number": null,     "cluster_access_token": null,     "cluster_user_name": null,     "connection_type": "application_server",     "defer_credentials": false,     "gateway_url": null,     "group": null,     "enable_x_509_certificate": false,     "jar_uris": null,     "language": null,     "message_server": null,     "sap_password": null,     "application_server": null,     "sap_application_system_number": null,     "sap_router": null,     "snc_name": null,     "snc_partner_name": null,     "snc_qop": "3",     "odp_subscriber_name": "",     "system_id": null,     "system_number": null,     "use_system_number": true,     "username": null,     "vaulted_properties": null,     "x_509_certificate": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field data_fetch_mode: DataFetchMode | None = DataFetchMode.full_mode (alias 'sap_data_fetch_mode')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field hierarchy_class: str | None = None (alias 'sap_hierarchy_class')#
field hierarchy_name: str | None = None (alias 'sap_hierarchy_name')#
field hierarchy_valid_from: str | None = None (alias 'sap_hierarchy_valid_from')#
field hierarchy_valid_to: str | None = None (alias 'sap_hierarchy_valid_to')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field job_timeout_in_seconds: int | None = 600 (alias 'timeout')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field odp_context: OdpContext | None = OdpContext.sapi#
field odp_description: str | None = '<Empty>' (alias 'sap_odp_description')#
field odp_name: str | None = None (alias 'sap_odp_name')#
field odp_type: OdpType | None = OdpType.transactional (alias 'sap_odp_type')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field package_size_in_bytes: int | None = None (alias 'package_size')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field sap_data_filter: str | None = None (alias 'sap_data_filter_list')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field unique: bool | None = None (alias 'part_unique')#
field use_custom_package_size: bool | None = None (alias 'ispackage_size_enabled')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/sapdeltaextract.svg'#
label: ClassVar[str] = 'SAP Delta Extract'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'sapdeltaextract'#

This module defines configuration or the SAP HANA stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.saphana.saphana#

Bases: BaseStage

Properties for the SAP HANA stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: SaphanaConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "gateway_url": null,     "hostname_or_ip_address": null,     "jar_uris": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/saphana.svg'#
label: ClassVar[str] = 'SAP HANA'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'saphana'#

This module defines configuration or the SAP IDoc stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sapidoc.sapidoc#

Bases: BaseStage

Properties for the SAP IDoc stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field acknowledge_idoc_reception_to_sap: bool | None = None (alias 'acknowledgeidocreceipt')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field check_to_run_test_case: str | None = None (alias 'testcaseflag')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: SapidocConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "client_number": null,     "cluster_access_token": null,     "cluster_user_name": null,     "connection_type": "application_server",     "defer_credentials": false,     "gateway_url": null,     "group": null,     "enable_x_509_certificate": false,     "jar_uris": null,     "language": null,     "message_server": null,     "sap_password": null,     "application_server": null,     "sap_application_system_number": null,     "sap_router": null,     "snc_name": null,     "snc_partner_name": null,     "snc_qop": "3",     "system_id": null,     "system_number": null,     "use_system_number": true,     "username": null,     "vaulted_properties": null,     "x_509_certificate": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field destination_partner_number: str | None = None (alias 'iload_dest_partnernumber')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field generate_idoc_number: bool | None = None#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field idoc_load_packet_size: str | None = None (alias 'idocspertrans')#
field idoc_message_type: str | None = None (alias 'mestyp')#
field idoc_name: str = None (alias 'idoc_type_name')#
field idoc_server_connection_count: str | None = None (alias 'rfcservercount')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field node_count: int | None = 1#
field node_number: int | None = 0#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partner_number: str | None = None (alias 'iload_partnernumber')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field sap_gateway_host: str | None = None (alias 'gwhost')#
field sap_gateway_host_system_number: str | None = None (alias 'gwsysnbr')#
field sap_idoc_server_timeout: str | None = '600' (alias 'listener_timeout')#
field sap_release_version: str = None (alias 'idoc_type_version')#
field sap_server_program_id: str | None = None (alias 'refserverprogid')#
field saprouterstring: str | None = None#
field segment_name: str | None = None (alias 'segnam')#
field segment_type_name: str | None = None (alias 'segtyp')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field unique: bool | None = None (alias 'part_unique')#
field use_secured_port: bool | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/sapidoc.svg'#
label: ClassVar[str] = 'SAP IDoc'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'sapidoc'#

This module defines configuration or the SAP IQ stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sapiq.sapiq#

Bases: BaseStage

Properties for the SAP IQ stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: SapiqConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/sybaseiq.svg'#
label: ClassVar[str] = 'SAP IQ'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'sybaseiq'#

This module defines configuration or the SAP OData stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sapodata.sapodata#

Bases: BaseStage

Properties for the SAP OData stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: SapodataConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "api_key": null,     "authentication_type": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "password": null,     "client_private_key": null,     "client_certificate_chain": null,     "private_key_passphrase": null,     "sap_catalog_service_version": null,     "sap_gateway_url": null,     "sap_odata_service_version": null,     "sl_client_cert": null,     "sl_client_private_key": null,     "sl_connector_id": null,     "sl_endpoint_host": null,     "sl_endpoint_name": null,     "sl_endpoint_port": null,     "sl_host_original": null,     "sl_http_proxy": null,     "sl_location_id": null,     "sl_service_url": null,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "timeout_seconds": null,     "use_mutual_authentication": false,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field csrf_protection_enabled: bool | None = True (alias 'csrf_protection')#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field entity_set: str = None (alias 'entity_set_name')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field job_start_timestamp: str | None = None (alias 'datastage_job_start_timestamp')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field query_options: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field row_limit: int | None = None#
field row_to_start_reading: int | None = None (alias 'row_start')#
field runtime_column_propagation: bool | None = None#
field service_group: str | None = None#
field service_name: str = None#
field service_version: int | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sl_client_cert: str | None = None#
field sl_client_private_key: str | None = None#
field sl_connector_id: str | None = None#
field sl_endpoint_host: str | None = None#
field sl_endpoint_name: str | None = None#
field sl_endpoint_port: int | None = None#
field sl_host_original: str | None = None#
field sl_http_proxy: bool | None = None#
field sl_location_id: str | None = None#
field sl_service_url: str | None = None#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field ssl_certificate_hostname: str | None = None (alias 'ssl_certificate_host')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field unique: bool | None = None (alias 'part_unique')#
field validate_ssl_certificate: bool | None = None (alias 'ssl_certificate_validation')#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/sapodataPX.svg'#
label: ClassVar[str] = 'SAP OData'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'sapodata'#

This module defines configuration or the Sequential file stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sequentialfile.sequentialfile#

Bases: BaseStage

Properties for the Sequential file stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field actual_field_length: int | None = None (alias 'actual_length')#
field allow_all_zeros: AllowAllZeros | None = AllowAllZeros.nofix_zero#
field allow_per_column_mapping: AllowPerColumnMapping | None = AllowPerColumnMapping.false (alias 'allow_column_mapping')#
field allow_signed_import: AllowSignedImport | None = AllowSignedImport.allow_signed_import#
field bucket: str | None = None#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_order: ByteOrder | None = ByteOrder.native_endian#
field c_format: str | None = ''#
field case_sensitive: CiCs | None = CiCs.cs (alias 'ci-cs')#
field character_set: CharacterSet | None = CharacterSet.ascii (alias 'charset')#
field check_intact: CheckIntact | None = CheckIntact.check_intact#
field cleanup_on_failure: CleanupOnFailure | None = CleanupOnFailure.false (alias 'nocleanup')#
field collate: Collate | None = Collate.OFF (alias 'collation_sequence')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field column_name_check: bool | None = None (alias 'columnNameCheck')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field compression_codec: CompressionCodec | None = CompressionCodec.snappy (alias 'codec_parquet')#
field connection: str | None = '' (alias 'file_connector')#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field data_format: DataFormat | None = DataFormat.text#
field date_format: str | None = ''#
field date_options: DateOption | None = DateOption.none (alias 'dateOption')#
field days_since: str | None = ''#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_options: DecimalOption | None = [] (alias 'decimalOption')#
field decimal_packed: DecimalPacked | None = DecimalPacked.packed#
field decimal_packed_check: DecimalPackedCheck | None = DecimalPackedCheck.check#
field decimal_packed_sign_position: DecimalPackedSignPosition | None = DecimalPackedSignPosition.trailing#
field decimal_packed_signed: DecimalPackedSigned | None = DecimalPackedSigned.signed#
field decimal_sep_value: str | None = ''#
field decimal_separator: DecimalSeparator | None = DecimalSeparator.period#
field delim_value: str | None = ''#
field delimiter: Delimiter | None = Delimiter.comma (alias 'delim')#
field delimiter_string: str | None = '' (alias 'delim_string')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field exclude_partition_string: ExcludePart | None = ExcludePart.false (alias 'excludePart')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_seq (alias 'execmode')#
field export_ebcdic_as_ascii: ExportEbcdicAsAscii | None = ExportEbcdicAsAscii.export_ebcdic_as_ascii#
field field_defaults_options: FieldOption | None = FieldOption.delimiter (alias 'fieldOption')#
field field_max_width: int | None = None (alias 'max_width')#
field field_width: int | None = None (alias 'width')#
field file: list = []#
field file_format: FileFormat | None = FileFormat.sequential#
field file_location: FileLocation | None = FileLocation.file_system#
field file_name_column: str | None = None (alias 'sourceNameField')#
field file_pattern: list | None = [] (alias 'filepattern')#
field file_plus: list = [] (alias 'file +')#
field file_update_mode: AppendOverwrite | None = AppendOverwrite.overwrite (alias 'append-overwrite')#
field fill_char: FillChar | None = FillChar.null (alias 'fill')#
field fill_char_value: str | None = ''#
field filter: str | None = None#
field final_delim_value: str | None = ''#
field final_delimiter: FinalDelimiter | None = FinalDelimiter.end (alias 'final_delim')#
field final_delimiter_string: str | None = '' (alias 'final_delim_string')#
field first_line_is_column_names: FirstLineColumnNames | None = FirstLineColumnNames.false (alias 'firstLineColumnNames')#
field flow_dirty: str | None = 'false'#
field force_sequential: ForceSequential | None = ForceSequential.false (alias 'sequential')#
field general_options: GeneralOption | None = [] (alias 'generalOption')#
field hide: bool | None = False#
field import_ascii_as_ebcdic: ImportAsciiAsEbcdic | None = ImportAsciiAsEbcdic.import_ascii_as_ebcdic#
field in_format: str | None = ''#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field intact: str | None = ''#
field is_julian: IsJulian | None = IsJulian.julian#
field is_midnight_seconds: IsMidnightSeconds | None = IsMidnightSeconds.midnight_seconds#
field keep_file_partitions: KeepPartitions | None = KeepPartitions.false (alias 'keepPartitions')#
field key: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field map_name: MapName | None = MapName.UTF_8 (alias 'nls_map_name')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_file_size: str = None (alias 'maxFileSize')#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field missing_file_mode: MissingFile | None = MissingFile.custom (alias 'missingFile')#
field null_field_length: int | None = None (alias 'null_length')#
field null_field_sep_value: str | None = ''#
field null_field_value: str | None = 'NULL' (alias 'null_field')#
field null_field_value_separator: NullFieldValueSeparator | None = NullFieldValueSeparator.comma (alias 'null_field_sep')#
field nulls_position: NullsPosition | None = NullsPosition.first (alias 'nulls')#
field number_of_readers_per_node: int | None = 1 (alias 'readers')#
field numeric_options: NumericOption | None = [] (alias 'numericOption')#
field out_format: str | None = ''#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field pad_char: PadChar | None = PadChar.false_ (alias 'padchar')#
field padchar_value: str | None = ''#
field page_size: int | None = 1048576#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field precision: int | None = None#
field prefix_bytes: PrefixBytes | None = PrefixBytes.one (alias 'prefix')#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_clear (alias 'preserve')#
field print_field: PrintField | None = PrintField.print_field#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote: Quote | None = Quote.double#
field quote_value: str | None = ''#
field read_batch_size: int | None = 10000#
field read_entire_file_as_one_column: bool | None = False (alias 'read_file_as_one_column')#
field read_first_rows: int | None = None (alias 'first')#
field read_from_multiple_nodes: ReadFromMultipleNodes | None = ReadFromMultipleNodes.no (alias 'multinode')#
field read_method: ReadMethod | None = ReadMethod.file (alias 'selection')#
field record_delim_value: str | None = ''#
field record_delimiter: RecordDelimiter | None = RecordDelimiter.newline (alias 'record_delim')#
field record_delimiter_string: str | None = '' (alias 'record_delim_string')#
field record_len_value: int | None = 0#
field record_length: RecordLength | None = RecordLength.fixed#
field record_level_options: RecLevelOption | None = [] (alias 'recLevelOption')#
field record_prefix: RecordPrefix | None = RecordPrefix.one#
field record_type: RecordType | None = RecordType.type_implicit (alias 'record_format')#
field reject_mode: RejectMode | None = RejectMode.cont (alias 'rejects')#
field reject_reason_column: str | None = None (alias 'rejectReasonField')#
field report_progress: ReportProgress | None = ReportProgress.yes (alias 'reportProgress')#
field rich_file: list | None = []#
field rich_file_plus: list | None = [] (alias 'rich_file +')#
field richfile: list = []#
field richfile_plus: list = [] (alias 'richfile +')#
field root_file_string: str | None = None (alias 'filepath')#
field rounding: Rounding | None = Rounding.trunc_zero (alias 'round')#
field row_group_size: int | None = 1000000 (alias 'block_size')#
field row_number_column: str | None = None (alias 'recordNumberField')#
field runtime_column_propagation: bool | None = None#
field scale: int | None = None#
field schema_file: str | None = None (alias 'schemafile')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_as_ebcdic: SortAsEbcdic | None = SortAsEbcdic.false (alias 'ebcdic')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sort_order: AscDesc | None = AscDesc.asc (alias 'asc-desc')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field string_options: StringOption | None = [] (alias 'stringOption')#
field strip_bom: StripBom | None = StripBom.false (alias 'stripbom')#
field time_format: str | None = ''#
field time_options: TimeOption | None = TimeOption.none (alias 'timeOption')#
field timestamp_format: str | None = ''#
field timestamp_options: TimestampOption | None = TimestampOption.none (alias 'timestampOption')#
field unique: bool | None = None (alias 'part_unique')#
field use_value_in_filename: UseValueInFilename | None = UseValueInFilename.false (alias 'include')#
field vector_prefix: VectorPrefix | None = VectorPrefix.one#
field whether_check_intact: bool | None = False (alias 'check_intact_flag')#
field whether_specify_null_field_value_separator: bool | None = False (alias 'null_field_sep_flag')#
field write_method: WriteMethod | None = WriteMethod.specific (alias 'writemethod')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxSequentialFile.svg'#
label: ClassVar[str] = 'Sequential file'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'PxSequentialFile'#

This module defines configuration or the SingleStoreDB stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.singlestore.singlestore#

Bases: BaseStage

Properties for the SingleStoreDB stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: SinglestoreConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/singlestore.svg'#
label: ClassVar[str] = 'SingleStoreDB'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'singlestore'#

This module defines configuration or the Snowflake stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.snowflake.snowflake#

Bases: BaseStage

Properties for the Snowflake stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field before_sql: str | None = None (alias 'static_before_sql')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field conn_query_timeout: int | None = None#
field connection: SnowflakeConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "account_name": null,     "authentication_method": "username_password",     "url_endpoint": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "error_on_nondeterministic_merge": null,     "hostname_or_ip_address": "@account_name@.snowflakecomputing.com",     "keep_client_session_alive": null,     "heartbeat_frequency": null,     "key_passphrase": null,     "lineage_extraction_type": null,     "password": null,     "port": 443,     "private_key": null,     "query_timeout": null,     "role": null,     "username": null,     "vaulted_properties": null,     "warehouse": null,     "properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_statement: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_auto_commit_mode: DSAutoCommitMode | None = DSAutoCommitMode.enable (alias '_auto_commit_mode')#
field ds_before_after: bool | None = False (alias '_before_after')#
field ds_before_after_after_sql: str | None = None (alias '_before_after._after_sql')#
field ds_before_after_after_sql_fail_on_error: bool | None = True (alias '_before_after._after_sql._fail_on_error')#
field ds_before_after_after_sql_node: str | None = None (alias '_before_after._after_sql_node')#
field ds_before_after_after_sql_node_fail_on_error: bool | None = True (alias '_before_after._after_sql_node._fail_on_error')#
field ds_before_after_after_sql_node_read_from_file_after_sql_node: bool | None = False (alias '_before_after._after_sql_node._read_from_file_after_sql_node')#
field ds_before_after_after_sql_read_from_file_after_sql: bool | None = False (alias '_before_after._after_sql._read_from_file_after_sql')#
field ds_before_after_before_sql: str | None = None (alias '_before_after._before_sql')#
field ds_before_after_before_sql_fail_on_error: bool | None = True (alias '_before_after._before_sql._fail_on_error')#
field ds_before_after_before_sql_node: str | None = None (alias '_before_after._before_sql_node')#
field ds_before_after_before_sql_node_fail_on_error: bool | None = True (alias '_before_after._before_sql_node._fail_on_error')#
field ds_before_after_before_sql_node_read_from_file_before_sql_node: bool | None = False (alias '_before_after._before_sql_node._read_from_file_before_sql_node')#
field ds_before_after_before_sql_read_from_file_before_sql: bool | None = False (alias '_before_after._before_sql._read_from_file_before_sql')#
field ds_begin_end_sql: bool | None = False (alias '_begin_end_sql')#
field ds_begin_sql: str | None = None (alias '_begin_sql')#
field ds_custom_statements: str | None = None (alias '_custom_statements')#
field ds_custom_statements_read_from_file_custom: bool | None = False (alias '_custom_statements._read_from_file_custom')#
field ds_delete_statement: str = None (alias '_delete_statement')#
field ds_delete_statement_read_from_file_delete: bool | None = False (alias '_delete_statement._read_from_file_delete')#
field ds_enable_partitioned_reads: bool | None = False (alias '_enable_partitioned_reads')#
field ds_enable_quoted_ids: bool | None = False (alias '_enable_quoted_ids')#
field ds_end_of_wave: DSEndOfWave | None = DSEndOfWave._no (alias '_end_of_wave')#
field ds_end_sql: str | None = None (alias '_end_sql')#
field ds_generate_sql: bool | None = True (alias '_generate_sql')#
field ds_insert_statement: str = None (alias '_insert_statement')#
field ds_insert_statement_read_from_file_insert: bool | None = False (alias '_insert_statement._read_from_file_insert')#
field ds_isolation_level: DSIsolationLevel | None = DSIsolationLevel.default (alias '_isolation_level')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field ds_limit_rows_limit: int | None = None (alias '_limit_rows._limit')#
field ds_load_from_file_azure_encryption: DSLoadFromFileAzureEncryption | None = DSLoadFromFileAzureEncryption.none (alias '_load_from_file._azure._encryption')#
field ds_load_from_file_azure_file_format_name: str = None (alias '_load_from_file._azure._file_format_name')#
field ds_load_from_file_azure_file_name: str = None (alias '_load_from_file._azure._file_name')#
field ds_load_from_file_azure_master_key: str | None = None (alias '_load_from_file._azure._master_key')#
field ds_load_from_file_azure_sastoken: str = None (alias '_load_from_file._azure._sastoken')#
field ds_load_from_file_azure_storage_area_name: str = None (alias '_load_from_file._azure._storage_area_name')#
field ds_load_from_file_azure_use_existing_file_format: bool = True (alias '_load_from_file._azure._use_existing_file_format')#
field ds_load_from_file_copy_options_on_error: DSLoadFromFileCopyOptionsOnError | None = DSLoadFromFileCopyOptionsOnError.abort_statement (alias '_load_from_file._copy_options._on_error')#
field ds_load_from_file_copy_options_other_copy_options: str | None = None (alias '_load_from_file._copy_options._other_copy_options')#
field ds_load_from_file_create_staging_area: bool | None = True (alias '_load_from_file._create_staging_area')#
field ds_load_from_file_credentials_file_name: str | None = None (alias '_load_from_file._credentials_file_name')#
field ds_load_from_file_delete_staging_area: bool | None = False (alias '_load_from_file._delete_staging_area')#
field ds_load_from_file_directory_path: str = None (alias '_load_from_file._directory_path')#
field ds_load_from_file_file_format: DSLoadFromFileFileFormat | None = DSLoadFromFileFileFormat.csv (alias '_load_from_file._file_format')#
field ds_load_from_file_file_format_binary_as_text: bool | None = False (alias '_load_from_file._file_format._binary_as_text')#
field ds_load_from_file_file_format_compression: DSLoadFromFileFileFormatCompression | None = DSLoadFromFileFileFormatCompression.none (alias '_load_from_file._file_format._compression')#
field ds_load_from_file_file_format_date_format: str | None = None (alias '_load_from_file._file_format._date_format')#
field ds_load_from_file_file_format_encoding: str | None = None (alias '_load_from_file._file_format._encoding')#
field ds_load_from_file_file_format_field_delimiter: str | None = None (alias '_load_from_file._file_format._field_delimiter')#
field ds_load_from_file_file_format_other_format_options: str | None = None (alias '_load_from_file._file_format._other_format_options')#
field ds_load_from_file_file_format_record_delimiter: str | None = None (alias '_load_from_file._file_format._record_delimiter')#
field ds_load_from_file_file_format_skip_byte_order_mark: bool | None = False (alias '_load_from_file._file_format._skip_byte_order_mark')#
field ds_load_from_file_file_format_snappy_compression: bool | None = False (alias '_load_from_file._file_format._snappy_compression')#
field ds_load_from_file_file_format_time_format: str | None = None (alias '_load_from_file._file_format._time_format')#
field ds_load_from_file_file_format_timestamp_format: str | None = None (alias '_load_from_file._file_format._timestamp_format')#
field ds_load_from_file_gcs_file_format: str = None (alias '_load_from_file._gcs._file_format')#
field ds_load_from_file_gcs_file_name: str = None (alias '_load_from_file._gcs._file_name')#
field ds_load_from_file_gcs_storage_integration: str = None (alias '_load_from_file._gcs._storage_integration')#
field ds_load_from_file_gcs_use_existing_file_format: bool = True (alias '_load_from_file._gcs._use_existing_file_format')#
field ds_load_from_file_max_file_size: int | None = 64 (alias '_load_from_file._max_file_size')#
field ds_load_from_file_purge_copied_files: bool | None = True (alias '_load_from_file._purge_copied_files')#
field ds_load_from_file_s3_access_key: str = None (alias '_load_from_file._s3._access_key')#
field ds_load_from_file_s3_bucket_name: str = None (alias '_load_from_file._s3._bucket_name')#
field ds_load_from_file_s3_encryption: DSLoadFromFileS3Encryption | None = DSLoadFromFileS3Encryption.none (alias '_load_from_file._s3._encryption')#
field ds_load_from_file_s3_file_name: str = None (alias '_load_from_file._s3._file_name')#
field ds_load_from_file_s3_secret_key: str = None (alias '_load_from_file._s3._secret_key')#
field ds_load_from_file_staging_area_format_encoding: str | None = 'UTF-8' (alias '_load_from_file._staging_area_format._encoding')#
field ds_load_from_file_staging_area_format_escape_character: str | None = '' (alias '_load_from_file._staging_area_format._escape_character')#
field ds_load_from_file_staging_area_format_field_delimiter: str | None = ',' (alias '_load_from_file._staging_area_format._field_delimiter')#
field ds_load_from_file_staging_area_format_null_value: str | None = '' (alias '_load_from_file._staging_area_format._null_value')#
field ds_load_from_file_staging_area_format_other_file_format_options: str | None = '' (alias '_load_from_file._staging_area_format._other_file_format_options')#
field ds_load_from_file_staging_area_format_quotes: DSLoadFromFileStagingAreaFormatQuotes | None = DSLoadFromFileStagingAreaFormatQuotes.none (alias '_load_from_file._staging_area_format._quotes')#
field ds_load_from_file_staging_area_format_record_delimiter: str | None = '<NL>' (alias '_load_from_file._staging_area_format._record_delimiter')#
field ds_load_from_file_staging_area_name: str = None (alias '_load_from_file._staging_area_name')#
field ds_load_from_file_staging_area_type: DSLoadFromFileStagingAreaType | None = DSLoadFromFileStagingAreaType.internal_location (alias '_load_from_file._staging_area_type')#
field ds_load_from_file_use_credentials_file: bool | None = False (alias '_load_from_file._use_credentials_file')#
field ds_read_mode: int | None = 0 (alias '_read_mode')#
field ds_record_count: int | None = 2000 (alias '_record_count')#
field ds_record_ordering: DSRecordOrdering | None = DSRecordOrdering.zero (alias '_record_ordering')#
field ds_record_ordering_properties: list | None = [] (alias '_record_ordering_properties')#
field ds_run_end_sql_if_no_records_processed: bool | None = False (alias '_run_end_sql_if_no_records_processed')#
field ds_select_statement: str = None (alias '_select_statement')#
field ds_select_statement_other_clause: str | None = None (alias '_select_statement._other_clause')#
field ds_select_statement_read_from_file_select: bool | None = False (alias '_select_statement._read_from_file_select')#
field ds_select_statement_where_clause: str | None = None (alias '_select_statement._where_clause')#
field ds_session_array_size: int | None = 1 (alias '_session._array_size')#
field ds_session_batch_size: int | None = 2000 (alias '_session._batch_size')#
field ds_session_character_set_for_non_unicode_columns: DSSessionCharacterSetForNonUnicodeColumns | None = DSSessionCharacterSetForNonUnicodeColumns._default (alias '_session._character_set_for_non_unicode_columns')#
field ds_session_character_set_for_non_unicode_columns_character_set_name: str = None (alias '_session._character_set_for_non_unicode_columns._character_set_name')#
field ds_session_default_length_for_columns: int | None = 200 (alias '_session._default_length_for_columns')#
field ds_session_default_length_for_long_columns: int | None = 20000 (alias '_session._default_length_for_long_columns')#
field ds_session_drop_unmatched_fields: bool | None = False (alias '_session._drop_unmatched_fields')#
field ds_session_fetch_size: int | None = 0 (alias '_session._fetch_size')#
field ds_session_generate_all_columns_as_unicode: bool | None = False (alias '_session._generate_all_columns_as_unicode')#
field ds_session_keep_conductor_connection_alive: bool | None = True (alias '_session._keep_conductor_connection_alive')#
field ds_session_report_schema_mismatch: bool | None = False (alias '_session._report_schema_mismatch')#
field ds_table_action: DSTableAction = DSTableAction._append (alias '_table_action')#
field ds_table_action_generate_create_statement: bool | None = True (alias '_table_action._generate_create_statement')#
field ds_table_action_generate_create_statement_create_statement: str = None (alias '_table_action._generate_create_statement._create_statement')#
field ds_table_action_generate_create_statement_fail_on_error: bool | None = True (alias '_table_action._generate_create_statement._fail_on_error')#
field ds_table_action_generate_drop_statement: bool | None = True (alias '_table_action._generate_drop_statement')#
field ds_table_action_generate_drop_statement_drop_statement: str = None (alias '_table_action._generate_drop_statement._drop_statement')#
field ds_table_action_generate_drop_statement_fail_on_error: bool | None = False (alias '_table_action._generate_drop_statement._fail_on_error')#
field ds_table_action_generate_truncate_statement: bool | None = True (alias '_table_action._generate_truncate_statement')#
field ds_table_action_generate_truncate_statement_fail_on_error: bool | None = True (alias '_table_action._generate_truncate_statement._fail_on_error')#
field ds_table_action_generate_truncate_statement_truncate_statement: str = None (alias '_table_action._generate_truncate_statement._truncate_statement')#
field ds_table_action_table_action_first: bool | None = True (alias '_table_action._table_action_first')#
field ds_table_name: str = None (alias '_table_name')#
field ds_update_statement: str = None (alias '_update_statement')#
field ds_update_statement_read_from_file_update: bool | None = False (alias '_update_statement._read_from_file_update')#
field ds_use_datastage: bool | None = True (alias '_use_datastage')#
field ds_use_merge_statement: bool | None = True (alias '_use_merge_statement')#
field ds_write_mode: DSWriteMode | None = DSWriteMode.insert (alias '_write_mode')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_partition_sql: bool | None = None (alias 'static_after_partition_sql_fail_on_error')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_partition_sql: bool | None = None (alias 'static_before_partition_sql_fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field hostname_or_ip_address: str | None = '@account_name@.snowflakecomputing.com' (alias 'host')#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field is_reject_output: bool | None = False#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_after_sql: str | None = None (alias 'static_after_partition_sql')#
field partition_before_sql: str | None = None (alias 'static_before_partition_sql')#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field port: int | None = 443#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field procedure_name: str | None = None#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_condition_row_not_deleted: bool | None = False#
field reject_condition_row_not_inserted: bool | None = False#
field reject_condition_row_not_updated: bool | None = False#
field reject_condition_sql_error: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/SnowflakeConnectorPX.svg'#
label: ClassVar[str] = 'Snowflake'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'SnowflakeConnectorPX'#

This module defines configuration or the Microsoft SQL Server stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sqlserver.sqlserver#

Bases: BaseStage

Properties for the Microsoft SQL Server stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field catalog_name: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field conn_query_timeout: int | None = None#
field connection: SqlserverConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "domain": null,     "hostname_or_ip_address": null,     "instance_name": null,     "ntlm_version": null,     "password": null,     "port": null,     "proxy": false,     "proxy_host": null,     "proxy_password": null,     "proxy_port": null,     "proxy_username": null,     "query_timeout": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "use_active_directory": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field inout_parameters: str | None = 'false'#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_message: bool | None = False (alias 'no_procmessage_column')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field version_number: int | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/sqlserver.svg'#
label: ClassVar[str] = 'Microsoft SQL Server'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'sqlserver'#

This module defines configuration or the Storage volume stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.storage_volume.storage_volume#

Bases: BaseStage

Properties for the Storage volume stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field cell_range: str | None = None (alias 'range')#
field codec_avro: CodecAvro | None = None#
field codec_csv: CodecCsv | None = None#
field codec_delimited: CodecDelimited | None = None#
field codec_orc: CodecOrc | None = None#
field codec_parquet: CodecParquet | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: StorageVolumeConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_host": null,     "ds_port": null,     "access_token": null,     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "gateway_url": null,     "inherit_access_token": false,     "instance_id": null,     "password": null,     "pvc": null,     "read_timeout": 300,     "ssl_certificate": null,     "trust_all_ssl_cert": false,     "username": null,     "vaulted_properties": null,     "volume": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field date_format: str | None = None#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_format: str | None = None#
field decimal_grouping_separator: str | None = None (alias 'decimal_format_grouping_separator')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field decimal_separator: str | None = None (alias 'decimal_format_decimal_separator')#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field display_value_labels: bool | None = None#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field encoding: str | None = 'utf-8'#
field encryption_key: str | None = None#
field escape_character: EscapeCharacter | None = EscapeCharacter.none#
field escape_character_value: str = None#
field exclude_missing_values: bool | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field field_delimiter: FieldDelimiter | None = FieldDelimiter.comma#
field field_delimiter_value: str = None#
field fields_xml_path: str | None = None (alias 'xml_path_fields')#
field file_format: FileFormat | None = FileFormat.csv#
field file_name: str = None#
field first_line: int | None = 0#
field first_line_is_header: bool | None = False (alias 'first_line_header')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field include_types: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_null_as_empty_string: bool | None = False#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field infer_timestamp_as_date: bool | None = True#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field json_infer_record_count: int | None = None#
field json_path: str | None = None#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field labels_as_names: bool | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field names_as_labels: bool | None = None#
field null_value: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_as_json: bool | None = None (alias 'output_avro_as_json')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_name_prefix: str | None = 'part'#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field partitioned: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote_character: QuoteCharacter | None = QuoteCharacter.none#
field quote_numeric_values: bool | None = True (alias 'quote_numerics')#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_delimiter: RowDelimiter | None = RowDelimiter.new_line#
field row_delimiter_value: str = None#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_of_xml: str | None = None (alias 'xml_schema')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field store_shared_strings_in_the_temporary_file: bool | None = None (alias 'use_sst_temp_file')#
field time_format: str | None = None#
field timestamp_format: str | None = None#
field timezone_format: str | None = None (alias 'time_zone_format')#
field trust_all_ssl_cert: bool | None = False#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
field use_4_digit_years_in_date_formats: bool | None = None (alias 'use_4_digit_year')#
field use_field_formats: bool | None = None#
field use_variable_formats: bool | None = None#
field worksheet_name: str | None = None (alias 'sheet_name')#
field write_mode: WriteMode | None = WriteMode.write#
field xml_path: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/volumes.svg'#
label: ClassVar[str] = 'Storage volume'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'volumes'#

This module defines configuration or the SAP ASE stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sybase.sybase#

Bases: BaseStage

Properties for the SAP ASE stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field catalog_name: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: SybaseConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/sybase.svg'#
label: ClassVar[str] = 'SAP ASE'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'sybase'#

This module defines configuration or the Tableau stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.tableau.tableau#

Bases: BaseStage

Properties for the Tableau stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: TableauConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "ds_port": null,     "access_token_name": null,     "access_token_secret": null,     "authentication_method": "username_and_password",     "cluster_access_token": null,     "cluster_user_name": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "site": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": true,     "ssl_certificate": null,     "trust_all_ssl_certificates": false,     "username": null,     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field file_name: str = None#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field hide: bool | None = False#
field infer_as_varchar: bool | None = None#
field infer_record_count: int | None = 1000#
field infer_schema: bool | None = None#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field invalid_data_handling: InvalidDataHandling | None = InvalidDataHandling.fail#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_mode: ReadMode | None = ReadMode.read_single#
field row_limit: int | None = None#
field row_start: int | None = None#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field type_mapping: str | None = None#
field unique: bool | None = None (alias 'part_unique')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/tableau.svg'#
label: ClassVar[str] = 'Tableau'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'tableau'#

This module defines configuration or the Teradata stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.teradata.teradata#

Bases: BaseStage

Properties for the Teradata stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_procedure_return_value_to_schema: bool | None = False (alias 'add_proccode_column')#
field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field call_procedure_statement: str | None = None (alias 'call_statement')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: TeradataConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "authentication_method": "td2",     "client_character_set": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "password": null,     "port": 1025,     "query_band_expression": null,     "session_username": null,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "enable_trusted_session": false,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_warning: str | None = None#
field execute_procedure_for_each_row: bool | None = True (alias 'call_each_row')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fatal_error: str | None = None (alias 'error_fatal')#
field flow_dirty: str | None = 'false'#
field forward_row_data: bool | None = False#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field proc_param_properties: list | None = [] (alias 'procParamProperties')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field query_timeout: int | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field stored_procedure_name: str | None = None (alias 'procedure_name')#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field transform: str | None = 'false'#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field user_defined_function: bool | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/teradata.svg'#
label: ClassVar[str] = 'Teradata'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'teradata'#

This module defines configuration or the Teradata database for DataStage stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.teradata_datastage.teradata_datastage#

Bases: BaseStage

Properties for the Teradata database for DataStage stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field access_method: AccessMethod | None = AccessMethod.immediate#
field after_sql_file: str | None = None (alias 'before_after.after_sql_file')#
field allow_duplicate_rows: TableActionGenerateCreateStatementCreateTableOptionsAllowDuplicateRows | None = TableActionGenerateCreateStatementCreateTableOptionsAllowDuplicateRows.default (alias 'table_action.generate_create_statement.create_table_options.allow_duplicate_rows')#
field array_size: int | None = 2000 (alias 'session.array_size')#
field before_sql_file: str | None = None (alias 'before_after.before_sql_file')#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffer_usage: ImmediateAccessBufferUsage | None = ImmediateAccessBufferUsage.share (alias 'immediate_access.buffer_usage')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field character_set: str | None = None (alias 'sql.user_defined.file.character_set')#
field character_set_after_sql_file: str | None = None (alias 'before_after.after_sql_file.character_set')#
field character_set_before_sql_file: str | None = None (alias 'before_after.before_sql_file.character_set')#
field checkpoint_timeout: int | None = 0 (alias 'parallel_synchronization.checkpoint_timeout')#
field cleanup_mode: BulkAccessCleanupMode | None = BulkAccessCleanupMode.drop (alias 'bulk_access.cleanup_mode')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_delimiter: LoggingLogColumnValuesDelimiter | None = LoggingLogColumnValuesDelimiter.space (alias 'logging.log_column_values.delimiter')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field columns: str = '' (alias 'session.pass_lob_locator.column')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: TeradataDatastageConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "account": null,     "automatically_map_character_set_encoding": true,     "cas_lite_service_authorization_header": null,     "client_character_set": "UTF8",     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "default_port": 1025,     "delimiter": ":",     "logon_mechanism": "default",     "maximum_bytes_per_character": null,     "nls_map_name": null,     "password": null,     "query_band_expression": null,     "read_query_band_expression_from_the_file": false,     "iana_character_set_name": null,     "server": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "original_hostname_of_the_resource": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "ssl_mode": null,     "ssl_certificate": null,     "transaction_mode": "ansi",     "unicode_pass_through": false,     "use_cas_lite_service": true,     "username": null,     "vaulted_properties": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_table_statement: str = '' (alias 'table_action.generate_create_statement.create_statement')#
field credentials_input_method_ssl: CredentialsInputMethodSsl | None = CredentialsInputMethodSsl.enter_credentials_manually#
field data_block_size: int | None = 0 (alias 'table_action.generate_create_statement.create_table_options.data_block_size')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field default_port: int | None = 1025#
field delete_multiple_rows: bool | None = False (alias 'bulk_access.update_load.delete_multiple_rows')#
field delete_statement: str = None (alias 'sql.delete_statement')#
field delimiter: str | None = ':' (alias 'host_port_separator')#
field describe_strings_in_bytes: bool | None = False#
field disconnect: Disconnect | None = Disconnect.never#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field drop_table_statement: str = '' (alias 'table_action.generate_drop_statement.drop_statement')#
field duplicate_insert_rows: BulkAccessErrorControlDuplicateInsertRows | None = BulkAccessErrorControlDuplicateInsertRows.default (alias 'bulk_access.error_control.duplicate_insert_rows')#
field duplicate_update_rows: BulkAccessErrorControlDuplicateUpdateRows | None = BulkAccessErrorControlDuplicateUpdateRows.default (alias 'bulk_access.error_control.duplicate_update_rows')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_and_after_sql: bool | None = False (alias 'before_after')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_lob_references: bool | None = False (alias 'session.pass_lob_locator')#
field enable_quoted_identifiers: bool | None = True (alias 'enable_quoted_i_ds')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field end_of_data: bool | None = False (alias 'transaction.end_of_wave.end_of_data')#
field end_of_wave: TransactionEndOfWave | None = TransactionEndOfWave.none (alias 'transaction.end_of_wave')#
field end_row: int | None = 0 (alias 'limit_settings.end_row')#
field end_timeout: int | None = 0 (alias 'parallel_synchronization.end_timeout')#
field error_limit: int | None = 0 (alias 'bulk_access.error_limit')#
field error_table_1: str | None = None (alias 'bulk_access.error_table1')#
field error_table_2: str | None = None (alias 'bulk_access.error_table2')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error: bool | None = True (alias 'sql.user_defined.request_type.fail_on_error')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_file: bool | None = True (alias 'before_after.after_sql_file.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_file: bool | None = True (alias 'before_after.before_sql_file.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field fail_on_error_create_statement: bool | None = True (alias 'table_action.generate_create_statement.fail_on_error')#
field fail_on_error_drop_statement: bool | None = False (alias 'table_action.generate_drop_statement.fail_on_error')#
field fail_on_error_truncate_statement: bool | None = True (alias 'table_action.generate_truncate_statement.fail_on_error')#
field fail_on_mload_errors: bool | None = True (alias 'bulk_access.fail_on_mloa_derrs')#
field fail_on_size_mismatch: bool | None = True (alias 'session.schema_reconciliation.fail_on_size_mismatch')#
field fail_on_type_mismatch: bool | None = True (alias 'session.schema_reconciliation.fail_on_type_mismatch')#
field file: str = None (alias 'sql.user_defined.file')#
field flow_dirty: str | None = 'false'#
field free_space_percent: int | None = 0 (alias 'table_action.generate_create_statement.create_table_options.table_free_space.free_space_percent')#
field generate_create_statement_at_runtime: bool | None = True (alias 'table_action.generate_create_statement')#
field generate_drop_statement_at_runtime: bool | None = True (alias 'table_action.generate_drop_statement')#
field generate_sql_at_runtime: bool | None = False (alias 'generate_sql')#
field generate_truncate_statement_at_runtime: bool | None = True (alias 'table_action.generate_truncate_statement')#
field generate_uow_id: bool = False (alias 'tmsmevents.generate_uowid')#
field has_reference_output: bool | None = False (alias 'has_ref_output')#
field hide: bool | None = False#
field inactivity_period: int = 300 (alias 'disconnect.inactivity_period')#
field input_count: int | None = 0#
field input_method: InputMethod | None = InputMethod.enter_credentials_manually (alias 'credentials_input_method')#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field insert_statement: str = None (alias 'sql.insert_statement')#
field interval_between_retries: int = 10 (alias 'reconnect.retry_interval')#
field isolation_level: SessionIsolationLevel | None = SessionIsolationLevel.default (alias 'session.isolation_level')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column: list | None = [] (alias 'record_ordering.key_column')#
field limit: int | None = 1000 (alias 'limit_rows.limit')#
field limit_number_of_returned_rows: bool | None = False (alias 'limit_rows')#
field load_type: BulkAccessLoadType | None = BulkAccessLoadType.load (alias 'bulk_access.load_type')#
field log_column_values_on_first_row_error: bool | None = False (alias 'logging.log_column_values')#
field log_key_values_only: bool | None = False (alias 'logging.log_column_values.log_keys_only')#
field log_table: str | None = None (alias 'bulk_access.log_table')#
field lookup_type: LookupType | None = LookupType.empty#
field macro_database: str | None = None (alias 'bulk_access.stream_load.macro_database')#
field make_duplicate_copies: TableActionGenerateCreateStatementCreateTableOptionsMakeDuplicateCopies | None = TableActionGenerateCreateStatementCreateTableOptionsMakeDuplicateCopies.default (alias 'table_action.generate_create_statement.create_table_options.make_duplicate_copies')#
field max_buffer_size: int | None = 0 (alias 'limit_settings.max_buffer_size')#
field max_mem_buf_size_ronly: int | None = 3145728#
field max_partition_sessions: int | None = 0 (alias 'limit_settings.max_partition_sessions')#
field max_sessions: int | None = 0 (alias 'limit_settings.max_sessions')#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field migrated_job: bool | None = False (alias 'is_migrated_job')#
field min_sessions: int | None = 0 (alias 'limit_settings.min_sessions')#
field missing_delete_rows: BulkAccessErrorControlMissingDeleteRows | None = BulkAccessErrorControlMissingDeleteRows.default (alias 'bulk_access.error_control.missing_delete_rows')#
field missing_update_rows: BulkAccessErrorControlMissingUpdateRows | None = BulkAccessErrorControlMissingUpdateRows.default (alias 'bulk_access.error_control.missing_update_rows')#
field number_of_retries: int = 3 (alias 'reconnect.retry_count')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field pack_size: int | None = 0 (alias 'bulk_access.stream_load.pack_size')#
field parallel_synchronization: bool | None = False#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_by_expression: str | None = None (alias 'table_action.generate_create_statement.create_table_options.partition_by_expression')#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field primary_index_type: TableActionGenerateCreateStatementCreateTableOptionsPrimaryIndexType | None = TableActionGenerateCreateStatementCreateTableOptionsPrimaryIndexType.non_unique (alias 'table_action.generate_create_statement.create_table_options.primary_index_type')#
field progress_interval: int | None = 100000 (alias 'limit_settings.progress_interval')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_delete_statement_from_file: bool | None = False (alias 'sql.delete_statement.read_from_file_delete')#
field read_insert_statement_from_file: bool | None = False (alias 'sql.insert_statement.read_from_file_insert')#
field read_select_statement_from_file: bool | None = False (alias 'sql.select_statement.read_from_file_select')#
field read_update_statement_from_file: bool | None = False (alias 'sql.update_statement.read_from_file_update')#
field reconnect: bool | None = False#
field record_count: int | None = 2000 (alias 'transaction.record_count')#
field record_ordering: RecordOrdering | None = RecordOrdering.zero#
field request_type: SqlUserDefinedRequestType | None = SqlUserDefinedRequestType.individual (alias 'sql.user_defined.request_type')#
field robust: BulkAccessStreamLoadRobust | None = BulkAccessStreamLoadRobust.yes (alias 'bulk_access.stream_load.robust')#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None (alias 'sql.select_statement')#
field select_statement_column: str | None = None (alias 'sql.select_statement.columns.column')#
field serialize: BulkAccessStreamLoadSerialize | None = BulkAccessStreamLoadSerialize.yes (alias 'bulk_access.stream_load.serialize')#
field serialize_modified_properties: bool | None = True#
field server_character_set: str | None = None (alias 'table_action.generate_create_statement.create_table_options.server_character_set')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sleep: int | None = 0 (alias 'bulk_access.sleep')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field source_contains_temporal_colummns: bool | None = False (alias 'source_temporal_support')#
field source_temporal_columns: SourceTemporalSupportTemporalColumns | None = SourceTemporalSupportTemporalColumns.none (alias 'source_temporal_support.temporal_columns')#
field source_temporal_qualifier_period_expression: str | None = None (alias 'target_temporal_support.temporal_qualifier.period_expression')#
field source_transaction_time_column: str = None (alias 'source_temporal_support.temporal_columns.transaction_time_column')#
field source_transaction_time_qualifier_date_timestamp_expression: str = None (alias 'source_temporal_support.transaction_time_qualifier.date_timestamp_expression')#
field source_valid_time_qualifier: SourceTemporalSupportValidTimeQualifier | None = SourceTemporalSupportValidTimeQualifier.none (alias 'source_temporal_support.valid_time_qualifier')#
field source_valid_time_qualifier_date_timestamp_expression: str = None (alias 'source_temporal_support.valid_time_qualifier.date_timestamp_expression')#
field source_valid_time_qualifier_period_expression: str | None = None (alias 'source_temporal_support.valid_time_qualifier.period_expression')#
field source_validate_time_column: str = None (alias 'source_temporal_support.temporal_columns.valid_time_column')#
field sql_delete_statement: str | None = None (alias 'sql.delete_statement.tables.table')#
field sql_delete_statement_character_set_name: str | None = None (alias 'sql.delete_statement.read_from_file_delete.character_set')#
field sql_delete_statement_parameters: str | None = None (alias 'sql.delete_statement.parameters.parameter')#
field sql_delete_statement_where_clause: str | None = None (alias 'sql.delete_statement.where_clause')#
field sql_insert_statement: str | None = None (alias 'sql.insert_statement.tables.table')#
field sql_insert_statement_character_set_name: str | None = None (alias 'sql.insert_statement.read_from_file_insert.character_set')#
field sql_insert_statement_parameters: str | None = None (alias 'sql.insert_statement.parameters.parameter')#
field sql_insert_statement_where_clause: str | None = None (alias 'sql.insert_statement.where_clause')#
field sql_other_clause: str | None = None (alias 'sql.other_clause')#
field sql_select_statement_character_set_name: str | None = None (alias 'sql.select_statement.read_from_file_select.character_set')#
field sql_select_statement_other_clause: str | None = None (alias 'sql.select_statement.other_clause')#
field sql_select_statement_parameters: str | None = None (alias 'sql.select_statement.parameters.parameter')#
field sql_select_statement_table_name: str | None = None (alias 'sql.select_statement.tables.table')#
field sql_select_statement_where_clause: str | None = None (alias 'sql.select_statement.where_clause')#
field sql_update_statement: str | None = None (alias 'sql.update_statement.tables.table')#
field sql_update_statement_character_set_name: str | None = None (alias 'sql.update_statement.read_from_file_update.character_set')#
field sql_update_statement_parameters: str | None = None (alias 'sql.update_statement.parameters.parameter')#
field sql_update_statement_where_clause: str | None = None (alias 'sql.update_statement.where_clause')#
field sql_where_clause: str | None = None (alias 'sql.where_clause')#
field ssl_mode_ssl_protocol: SslModeSslProtocol | None = None (alias 'ssl_mode.ssl_protocol')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field start_mode: BulkAccessStartMode | None = BulkAccessStartMode.clean (alias 'bulk_access.start_mode')#
field start_row: int | None = 0 (alias 'limit_settings.start_row')#
field sync_database: str | None = None (alias 'parallel_synchronization.sync_database')#
field sync_id: str | None = None (alias 'parallel_synchronization.sync_id')#
field sync_password: str | None = None (alias 'parallel_synchronization.sync_password')#
field sync_poll: int | None = 0 (alias 'parallel_synchronization.sync_poll')#
field sync_server: str | None = None (alias 'parallel_synchronization.sync_server')#
field sync_table: str = None (alias 'parallel_synchronization.sync_table')#
field sync_table_action: ParallelSynchronizationSyncTableAction | None = ParallelSynchronizationSyncTableAction.create (alias 'parallel_synchronization.sync_table_action')#
field sync_table_cleanup: ParallelSynchronizationSyncTableCleanup | None = ParallelSynchronizationSyncTableCleanup.keep (alias 'parallel_synchronization.sync_table_cleanup')#
field sync_table_write_mode: ParallelSynchronizationSyncTableWriteMode | None = ParallelSynchronizationSyncTableWriteMode.insert (alias 'parallel_synchronization.sync_table_write_mode')#
field sync_timeout: int | None = 0 (alias 'parallel_synchronization.sync_timeout')#
field sync_user: str | None = None (alias 'parallel_synchronization.sync_user')#
field table_action: TableAction = TableAction.append#
field table_free_space: TableActionGenerateCreateStatementCreateTableOptionsTableFreeSpace | None = TableActionGenerateCreateStatementCreateTableOptionsTableFreeSpace.default (alias 'table_action.generate_create_statement.create_table_options.table_free_space')#
field table_name: str = None#
field target_temporal_columns: TargetTemporalSupportTemporalColumns | None = TargetTemporalSupportTemporalColumns.none (alias 'target_temporal_support.temporal_columns')#
field target_transaction_time_column: str = None (alias 'target_temporal_support.temporal_columns.transaction_time_column')#
field target_validate_time_column: str = None (alias 'target_temporal_support.temporal_columns.valid_time_column')#
field temporal_qualifier: TargetTemporalSupportTemporalQualifier | None = TargetTemporalSupportTemporalQualifier.none (alias 'target_temporal_support.temporal_qualifier')#
field temporal_support: bool | None = False (alias 'target_temporal_support')#
field tenacity: int | None = 0 (alias 'bulk_access.tenacity')#
field tmsm_event_options: bool | None = False (alias 'tmsmevents')#
field transaction_time_qualifier: SourceTemporalSupportTransactionTimeQualifier | None = SourceTemporalSupportTransactionTimeQualifier.none (alias 'source_temporal_support.transaction_time_qualifier')#
field truncate_table_statement: str = '' (alias 'table_action.generate_truncate_statement.truncate_statement')#
field unique: bool | None = None (alias 'part_unique')#
field unused_field_action: SessionSchemaReconciliationUnusedFieldAction | None = SessionSchemaReconciliationUnusedFieldAction.abort (alias 'session.schema_reconciliation.unused_field_action')#
field uow_class: str | None = None (alias 'tmsmevents.uowclass')#
field uow_id: str = None (alias 'tmsmevents.generate_uowid.uowid')#
field uow_source: str | None = None (alias 'tmsmevents.uowsourcesystem')#
field update_statement: str = None (alias 'sql.update_statement')#
field use_cas_lite_service: bool | None = True (alias 'use_cas_lite')#
field user_defined_sql: SqlUserDefined = SqlUserDefined.statements (alias 'sql.user_defined')#
field user_defined_sql_statements: str = None (alias 'sql.user_defined.statements')#
field work_table: str | None = None (alias 'bulk_access.work_table')#
field write_mode: WriteMode = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/TeradataConnectorPX.svg'#
label: ClassVar[str] = 'Teradata database for DataStage'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'TeradataConnectorPX'#

This module defines configuration or the Vertica stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.vertica.vertica#

Bases: BaseStage

Properties for the Vertica stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field batch_size: int | None = 2000#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: VerticaConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "discover_data_assets": null,     "cluster_access_token": null,     "cluster_user_name": null,     "database": null,     "defer_credentials": false,     "hostname_or_ip_address": null,     "the_certificate_for_a_key": null,     "password": null,     "port": 5433,     "private_key": null,     "query_timeout": 300,     "retry_limit": 2,     "secure_gateway_id": null,     "sg_host_original": null,     "secure_gateway_as_http_proxy": null,     "secure_gateway_security_token": null,     "secure_gateway_service_url": null,     "satellite_client_certificate": null,     "satellite_client_private_key": null,     "satellite_connector_id": null,     "satellite_endpoint_host": null,     "satellite_endpoint_display_name": null,     "satellite_endpoint_port": null,     "sl_host_original": null,     "satellite_as_http_proxy": null,     "satellite_location_id": null,     "satellite_service_url": null,     "port_is_ssl_enabled": false,     "ssl_certificate": null,     "ssl_certificate_hostname": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "additional_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field create_statement: str | None = None#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_rounding_mode: DecimalRoundingMode | None = DecimalRoundingMode.floor#
field default_maximum_length_for_columns: int | None = 20000 (alias 'default_max_string_binary_precision')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 256 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field existing_table_action: ExistingTableAction | None = ExistingTableAction.append#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field generate_unicode_type_columns: bool | None = False (alias 'generate_unicode_columns')#
field has_reject_output: bool | None = False#
field hide: bool | None = False#
field infer_schema: bool | None = True (alias 'rcp')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_column_names: str | None = None#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field push_filters: str | None = None#
field pushed_filters: str | None = None#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field reject_condition_row_is_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.rows#
field rejected_filters: str | None = None#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field sampling_percentage: str | None = None#
field sampling_seed: int | None = None#
field sampling_type: SamplingType | None = SamplingType.none#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str = None#
field table_action: TableAction | None = TableAction.append#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
field update_statement: str | None = None#
field write_mode: WriteMode | None = WriteMode.insert#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/vertica.svg'#
label: ClassVar[str] = 'Vertica'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'vertica'#

This module defines configuration or the IBM watsonx.data Presto stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.watsonx_data.watsonx_data#

Bases: BaseStage

Properties for the IBM watsonx.data Presto stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_limit: str | None = None#
field catalog_name: str | None = None#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field connection: WatsonxDataConn = {     "asset_id": null,     "proj_id": null,     "raw_properties": null,     "name": null,     "api_key": null,     "authentication_method": "username_password",     "cloud_resource_name": null,     "cluster_access_token": null,     "cluster_user_name": null,     "crn": null,     "defer_credentials": false,     "deployment_type": null,     "engine_hostname": null,     "engine_id": null,     "engine_port": null,     "engine_ssl_enabled": null,     "engine_ssl_certificate": null,     "hostname_or_ip_address": null,     "instance_id": null,     "instance_name": null,     "connect_to_ibm_watsonx_data_on_red_hat_open_shift": false,     "password": null,     "port": null,     "service_to_service_authorization": null,     "ssl_is_enabled": true,     "ssl_certificate": null,     "validate_ssl_certificate": null,     "username": null,     "vaulted_properties": null,     "ssl_certificate_file": null,     "hidden_dummy_property1": null,     "hidden_dummy_property2": null }#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field defer_credentials: bool | None = False#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field ds_java_heap_size: int | None = 512 (alias '_java._heap_size')#
field enable_after_sql: str | None = '' (alias 'before_after.after')#
field enable_after_sql_node: str | None = '' (alias 'before_after.after_node')#
field enable_before_sql: str | None = '' (alias 'before_after.before')#
field enable_before_sql_node: str | None = '' (alias 'before_after.before_node')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_engine_id: str | None = None#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field fail_on_error_after_sql: bool | None = True (alias 'before_after.after.fail_on_error')#
field fail_on_error_after_sql_node: bool | None = True (alias 'before_after.after_node.fail_on_error')#
field fail_on_error_before_sql: bool | None = True (alias 'before_after.before.fail_on_error')#
field fail_on_error_before_sql_node: bool | None = True (alias 'before_after.before_node.fail_on_error')#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 1#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_size: int | None = 20 (alias 'write_part_size')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field read_method: ReadMethod | None = ReadMethod.general (alias 'read_mode')#
field row_limit: int | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name: str | None = None#
field select_statement: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field show_success_notification: bool | None = False#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field static_statement: str | None = None#
field table_action: TableAction | None = TableAction.create#
field table_name: str = None#
field unique: bool | None = None (alias 'part_unique')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/lakehouse.svg'#
label: ClassVar[str] = 'IBM watsonx.data Presto'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'lakehouse'#

Batch Flow#

Module for batch flow.

pydantic model ibm_watsonx_data_integration.services.datastage.models.flow.batch_flow.BatchFlow#

Bases: FlowComposer, Flow

Represents a batch flow.

Initialize a batch flow.

Parameters:

  • project (Project) – The project for the flow to be created in. If None, then the flow is not created remotely.

  • name – The flow name.

  • description – The flow description.

  • dag (Optional[DAG], default: None) – Optional dag to copy.

  • compile_mode (Optional[CompileMode], default: None) – Compilation mode of the flow.

  • elt_materialization_policy (Optional[ELTMaterializationPolicy], default: None) – If compiling in ELT mode, which materialization policy to use.

  • flow_type – The type of flow. This should always be datastage.

  • kwargs – Additional key word arguments.

field acp: bool = True#
field configuration: BatchFlowConfiguration = BatchFlowConfiguration()#
field description: str = ''#
field flow_id: str = ''#
field job_parameters: list[dict[str, Any]] = []#
field local_parameters: list[Parameter] = []#
field name: str = 'unnamed_flow'#
field parameter_sets: list[ParameterSet] = []#
field rcp: bool = False#
field value_set_selected: dict[str, str] = {}#
add_local_parameter(parameter_type: str, name: str, description: str = '', prompt: str = '', value: Any = '', valid_values: list[Any] | None = [])#

Add a local parameter to flow.

Return type:

BatchFlow

add_stage(type: Literal['Address Verification', 'Aggregator', 'Amazon RDS for MySQL', 'Amazon RDS for Oracle', 'Amazon RDS for PostgreSQL', 'Amazon Redshift', 'Amazon S3', 'Apache Cassandra', 'Apache Cassandra for DataStage', 'Apache Derby', 'Apache HBase', 'Apache HDFS', 'Apache Hive', 'Apache Impala', 'Apache Kafka', 'Azure PostgreSQL', 'Bloom Filter', 'Box', 'Change Apply', 'Change Capture', 'Checksum', 'Column Export', 'Column Generator', 'Column Import', 'Combine Records', 'Compare', 'Complex Flat File', 'Compress', 'Copy', 'Data Rules', 'Data set', 'DataStax Enterprise', 'Decode', 'Denodo', 'Difference', 'Dremio', 'Dropbox', 'Elasticsearch', 'Encode', 'Exasol', 'Expand', 'External Filter', 'External Source', 'External Target', 'FTP', 'File set', 'Filter', 'Funnel', 'Generic', 'Generic JDBC', 'Generic S3', 'Google BigQuery', 'Google Cloud Pub/Sub', 'Google Cloud Storage', 'Google Looker', 'Greenplum', 'HTTP', 'Head', 'Hierarchical Data', 'IBM Cloud Databases for MongoDB', 'IBM Cloud Databases for MySQL', 'IBM Cloud Databases for PostgreSQL', 'IBM Cloud Object Storage', 'IBM Cognos Analytics', 'IBM Data Virtualization', 'IBM Data Virtualization Manager for z/OS', 'IBM Db2', 'IBM Db2 Big SQL', 'IBM Db2 Warehouse', 'IBM Db2 for DataStage', 'IBM Db2 for i', 'IBM Db2 for z/OS', 'IBM Db2 on Cloud', 'IBM Informix', 'IBM MQ', 'IBM Match 360', 'IBM Netezza Performance Server', 'IBM Netezza Performance Server for DataStage', 'IBM Planning Analytics', 'IBM watsonx.data Presto', 'Investigate', 'Java Integration', 'Join', 'Lookup', 'Lookup file set', 'Make Subrecord', 'Make Vector', 'MariaDB', 'Match Frequency', 'Merge', 'Microsoft Azure Blob Storage', 'Microsoft Azure Cosmos DB', 'Microsoft Azure Data Lake Storage', 'Microsoft Azure Databricks', 'Microsoft Azure File Storage', 'Microsoft Azure SQL Database', 'Microsoft Azure Synapse Analytics', 'Microsoft SQL Server', 'MinIO', 'Modify', 'MongoDB', 'MySQL', 'ODBC', 'One-source Match', 'Oracle', 'Oracle Database for DataStage', 'Peek', 'Pivot Enterprise', 'PostgreSQL', 'Presto', 'Promote Subrecord', 'Remove Duplicates', 'Rest', 'Row Generator', 'SAP ASE', 'SAP BAPI', 'SAP Bulk Extract', 'SAP Delta Extract', 'SAP HANA', 'SAP IDoc', 'SAP IQ', 'SAP OData', 'Salesforce API for DataStage', 'Salesforce.com', 'Sample', 'Sequential file', 'SingleStoreDB', 'Slowly Changing Dimension', 'Snowflake', 'Sort', 'Split Subrecord', 'Split Vector', 'Standardize', 'Storage volume', 'Surrogate Key Generator', 'Survive', 'Switch', 'Tableau', 'Tail', 'Teradata', 'Teradata database for DataStage', 'Transformer', 'Two-source Match', 'Vertica', 'Wave Generator', 'Web Service', 'Write Range Map', 'XML Input', 'XML Output'], label: str | None = None)#

Add a stage to a batch flow.

add_subflow(subflow: Subflow, label: str)#

Add a subflow to a batch flow.

Return type:

Subflow

compile()#

Compile a flow and return the response.

Return type:

Response

create_subflow(label: str, nodes: set[Node] | None = None, *, is_local: bool = True, name: str = '', description: str = '')#

Creates a subflow within the flow.

Parameters:

  • label (str) – Label for the subflow in the flow

  • nodes (Optional[set[Node]], default: None) – Set of nodes to create a sublow from. If None, will create an empty subflow. Defaults to None.

  • is_local (bool, default: True) – If True, will create a local subflow. Will create an global subflow otherwise. Defaults to True.

  • name (str, default: '') – Name of the subflow if creating a global subflow. Defaults to “”.

  • description (str, default: '') – Description of the subflow if creating a global subflow. Defaults to “”.

Raises:

  • ValueError – if ‘name’ is set when creating a local subflow

  • ValueError – if ‘description’ is set when creating a local subflow

  • ValueError – if ‘name’ is NOT set when creating a global subflow

Returns:

subflow object created within the flow

Return type:

Subflow

deconstruct_subflow(subflow: Subflow)#

Deconstructs a local subflow in the flow.

Parameters:

subflow (Subflow) – subflow to desconstruct

Return type:

None

delete_subflow(subflow: Subflow)#

Deletes a local subflow in a flow.

Parameters:

subflow (Subflow) – subflow to delete

Return type:

None

duplicate_subflow(subflow: Subflow, name: str)#

Duplicates a local subflow in a flow.

Parameters:

  • subflow (Subflow) – subflow to duplicate

  • name (str) – name of the new subflow

Returns:

duplicated subflow

Return type:

duplicate_subflow

get_compile_info(include_osh: bool | None = None)#

Get response for compiling a flow.

Return type:

Response

get_compile_status()#

Get status for compiling a flow.

Return type:

Response

Gets the link between two nodes in the flow. If there are no links or multiple links, it raises an error.

Parameters:

  • source (Node) – Node from which the link originates.

  • destination (Node) – Node to which the link points.

Return type:

Link

Returns:

The single link between the source and destination nodes.

Gets all links between two nodes in the flow.

Parameters:

  • source (Node) – Node from which the links originate.

  • destination (Node) – Node to which the links point.

Return type:

list[Link]

Returns:

A list of Links between the source and destination nodes.

remove_stage(node: Node)#

Removes a node from the flow.

Parameters:

node (Node) – The node to be removed.

Return type:

None

set_runtime_local_parameter(local_parameter_name: str, value: Any)#

Set the value of a runtime local parameter for this flow to use during all job runs.

Return type:

BatchFlow

set_runtime_parameter_value(parameter_set_name: str, parameter_name: str, value: Any)#

Set the value of a runtime parameter in a parameter set for this flow to use during all job runs.

Return type:

BatchFlow

set_runtime_value_set(parameter_set_name: str, value_set_name: str)#

Set the value set of a parameter set for this flow.

Return type:

BatchFlow

use_auto_column_propagation(acp: bool = True)#

Enable acp.

Return type:

BatchFlow

use_parameter_set(parameter_set: ParameterSet)#

Use parameter set in flow.

Return type:

BatchFlow

use_projdef_parameter(parameter: Parameter)#

Use a PROJDEF parameter in flow.

Return type:

BatchFlow

use_runtime_column_propagation(rcp: bool = True)#

Enable rcp.

Return type:

BatchFlow

property compile_mode: CompileMode#

The compile mode for the BatchFlow (ETL/TETL/ELT).

property elt_materialization_policy: ELTMaterializationPolicy | None#

ELT materialization policy used when the flow’s compile_mode is CompileMode.ELT.

Returns the stored ELTMaterializationPolicy or None if not set. Setting this property is only allowed when compile_mode is CompileMode.ELT; attempting to set it in any other compile mode raises a RuntimeError.

Get Links in BatchFlow.

property stages: Stages#

Get stages in BatchFlow.

pydantic model ibm_watsonx_data_integration.services.datastage.models.flow.batch_flow.BatchFlowConfiguration#

Bases: BaseModel

Configuration for a BatchFlow.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field environment: str | None = None#
field retention_amount: int | None = None#
field retention_days: int | None = None#
field warn_limit: int | None = None#
as_dict()#

Return dict version of configuration for job._create call.

Return type:

dict

class ibm_watsonx_data_integration.services.datastage.models.flow.batch_flow.BatchFlowPayloadExtender#

Bases: PayloadExtender

Batch flow extender setup also compiles the flow.

extend(payload: dict[str, Any], flow: Flow)#

Entry point for starting pre-check and extending create job payload.

Return type:

dict[str, Any]

prepare_payload(payload: dict[str, Any], flow: Flow)#

Here we should modify and return payload for job creation.

Return type:

dict[str, Any]

run_pre_start_operations(flow: Flow)#

Here we should perform all flow prerequisite operations like validation, compilation etc.

When pre-checks fail we should raise FlowPreStartOperationError exception to carry out this information to user.

Raises:

FlowPreStartOperationError – If any pre-check operation failed.

Return type:

None

class ibm_watsonx_data_integration.services.datastage.models.flow.batch_flow.BatchFlows(project: Project)#

Bases: CollectionModel[BatchFlow]

Collection of BatchFlow objects.

The __init__ of the BatchFlows class.

Parameters:

project (Project) – The Project object.

get(**kwargs: dict)#

Used to get an instant result from the api.

Parameters:

**kwargs (dict) – Optional arguments to be passed to filter the results.

Return type:

BatchFlow

Returns:

An inherited instance of streamsets.sdk.sch_models.BaseModel.

Raises:

ValueError – If instance is not in the list.

get_all(**kwargs: dict)#

Used to get multiple (all) batch flow instances from the api.

Parameters:

**kwargs (dict) – Optional arguments to be passed to filter the results.

Return type:

BatchFlowsList

Returns:

A BatchFlowsList with automatic full data loading on access.

class ibm_watsonx_data_integration.services.datastage.models.flow.batch_flow.BatchFlowsList(items: list[Any], batch_flows_collection: BatchFlows)#

Bases: SeekableList

SeekableList for BatchFlows with automatic full data loading on access.

Initialize BatchFlowsList.

Parameters:

  • items (list[Any]) – The list of BatchFlow items.

  • batch_flows_collection (BatchFlows) – The BatchFlows collection for fetching full data.

append(object, /)#

Append object to the end of the list.

clear()#

Remove all items from list.

count(value, /)#

Return number of occurrences of value.

extend(iterable, /)#

Extend list by appending elements from the iterable.

get(**kwargs: dict)#

Retrieve the first instance that matches the supplied arguments.

Automatically loads full flow data with all attachments before returning.

Parameters:

**kwargs (dict) – Optional arguments to be passed to filter the results offline.

Return type:

BatchFlow

Returns:

A fully loaded BatchFlow instance that matches the supplied arguments.

get_all(**kwargs: dict)#

Retrieve all instances that match the supplied arguments.

Returns a new BatchFlowsList with lazy loading capability preserved.

Parameters:

**kwargs (dict) – Optional arguments to be passed to filter the results offline.

Return type:

BatchFlowsList

Returns:

A BatchFlowsList of results that match the supplied arguments.

index(value, start=0, stop=9223372036854775807, /)#

Return first index of value.

Raises ValueError if the value is not present.

insert(index, object, /)#

Insert object before index.

pop(index=-1, /)#

Remove and return item at index (default last).

Raises IndexError if list is empty or index is out of range.

remove(value, /)#

Remove first occurrence of value.

Raises ValueError if the value is not present.

reverse()#

Reverse IN PLACE.

sort(*, key=None, reverse=False)#

Sort the list in ascending order and return None.

The sort is in-place (i.e. the list itself is modified) and stable (i.e. the order of two equal elements is maintained).

If a key function is given, apply it once to each list item and sort them, ascending or descending, according to their function values.

The reverse flag can be set to sort in descending order.

class ibm_watsonx_data_integration.services.datastage.models.flow.batch_flow.CompileMode(value)#

Bases: Enum

Defines the execution run mode for the DataStage flow.

class ibm_watsonx_data_integration.services.datastage.models.flow.batch_flow.ELTMaterializationPolicy(label: str, enum_value: str)#

Bases: Enum

Defines the SQL pushdown (ELT) optimization strategy.

Initializes the ELTMaterializationPolicy enum member.

to_dict()#

Returns the dictionary representation for the JSON payload.

Return type:

dict

Bases: SeekableList

Seekable List of Links.

append(object, /)#

Append object to the end of the list.

clear()#

Remove all items from list.

count(value, /)#

Return number of occurrences of value.

extend(iterable, /)#

Extend list by appending elements from the iterable.

get(**kwargs: dict)#

Retrieve the first link that matches the arguments.

Return type:

Link

get_all(**kwargs: dict)#

Retrieve all links that match the arguments.

Return type:

Links

index(value, start=0, stop=9223372036854775807, /)#

Return first index of value.

Raises ValueError if the value is not present.

insert(index, object, /)#

Insert object before index.

pop(index=-1, /)#

Remove and return item at index (default last).

Raises IndexError if list is empty or index is out of range.

remove(value, /)#

Remove first occurrence of value.

Raises ValueError if the value is not present.

reverse()#

Reverse IN PLACE.

sort(*, key=None, reverse=False)#

Sort the list in ascending order and return None.

The sort is in-place (i.e. the list itself is modified) and stable (i.e. the order of two equal elements is maintained).

If a key function is given, apply it once to each list item and sort them, ascending or descending, according to their function values.

The reverse flag can be set to sort in descending order.

class ibm_watsonx_data_integration.services.datastage.models.flow.batch_flow.Stages(iterable=(), /)#

Bases: SeekableList

Seekable List of Batch Stages.

append(object, /)#

Append object to the end of the list.

clear()#

Remove all items from list.

count(value, /)#

Return number of occurrences of value.

extend(iterable, /)#

Extend list by appending elements from the iterable.

get(**kwargs: dict)#

Retrieve the first stage that matches the arguments.

Return type:

StageNode

get_all(**kwargs: dict)#

Retrieve all stages that match the arguments.

Return type:

Stages

index(value, start=0, stop=9223372036854775807, /)#

Return first index of value.

Raises ValueError if the value is not present.

insert(index, object, /)#

Insert object before index.

pop(index=-1, /)#

Remove and return item at index (default last).

Raises IndexError if list is empty or index is out of range.

remove(value, /)#

Remove first occurrence of value.

Raises ValueError if the value is not present.

reverse()#

Reverse IN PLACE.

sort(*, key=None, reverse=False)#

Sort the list in ascending order and return None.

The sort is in-place (i.e. the list itself is modified) and stable (i.e. the order of two equal elements is maintained).

If a key function is given, apply it once to each list item and sort them, ascending or descending, according to their function values.

The reverse flag can be set to sort in descending order.

Batch Generator#

class ibm_watsonx_data_integration.services.datastage.codegen.python_generator.PythonGenerator#

Bases: object

Converts DataStage flows exported as a flow JSON or ZIP file to Python SDK code.

generate(input_path: str | BatchFlow | list[BatchFlow], output_path=None)#

Runs the Python converter, generating Python code and writing it to the specified output path.

Parameters:

  • input_path (str | BatchFlow | list[BatchFlow]) – The path to the input file. Can be a flow JSON file or a ZIP file containing the flow and its dependencies.

  • output_path (optional) – The output path where the generated Python code will be written.

Returns:

A dictionary containing the generated code strings.

get_code(flow_name='', input_path='', input_json=None)#

Returns the generated code as a string. Applicable only for JSON inputs by path or dict.

class ibm_watsonx_data_integration.services.datastage.codegen.python_generator.PythonGeneratorConfig(*, mode: Literal['single_file', 'file_per_flow'] = 'file_per_flow', create_job: bool = True, run_job: bool = True, overwrite: bool = False, use_flow_name: bool = True, api_key: str = '<TODO: insert your api_key>', project_id: str = '<TODO: insert your project_id>', base_auth_url: str = 'https://cloud.ibm.com', base_api_url: str = 'https://api.ca-tor.dai.cloud.ibm.com', persist_topology: bool = False)#

Bases: object

The underlying settngs for the python generator.

Parameters:

  • mode (Literal['single_file', 'file_per_flow'], default: 'file_per_flow') – The export mode, or how the generated code will be structured. single_file puts all code in one file. file_per_flow (default) puts each flow in its own file with relevant assets included as needed.

  • create_job (bool, default: True) – Whether to create a job for the flow(s).

  • run_job (bool, default: True) – Whether to run the created job. Only checked if create_job is True.

  • overwrite (bool, default: False) – Whether to overwrite the output path directory.

  • use_flow_name (bool, default: True) – Whether the flow and jobs use the flow’s original name (applies for zip structure only).

  • api_key (str, default: '<TODO: insert your api_key>') – Your API key. Provide the API key now or replace the placeholder later.

  • project_id (str, default: '<TODO: insert your project_id>') – Your project ID. Provide the project ID now or replace the placeholder later.

Batch Node#

class ibm_watsonx_data_integration.services.datastage.models.flow.dag.StageNode(dag: DAG, configuration: BaseModel, *, label: str = '')#

Bases: Node

Stage node for a dag.

Initializes a stage node.

connect_input_to(node: Node)#

Connects the input of this node to the output of another node.

Parameters:

node (Node) – The source node to connect from.

Return type:

Link

Returns:

A Link object representing the connection.

connect_output_to(node: Node)#

Connects the output of this node to the input of another node.

Parameters:

node (Node) – The destination node to connect to.

Return type:

Link

Returns:

A Link object representing the connection.

disconnect_input_from(src: Node)#

Disconnects the input of this node from the output of another node.

Parameters:

src (Node) – The source node to disconnect from.

Return type:

list[Link]

Returns:

The list of Links that were removed.

disconnect_output_from(dest: Node)#

Disconnects the output of this node from the input of another node.

Parameters:

dest (Node) – The destination node to disconnect from.

Return type:

list[Link]

Returns:

The list of Links that were removed.

Get all input links of node.

Return type:

list[Link]

Returns links that have the specified destination.

Return type:

Link

Returns: All links between this node and the destination node.

Return type:

list[Link]

Get all output links of node.

Return type:

list[Link]

use_connection(connection: Connection)#

Adds a project-level connection to this connector stage.

Return type:

None

Batch Schema#

Module for schema.

class ibm_watsonx_data_integration.services.datastage.models.schema.schema.Schema(fields: list[BaseField] | None = None)#

Bases: object

Column schema for batch stage.

Initializes an empty schema or a schema based on inputted fields.

add_field(odbc_type: str, name: str, description: str = '', nullable: bool | None = None, key: bool | None = None, source: str | None = None, pivot: str | None = None, length: int | None = None, scale: int | None = None, unicode: bool | None = None, unsigned: bool | None = None, extended_type: str | None = None, cluster_key_change: bool | None = None, key_change: bool | None = None, difference: bool | None = None, c_format: str | None = None, default: str | int | None = None, format_string: str | None = None, decimal_separator: DecimalSeparator | None = None, out_format: str | None = None, link_field_reference: str | None = None, padchar: PadChar | None = None, prefix_bytes: Prefix | None = None, vector_prefix: VectorPrefix | None = None, epoch: int | None = None, field_max_width: int | None = None, precision: int | None = None, scale_factor: int | None = None, start_position: int | None = None, percent_invalid: int | None = None, percent_zeros: int | None = None, tag_case_value: int | None = None, days_since: int | None = None, field_width: int | None = None, generate_on_output: bool | None = None, julian: bool | None = None, is_link_field: bool | None = None, check_packed: bool | None = None, export_ebcdic_as_ascii: bool | None = None, packed_signed: bool | None = None, is_midnight_seconds: bool | None = None, packed: DecimalPacked | None = None, byte_order: ByteOrder | None = None, charset: CharSet | None = None, allow_all_zeros: bool | None = None, sign_position: SignPosition | None = None, data_format: DataFormat | None = None, level_number: int | None = None, vector: VectorType | None = None, vector_occurs: int | None = None, byte_to_skip: int | None = None, delimeter: Delim | None = None, delimeter_string: str | None = None, quote: Quote | None = None, cycle_increment: CycleIncrement | int | None = None, cycle_initial_value: CycleInitialValue | int | None = None, cycle_limit: CycleLimit | int | None = None, cycle_values: list[int | str] | None = None, alphabet: str | None = None, generate_algorithm: GenerateAlgorithm | None = None, generate_type: GenerateType | None = None, random_limit: RandomLimit | None = None, random_seed: RandomSeed | None = None, random_signed: bool | None = None, use_current_date: bool | None = None, rounding: Round | None = None, actual_field_length: int | None = None, null_field_length: int | None = None, null_field_value: int | None = None, decimal_type_scale: int | None = None, null_seed: int | None = None, percent_null: int | None = None, change_code: bool | None = None, derivation: str | None = None, timezone: bool | None = None, microseconds: bool | None = None)#

Add a field to the schema.

clone()#

Clone a schema.

Return type:

Schema

property configuration: RecordSchema#

Computed configuration.

remove_field(name: str)#

Remove a field by name from the schema.

Return type:

Schema

select_fields(*args: str)#

Create a new schema with selected fields.

Return type:

Schema

Batch Stages#

This module defines configuration or the Address Verification stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.address_verification.address_verification#

Bases: BaseStage

Properties for the Address Verification stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field add_server_options: str | None = '' (alias 'addServerOptions')#
field address_count: str = '1' (alias 'addressCount')#
field address_line_properties: list | None = [] (alias 'addressLineProperties')#
field address_line_separator: AddressLineSeparator | None = AddressLineSeparator.PIPE (alias 'addressLineSeparator')#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: str | None = ''#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: str | None = ''#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field company_name: str = '' (alias 'companyName')#
field db2_instance_select: Db2InstanceSelect | None = Db2InstanceSelect.use_db2InstanceEnv (alias 'db2InstanceSelect')#
field db2_name_select: Db2NameSelect | None = Db2NameSelect.use_db2NameEnv (alias 'db2NameSelect')#
field db_path: str = '/opt/ibm/data/avi/LQTDB/' (alias 'dbPath')#
field default_country: str | None = '' (alias 'defaultCountry')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: str | None = ''#
field enable_field_match_score: EnableFieldMatchScore | None = EnableFieldMatchScore.No (alias 'enableFieldMatchScore')#
field enable_field_status: EnableFieldStatus | None = EnableFieldStatus.No (alias 'enableFieldStatus')#
field enhanced_g_b: EnhancedGB | None = EnhancedGB.No (alias 'enhancedGB')#
field enhanced_u_s: EnhancedUS | None = EnhancedUS.No (alias 'enhancedUS')#
field error: Error | None = Error.No#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field geo_coding: GeoCoding | None = GeoCoding.No (alias 'geoCoding')#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.Select_a_column (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field less_used_fields: LessUsedFields | None = LessUsedFields.No (alias 'lessUsedFields')#
field list_i_d: str = '' (alias 'listID')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: str | None = ''#
field max_results: int | None = 3 (alias 'maxResults')#
field minimum_verification_level: MinimumVerificationLevel | None = MinimumVerificationLevel.none (alias 'minimumVerificationLevel')#
field op_callbacks: str | None = 'qsavlibCalls' (alias 'opCallbacks')#
field output_casing: OutputCasing | None = OutputCasing.Title (alias 'outputCasing')#
field output_count: int | None = 0#
field output_script: OutputScript | None = OutputScript.AsProcessed (alias 'outputScript')#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field parse_properties: list | None = [] (alias 'parseProperties')#
field part_dbname: str | None = 'eg: SAMPLE'#
field part_server: str | None = 'eg: DB2INST'#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field processing_type: ProcessingType = ProcessingType.Parse (alias 'processingType')#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: str | None = ''#
field report_filefor_validation: str = '' (alias 'reportFileforValidation')#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = 'If sort keys are added, a sort will be performed.' (alias 'sortInstructions')#
field stage_description: list | None = '' (alias 'stageDescription')#
field use_city_abbreviations: UseCityAbbreviations | None = UseCityAbbreviations.No (alias 'useCityAbbreviations')#
field use_symbolic_transliteration: UseSymbolicTransliteration | None = UseSymbolicTransliteration.true (alias 'useSymbolicTransliteration')#
field validation_properties: list | None = [] (alias 'validationProperties')#
field validation_type: ValidationType | None = ValidationType.CorrectionOnly (alias 'validationType')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/AddressVerification2.svg'#
label: ClassVar[str] = 'Address Verification'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'AddressVerification2'#

This module defines configuration or the Aggregator stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.aggregator.aggregator#

Bases: BaseStage

Properties for the Aggregator stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field count_field_properties: list | None = [] (alias 'countFieldProperties')#
field default_to_decimal: str | None = '8,2' (alias 'defaultToDecimal')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field method: Method | None = Method.hash#
field nul_res: NulRes | None = NulRes.false#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_set#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field reduce_properties: list | None = [] (alias 'reduceProperties')#
field rereduce_properties: list | None = [] (alias 'rereduceProperties')#
field runtime_column_propagation: bool | None = None#
field selection: Selection | None = Selection.reduce#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxAggregator.svg'#
label: ClassVar[str] = 'Aggregator'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxAggregator'#

This module defines configuration or the Bloom Filter stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.bloom_filter.bloom_filter#

Bases: BaseStage

Properties for the Bloom Filter stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field data_date: str | None = None#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field drop_old: DropOld | None = DropOld.false#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field filter: str = None#
field filter_phases: int | None = 5#
field filter_size: str = None#
field flag_column: FlagColumn | None = FlagColumn.false#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field previous_days: int | None = 1#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field ref_date: str | None = ''#
field runtime_column_propagation: bool | None = None#
field selection: Selection | None = Selection.create#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field truncate: Truncate | None = Truncate.false#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxBLM.svg'#
label: ClassVar[str] = 'Bloom Filter'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxBLM'#

This module defines configuration or the Change Apply stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.change_apply.change_apply#

Bases: BaseStage

Properties for the Change Apply stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field code_field: str | None = None (alias 'codeField')#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = None#
field combinability: Combinability | None = Combinability.auto#
field copy_code: int | None = 0 (alias 'copyCode')#
field delete_code: int | None = 2 (alias 'deleteCode')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field do_stats: DoStats | None = DoStats.false (alias 'doStats')#
field dropkey: list | None = []#
field dropvalue: list | None = []#
field edit_code: int | None = 3 (alias 'editCode')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field hide_case_sensitive_column: bool | None = None (alias 'hideCaseSensitiveColumn')#
field ignore_delete_values: IgnoreDeleteValues | None = IgnoreDeleteValues.false (alias 'ignoreDeleteValues')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field insert_code: int | None = 1 (alias 'insertCode')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_set#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field selection: Selection | None = Selection.custom#
field selection_asc_desc: SelectionAscDesc | None = SelectionAscDesc.asc (alias 'selection-asc-desc')#
field selection_ci_cs: SelectionCiCs | None = SelectionCiCs.cs (alias 'selection-ci-cs')#
field selection_nulls: SelectionNulls | None = SelectionNulls.first (alias 'selection-nulls')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field value_properties: list | None = [] (alias 'valueProperties')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxChangeApply.svg'#
label: ClassVar[str] = 'Change Apply'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxChangeApply'#

This module defines configuration or the Change Capture stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.change_capture.change_capture#

Bases: BaseStage

Properties for the Change Capture stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field code_field: str | None = None (alias 'codeField')#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = None#
field combinability: Combinability | None = Combinability.auto#
field copy_code: int | None = 0 (alias 'copyCode')#
field delete_code: int | None = 2 (alias 'deleteCode')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field do_stats: DoStats | None = DoStats.false (alias 'doStats')#
field dropkey: list | None = []#
field dropvalue: list | None = []#
field edit_code: int | None = 3 (alias 'editCode')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field hide_case_sensitive_column: bool | None = None (alias 'hideCaseSensitiveColumn')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field insert_code: int | None = 1 (alias 'insertCode')#
field keep_copy_drop_copy: KeepCopyDropCopy | None = KeepCopyDropCopy.dropCopy (alias 'keepCopy-dropCopy')#
field keep_delete_drop_delete: KeepDeleteDropDelete | None = KeepDeleteDropDelete.keepDelete (alias 'keepDelete-dropDelete')#
field keep_edit_drop_edit: KeepEditDropEdit | None = KeepEditDropEdit.keepEdit (alias 'keepEdit-dropEdit')#
field keep_insert_drop_insert: KeepInsertDropInsert | None = KeepInsertDropInsert.keepInsert (alias 'keepInsert-dropInsert')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_set#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field selection: Selection | None = Selection.custom#
field selection_asc_desc: SelectionAscDesc | None = SelectionAscDesc.asc (alias 'selection-asc-desc')#
field selection_ci_cs: SelectionCiCs | None = SelectionCiCs.cs (alias 'selection-ci-cs')#
field selection_nulls: SelectionNulls | None = SelectionNulls.first (alias 'selection-nulls')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field value_properties: list | None = [] (alias 'valueProperties')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxChangeCapture.svg'#
label: ClassVar[str] = 'Change Capture'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxChangeCapture'#

This module defines configuration or the Checksum stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.checksum.checksum#

Bases: BaseStage

Properties for the Checksum stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field checksum_name: str | None = None#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field comp_mode: CompMode | None = CompMode.all#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field dropcol: list = []#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field export_name: str | None = None#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field keepcol: list = []#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxChecksum.svg'#
label: ClassVar[str] = 'Checksum'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxChecksum'#

This module defines configuration or the Column Export stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.column_export.column_export#

Bases: BaseStage

Properties for the Column Export stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field actual_length: int | None = None#
field allow_all_zeros: AllowAllZeros | None = AllowAllZeros.nofix_zero#
field allow_signed_import: AllowSignedImport | None = AllowSignedImport.allow_signed_import#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field byte_order: ByteOrder | None = ByteOrder.native_endian#
field c_format: str | None = ''#
field charset: Charset | None = Charset.ascii#
field check_intact: CheckIntact | None = CheckIntact.check_intact#
field check_intact_flag: bool | None = False#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field data_format: DataFormat | None = DataFormat.text#
field date_format: str | None = ''#
field date_option: DateOption | None = DateOption.none (alias 'dateOption')#
field days_since: str | None = ''#
field decimal_options: DecimalOption | None = [] (alias 'decimalOption')#
field decimal_packed: DecimalPacked | None = DecimalPacked.packed#
field decimal_packed_check: DecimalPackedCheck | None = DecimalPackedCheck.check#
field decimal_packed_sign_position: DecimalPackedSignPosition | None = DecimalPackedSignPosition.trailing#
field decimal_packed_signed: DecimalPackedSigned | None = DecimalPackedSigned.signed#
field decimal_sep_value: str | None = ''#
field decimal_separator: DecimalSeparator | None = DecimalSeparator.period#
field delim: Delim | None = Delim.comma#
field delim_string: str | None = ''#
field delim_value: str | None = ''#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field export_ebcdic_as_ascii: ExportEbcdicAsAscii | None = ExportEbcdicAsAscii.export_ebcdic_as_ascii#
field field: str = None#
field field_option: FieldOption | None = FieldOption.delimiter (alias 'fieldOption')#
field fill: Fill | None = Fill.null#
field fill_char_value: str | None = ''#
field final_delim: FinalDelim | None = FinalDelim.end#
field final_delim_string: str | None = ''#
field final_delim_value: str | None = ''#
field flow_dirty: str | None = 'false'#
field general_option: GeneralOption | None = [] (alias 'generalOption')#
field hide: bool | None = False#
field import_ascii_as_ebcdic: ImportAsciiAsEbcdic | None = ImportAsciiAsEbcdic.import_ascii_as_ebcdic#
field in_format: str | None = ''#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field intact: str | None = ''#
field is_julian: IsJulian | None = IsJulian.julian#
field is_midnight_seconds: IsMidnightSeconds | None = IsMidnightSeconds.midnight_seconds#
field keep_exported_fields: KeepExportedFields | None = KeepExportedFields.false (alias 'keepExportedFields')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_length: int | None = 200 (alias 'maxLength')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field max_width: int | None = None#
field null_field: str | None = 'NULL'#
field null_field_sep: NullFieldSep | None = NullFieldSep.comma#
field null_field_sep_flag: bool | None = False#
field null_field_sep_value: str | None = ''#
field null_length: int | None = None#
field numeric_option: NumericOption | None = [] (alias 'numericOption')#
field out_format: str | None = ''#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field padchar: PadChar | None = PadChar.space#
field padchar_value: str | None = ''#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field precision: int | None = None#
field prefix: Prefix | None = Prefix.one#
field preserve: Preserve | None = Preserve.default_propagate#
field print_field: PrintField | None = PrintField.print_field#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field quote: Quote | None = Quote.double#
field quote_value: str | None = ''#
field rec_level_option: RecLevelOption | None = [] (alias 'recLevelOption')#
field record_delim: RecordDelim | None = RecordDelim.newline#
field record_delim_string: str | None = ''#
field record_delim_value: str | None = ''#
field record_format: RecordFormat | None = RecordFormat.type_implicit#
field record_len_value: int | None = 0#
field record_length: RecordLength | None = RecordLength.fixed#
field record_prefix: RecordPrefix | None = RecordPrefix.one#
field round: Round | None = Round.trunc_zero#
field runtime_column_propagation: bool | None = None#
field save_rejects: SaveRejects | None = SaveRejects.custom (alias 'saveRejects')#
field scale: int | None = None#
field schema_: list | None = [] (alias 'schema')#
field schemafile: str = None#
field selection: Selection | None = Selection.explicit#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field string_option: StringOption | None = [] (alias 'stringOption')#
field time_format: str | None = ''#
field time_option: TimeOption | None = TimeOption.none (alias 'timeOption')#
field timestamp_format: str | None = ''#
field timestamp_option: TimestampOption | None = TimestampOption.none (alias 'timestampOption')#
field type: Type | None = Type.raw#
field vector_prefix: VectorPrefix | None = VectorPrefix.one#
field width: int | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxColumnExport.svg'#
label: ClassVar[str] = 'Column Export'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxColumnExport'#

This module defines configuration or the Column Generator stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.column_generator.column_generator#

Bases: BaseStage

Properties for the Column Generator stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field schema_: list = [] (alias 'schema')#
field schema_name_check: bool | None = None (alias 'schemaNameCheck')#
field schemafile: str = None#
field selection: Selection | None = Selection.explicit#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxColumnGenerator.svg'#
label: ClassVar[str] = 'Column Generator'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxColumnGenerator'#

This module defines configuration or the Column Import stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.column_import.column_import#

Bases: BaseStage

Properties for the Column Import stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field actual_length: int | None = None#
field allow_all_zeros: AllowAllZeros | None = AllowAllZeros.nofix_zero#
field allow_signed_import: AllowSignedImport | None = AllowSignedImport.allow_signed_import#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field byte_order: ByteOrder | None = ByteOrder.native_endian#
field c_format: str | None = ''#
field charset: Charset | None = Charset.ascii#
field check_intact: CheckIntact | None = CheckIntact.check_intact#
field check_intact_flag: bool | None = False#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field data_format: DataFormat | None = DataFormat.text#
field date_format: str | None = ''#
field date_option: DateOption | None = DateOption.none (alias 'dateOption')#
field days_since: str | None = ''#
field decimal_options: DecimalOption | None = [] (alias 'decimalOption')#
field decimal_packed: DecimalPacked | None = DecimalPacked.packed#
field decimal_packed_check: DecimalPackedCheck | None = DecimalPackedCheck.check#
field decimal_packed_sign_position: DecimalPackedSignPosition | None = DecimalPackedSignPosition.trailing#
field decimal_packed_signed: DecimalPackedSigned | None = DecimalPackedSigned.signed#
field decimal_sep_value: str | None = ''#
field decimal_separator: DecimalSeparator | None = DecimalSeparator.period#
field delim: Delim | None = Delim.comma#
field delim_string: str | None = ''#
field delim_value: str | None = ''#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field export_ebcdic_as_ascii: ExportEbcdicAsAscii | None = ExportEbcdicAsAscii.export_ebcdic_as_ascii#
field field: str = ' '#
field field_option: FieldOption | None = FieldOption.delimiter (alias 'fieldOption')#
field fill: Fill | None = Fill.null#
field fill_char_value: str | None = ''#
field final_delim: FinalDelim | None = FinalDelim.end#
field final_delim_string: str | None = ''#
field final_delim_value: str | None = ''#
field flow_dirty: str | None = 'false'#
field general_option: GeneralOption | None = [] (alias 'generalOption')#
field hide: bool | None = False#
field import_ascii_as_ebcdic: ImportAsciiAsEbcdic | None = ImportAsciiAsEbcdic.import_ascii_as_ebcdic#
field in_format: str | None = ''#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field intact: str | None = ''#
field is_julian: IsJulian | None = IsJulian.julian#
field is_midnight_seconds: IsMidnightSeconds | None = IsMidnightSeconds.midnight_seconds#
field keep_field: KeepField | None = KeepField.false (alias 'keepField')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field max_width: int | None = None#
field null_field: str | None = 'NULL'#
field null_field_sep: NullFieldSep | None = NullFieldSep.comma#
field null_field_sep_flag: bool | None = False#
field null_field_sep_value: str | None = ''#
field null_length: int | None = None#
field numeric_option: NumericOption | None = [] (alias 'numericOption')#
field out_format: str | None = ''#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field padchar: Padchar | None = Padchar.false_#
field padchar_value: str | None = ''#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field precision: int | None = None#
field prefix: Prefix | None = Prefix.one#
field preserve: Preserve | None = Preserve.default_propagate#
field print_field: PrintField | None = PrintField.print_field#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field quote: Quote | None = Quote.double#
field quote_value: str | None = ''#
field rec_level_option: RecLevelOption | None = [] (alias 'recLevelOption')#
field record_delim: RecordDelim | None = RecordDelim.newline#
field record_delim_string: str | None = ''#
field record_delim_value: str | None = ''#
field record_format: RecordFormat | None = RecordFormat.type_implicit#
field record_len_value: int | None = 0#
field record_length: RecordLength | None = RecordLength.fixed#
field record_prefix: RecordPrefix | None = RecordPrefix.one#
field round: Round | None = Round.trunc_zero#
field runtime_column_propagation: bool | None = None#
field save_rejects_fail_rejects: SaveRejectsFailRejects | None = SaveRejectsFailRejects.warn (alias 'saveRejects-failRejects')#
field scale: int | None = None#
field schema_: list | None = [] (alias 'schema')#
field schemafile: str = None#
field selection: Selection | None = Selection.explicit#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field string_option: StringOption | None = [] (alias 'stringOption')#
field time_format: str | None = ''#
field time_option: TimeOption | None = TimeOption.none (alias 'timeOption')#
field timestamp_format: str | None = ''#
field timestamp_option: TimestampOption | None = TimestampOption.none (alias 'timestampOption')#
field vector_prefix: VectorPrefix | None = VectorPrefix.one#
field width: int | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxColumnImport.svg'#
label: ClassVar[str] = 'Column Import'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxColumnImport'#

This module defines configuration or the Combine Records stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.combine_records.combine_records#

Bases: BaseStage

Properties for the Combine Records stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field hide_case_sensitive_column: bool | None = None (alias 'hideCaseSensitiveColumn')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field schema_name_check: bool | None = None (alias 'schemaNameCheck')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field subrecname: str = None#
field toplevelkeys: Toplevelkeys | None = Toplevelkeys.false#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxCombineRecords.svg'#
label: ClassVar[str] = 'Combine Records'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxCombineRecords'#

This module defines configuration or the Compare stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.compare.compare#

Bases: BaseStage

Properties for the Compare stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field abort_on_difference: AbortOnDifference | None = AbortOnDifference.false (alias 'abortOnDifference')#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field eq: int | None = 0#
field execmode: Execmode | None = Execmode.default_par#
field field_properties: list | None = [] (alias 'fieldProperties')#
field first: int | None = -2#
field flow_dirty: str | None = 'false'#
field gt: int | None = 1#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field lt: int | None = -1#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field second: int | None = -2#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field warn_record_count_mismatch: WarnRecordCountMismatch | None = WarnRecordCountMismatch.false (alias 'warnRecordCountMismatch')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxCompare.svg'#
label: ClassVar[str] = 'Compare'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxCompare'#

This module defines configuration or the Complex Flat File stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.complex_flat_file.complex_flat_file#

Bases: BaseStage

Properties for the Complex Flat File stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field allow_column_mapping: AllowColumnMapping | None = AllowColumnMapping.false#
field allowzeros: Allowzeros | None = Allowzeros.nofix_zero#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field byteorder: Byteorder | None = Byteorder.native_endian#
field chardefault: str | None = None#
field charset: Charset | None = Charset.ebcdic#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field connect_to_zos: ConnectToZos | None = ConnectToZos.custom#
field constraint: list | None = [] (alias 'predicate')#
field data_asset_name: str = None (alias 'dataAssetName')#
field dataformat: Dataformat | None = Dataformat.binary#
field decdefault: str | None = None#
field decrounding: Decrounding | None = Decrounding.round_inf#
field decsep: Decsep | None = Decsep.default#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = None#
field field_rejects: FieldRejects | None = FieldRejects.keepField#
field filename: str = None#
field filter: str | None = None#
field first: int | None = None#
field floatrepresentation: Floatrepresentation | None = Floatrepresentation.IEEE#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field intdefault: str | None = None#
field ismultipleformat: bool | None = False#
field keep_partitions: KeepPartitions | None = KeepPartitions.false (alias 'keepPartitions')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field missing_file: MissingFile | None = MissingFile.custom (alias 'missingFile')#
field multinode: Multinode | None = Multinode.no#
field nls_map_name: NlsMapName | None = NlsMapName.UTF_8#
field nocleanup: Nocleanup | None = Nocleanup.false#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_columns: list | None = []#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field padchar: str | None = None#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.clear#
field print_field: PrintField | None = PrintField.false#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field readers: int | None = 1#
field record_prefix: RecordPrefix | None = RecordPrefix.custom#
field recorddelimiter: Recorddelimiter | None = Recorddelimiter.custom#
field records: list | None = []#
field records_id: list | None = []#
field recordtype: Recordtype | None = Recordtype.F#
field register_data_asset: bool | None = False (alias 'registerDataAsset')#
field rejects: Rejects | None = Rejects.cont#
field report_progress: ReportProgress | None = ReportProgress.yes (alias 'reportProgress')#
field runtime_column_propagation: bool | None = None#
field selection: Selection | None = Selection.singlefile#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field writeoption: Writeoption | None = Writeoption.overwrite#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxCFF.svg'#
label: ClassVar[str] = 'Complex Flat File'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxCFF'#

This module defines configuration or the Compress stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.compress.compress#

Bases: BaseStage

Properties for the Compress stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field command: Command | None = Command.compress#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_set#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxCompress.svg'#
label: ClassVar[str] = 'Compress'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxCompress'#

This module defines configuration or the Copy stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.copy.copy#

Bases: BaseStage

Properties for the Copy stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field force: Force | None = Force.false#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxCopy.svg'#
label: ClassVar[str] = 'Copy'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxCopy'#

This module defines configuration or the Data Rules stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.data_rules.data_rules#

Bases: BaseStage

Properties for the Data Rules stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field logic: str = None#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/IADataRule.svg'#
label: ClassVar[str] = 'Data Rules'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'IADataRule'#

This module defines configuration or the Data set stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.dataset.dataset#

Bases: BaseStage

Properties for the Data set stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field file: str = None (alias 'dataset')#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field missing_columns_mode: MissingColumnsMode | None = MissingColumnsMode.custom (alias 'missingcolmode')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_propagate (alias 'preserve')#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field unique: bool | None = None (alias 'part_unique')#
field update_policy: UpdatePolicy | None = UpdatePolicy.overwrite (alias 'datasetmode')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxDataSet.svg'#
label: ClassVar[str] = 'Data set'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'PxDataSet'#

This module defines configuration or the Decode stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.decode.decode#

Bases: BaseStage

Properties for the Decode stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field command: str = None#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxDecode.svg'#
label: ClassVar[str] = 'Decode'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxDecode'#

This module defines configuration or the Difference stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.difference.difference#

Bases: BaseStage

Properties for the Difference stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field all_values: AllValues | None = AllValues.false (alias 'allValues')#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field ci_cs2: CiCs2 | None = CiCs2.cs (alias 'ci-cs2')#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field copy_code: int | None = 2 (alias 'copyCode')#
field delete_code: int | None = 1 (alias 'deleteCode')#
field diff_name: str | None = None#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field drop_copy: DropCopy | None = DropCopy.false (alias 'dropCopy')#
field drop_delete: DropDelete | None = DropDelete.false (alias 'dropDelete')#
field drop_edit: DropEdit | None = DropEdit.false (alias 'dropEdit')#
field drop_insert: DropInsert | None = DropInsert.false (alias 'dropInsert')#
field edit_code: int | None = 3 (alias 'editCode')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field hide_case_sensitive_column: bool | None = None (alias 'hideCaseSensitiveColumn')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field insert_code: int | None = 0 (alias 'insertCode')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field stats: Stats | None = Stats.false#
field tolerate_unsorted: TolerateUnsorted | None = TolerateUnsorted.false (alias 'tolerateUnsorted')#
field value_ci_cs: ValueCiCs | None = ValueCiCs.cs (alias 'value-ci-cs')#
field value_properties: list | None = [] (alias 'valueProperties')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxDifference.svg'#
label: ClassVar[str] = 'Difference'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxDifference'#

This module defines configuration or the Encode stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.encode.encode#

Bases: BaseStage

Properties for the Encode stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field command: str = None#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_set#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxEncode.svg'#
label: ClassVar[str] = 'Encode'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxEncode'#

This module defines configuration or the Expand stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.expand.expand#

Bases: BaseStage

Properties for the Expand stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field command: Command | None = Command.compress#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxExpand.svg'#
label: ClassVar[str] = 'Expand'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxExpand'#

This module defines configuration or the External Filter stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.external_filter.external_filter#

Bases: BaseStage

Properties for the External Filter stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field args: str | None = None#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field command: str = None#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_seq#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxExternalFilter.svg'#
label: ClassVar[str] = 'External Filter'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxExternalFilter'#

This module defines configuration or the External Source stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.external_source.external_source#

Bases: BaseStage

Properties for the External Source stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field actual_field_length: int | None = None (alias 'actual_length')#
field allow_all_zeros: AllowAllZeros | None = AllowAllZeros.nofix_zero#
field allow_per_column_mapping: AllowPerColumnMapping | None = AllowPerColumnMapping.false (alias 'allow_column_mapping')#
field allow_signed_import: AllowSignedImport | None = AllowSignedImport.allow_signed_import#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_order: ByteOrder | None = ByteOrder.native_endian#
field c_format: str | None = ''#
field character_set: CharacterSet | None = CharacterSet.ascii (alias 'charset')#
field check_intact: CheckIntact | None = CheckIntact.check_intact#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field data_format: DataFormat | None = DataFormat.text#
field date_format: str | None = ''#
field date_options: DateOption | None = DateOption.none (alias 'dateOption')#
field days_since: str | None = ''#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_options: DecimalOption | None = [] (alias 'decimalOption')#
field decimal_packed: DecimalPacked | None = DecimalPacked.packed#
field decimal_packed_check: DecimalPackedCheck | None = DecimalPackedCheck.check#
field decimal_packed_sign_position: DecimalPackedSignPosition | None = DecimalPackedSignPosition.trailing#
field decimal_packed_signed: DecimalPackedSigned | None = DecimalPackedSigned.signed#
field decimal_sep_value: str | None = ''#
field decimal_separator: DecimalSeparator | None = DecimalSeparator.period#
field delim_value: str | None = ''#
field delimiter: Delimiter | None = Delimiter.comma (alias 'delim')#
field delimiter_string: str | None = '' (alias 'delim_string')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field export_ebcdic_as_ascii: ExportEbcdicAsAscii | None = ExportEbcdicAsAscii.export_ebcdic_as_ascii#
field field_defaults_options: FieldOption | None = FieldOption.delimiter (alias 'fieldOption')#
field field_max_width: int | None = None (alias 'max_width')#
field field_width: int | None = None (alias 'width')#
field fill_char: FillChar | None = FillChar.null (alias 'fill')#
field fill_char_value: str | None = ''#
field final_delim_value: str | None = ''#
field final_delimiter: FinalDelimiter | None = FinalDelimiter.end (alias 'final_delim')#
field final_delimiter_string: str | None = '' (alias 'final_delim_string')#
field flow_dirty: str | None = 'false'#
field general_options: GeneralOption | None = [] (alias 'generalOption')#
field hide: bool | None = False#
field import_ascii_as_ebcdic: ImportAsciiAsEbcdic | None = ImportAsciiAsEbcdic.import_ascii_as_ebcdic#
field in_format: str | None = ''#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field intact: str | None = ''#
field is_julian: IsJulian | None = IsJulian.julian#
field is_midnight_seconds: IsMidnightSeconds | None = IsMidnightSeconds.midnight_seconds#
field keep_file_partitions: KeepPartitions | None = KeepPartitions.false (alias 'keepPartitions')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field map_name: MapName | None = MapName.UTF_8 (alias 'nls_map_name')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field null_field_length: int | None = None (alias 'null_length')#
field null_field_sep_value: str | None = ''#
field null_field_value: str | None = 'NULL' (alias 'null_field')#
field null_field_value_separator: NullFieldValueSeparator | None = NullFieldValueSeparator.comma (alias 'null_field_sep')#
field numeric_options: NumericOption | None = [] (alias 'numericOption')#
field out_format: str | None = ''#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field pad_char: PadChar | None = PadChar.false_ (alias 'padchar')#
field padchar_value: str | None = ''#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field precision: int | None = None#
field prefix_bytes: PrefixBytes | None = PrefixBytes.one (alias 'prefix')#
field preserve_partitioning: PreservePartitioning | None = None (alias 'preserve')#
field print_field: PrintField | None = PrintField.print_field#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote: Quote | None = Quote.double#
field quote_value: str | None = ''#
field record_delim_value: str | None = ''#
field record_delimiter: RecordDelimiter | None = RecordDelimiter.newline (alias 'record_delim')#
field record_delimiter_string: str | None = '' (alias 'record_delim_string')#
field record_len_value: int | None = 0#
field record_length: RecordLength | None = RecordLength.fixed#
field record_level_options: RecLevelOption | None = [] (alias 'recLevelOption')#
field record_prefix: RecordPrefix | None = RecordPrefix.one#
field record_type: RecordType | None = RecordType.type_implicit (alias 'record_format')#
field reject_mode: RejectMode | None = RejectMode.cont (alias 'rejects')#
field reject_reason_column: str | None = None (alias 'rejectReasonField')#
field rounding: Rounding | None = Rounding.trunc_zero (alias 'round')#
field row_number_column: str | None = None (alias 'recordNumberField')#
field runtime_column_propagation: bool | None = None#
field scale: int | None = None#
field schema_file: str | None = None (alias 'schemafile')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field source_method: SourceMethod | None = SourceMethod.program (alias 'selection')#
field source_name_column: str | None = None (alias 'sourceNameField')#
field source_program: list | None = [] (alias 'source')#
field source_programs_file: list | None = [] (alias 'sourcelist')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field string_options: StringOption | None = [] (alias 'stringOption')#
field strip_bom: StripBom | None = StripBom.false (alias 'stripbom')#
field time_format: str | None = ''#
field time_options: TimeOption | None = TimeOption.none (alias 'timeOption')#
field timestamp_format: str | None = ''#
field timestamp_options: TimestampOption | None = TimestampOption.none (alias 'timestampOption')#
field unique: bool | None = None (alias 'part_unique')#
field vector_prefix: VectorPrefix | None = VectorPrefix.one#
field whether_check_intact: bool | None = False (alias 'check_intact_flag')#
field whether_specify_null_field_value_separator: bool | None = False (alias 'null_field_sep_flag')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxExternalSource.svg'#
label: ClassVar[str] = 'External Source'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'PxExternalSource'#

This module defines configuration or the External Target stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.external_target.external_target#

Bases: BaseStage

Properties for the External Target stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field actual_length: int | None = None#
field allow_all_zeros: AllowAllZeros | None = AllowAllZeros.nofix_zero#
field allow_column_mapping: AllowColumnMapping | None = AllowColumnMapping.false#
field allow_signed_import: AllowSignedImport | None = AllowSignedImport.allow_signed_import#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field byte_order: ByteOrder | None = ByteOrder.native_endian#
field c_format: str | None = ''#
field charset: Charset | None = Charset.ascii#
field check_intact: CheckIntact | None = CheckIntact.check_intact#
field check_intact_flag: bool | None = False#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field data_format: DataFormat | None = DataFormat.text#
field date_format: str | None = ''#
field date_option: DateOption | None = DateOption.none (alias 'dateOption')#
field days_since: str | None = ''#
field decimal_options: DecimalOption | None = [] (alias 'decimalOption')#
field decimal_packed: DecimalPacked | None = DecimalPacked.packed#
field decimal_packed_check: DecimalPackedCheck | None = DecimalPackedCheck.check#
field decimal_packed_sign_position: DecimalPackedSignPosition | None = DecimalPackedSignPosition.trailing#
field decimal_packed_signed: DecimalPackedSigned | None = DecimalPackedSigned.signed#
field decimal_sep_value: str | None = ''#
field decimal_separator: DecimalSeparator | None = DecimalSeparator.period#
field delim: Delim | None = Delim.comma#
field delim_string: str | None = ''#
field delim_value: str | None = ''#
field destination: list | None = []#
field destinationlist: list | None = []#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field export_ebcdic_as_ascii: ExportEbcdicAsAscii | None = ExportEbcdicAsAscii.export_ebcdic_as_ascii#
field field_option: FieldOption | None = FieldOption.delimiter (alias 'fieldOption')#
field fill: Fill | None = Fill.null#
field fill_char_value: str | None = ''#
field final_delim: FinalDelim | None = FinalDelim.end#
field final_delim_string: str | None = ''#
field final_delim_value: str | None = ''#
field flow_dirty: str | None = 'false'#
field general_option: GeneralOption | None = [] (alias 'generalOption')#
field hide: bool | None = False#
field import_ascii_as_ebcdic: ImportAsciiAsEbcdic | None = ImportAsciiAsEbcdic.import_ascii_as_ebcdic#
field in_format: str | None = ''#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field intact: str | None = ''#
field is_julian: IsJulian | None = IsJulian.julian#
field is_midnight_seconds: IsMidnightSeconds | None = IsMidnightSeconds.midnight_seconds#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field max_width: int | None = None#
field nls_map_name: NlsMapName | None = NlsMapName.UTF_8#
field null_field: str | None = 'NULL'#
field null_field_sep: NullFieldSep | None = NullFieldSep.comma#
field null_field_sep_flag: bool | None = False#
field null_field_sep_value: str | None = ''#
field null_length: int | None = None#
field numeric_option: NumericOption | None = [] (alias 'numericOption')#
field out_format: str | None = ''#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field padchar: Padchar | None = Padchar.false_#
field padchar_value: str | None = ''#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field precision: int | None = None#
field prefix: Prefix | None = Prefix.one#
field preserve: Preserve | None = Preserve.default_propagate#
field print_field: PrintField | None = PrintField.print_field#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field quote: Quote | None = Quote.double#
field quote_value: str | None = ''#
field rec_level_option: RecLevelOption | None = [] (alias 'recLevelOption')#
field record_delim: RecordDelim | None = RecordDelim.newline#
field record_delim_string: str | None = ''#
field record_delim_value: str | None = ''#
field record_format: RecordFormat | None = RecordFormat.type_implicit#
field record_len_value: int | None = 0#
field record_length: RecordLength | None = RecordLength.fixed#
field record_prefix: RecordPrefix | None = RecordPrefix.one#
field rejects: Rejects | None = Rejects.cont#
field round: Round | None = Round.trunc_zero#
field runtime_column_propagation: bool | None = None#
field scale: int | None = None#
field schemafile: str | None = None#
field selection: Selection | None = Selection.program#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field string_option: StringOption | None = [] (alias 'stringOption')#
field time_format: str | None = ''#
field time_option: TimeOption | None = TimeOption.none (alias 'timeOption')#
field timestamp_format: str | None = ''#
field timestamp_option: TimestampOption | None = TimestampOption.none (alias 'timestampOption')#
field vector_prefix: VectorPrefix | None = VectorPrefix.one#
field width: int | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxExternalTarget.svg'#
label: ClassVar[str] = 'External Target'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'PxExternalTarget'#

This module defines configuration or the File set stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.fileset.fileset#

Bases: BaseStage

Properties for the File set stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field actual_field_length: int | None = None (alias 'actual_length')#
field allow_all_zeros: AllowAllZeros | None = AllowAllZeros.nofix_zero#
field allow_per_column_mapping: AllowPerColumnMapping | None = AllowPerColumnMapping.false (alias 'allow_column_mapping')#
field allow_signed_import: AllowSignedImport | None = AllowSignedImport.allow_signed_import#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field byte_order: ByteOrder | None = ByteOrder.native_endian#
field c_format: str | None = ''#
field character_set: CharacterSet | None = CharacterSet.ascii (alias 'charset')#
field check_intact: CheckIntact | None = CheckIntact.check_intact#
field cleanup_on_failure: CleanupOnFailure | None = CleanupOnFailure.false (alias 'nocleanup')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field create_data_asset: bool | None = False (alias 'registerDataAsset')#
field data_asset_name: str = None (alias 'dataAssetName')#
field data_format: DataFormat | None = DataFormat.text#
field date_format: str | None = ''#
field date_options: DateOption | None = DateOption.none (alias 'dateOption')#
field days_since: str | None = ''#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field decimal_options: DecimalOption | None = [] (alias 'decimalOption')#
field decimal_packed: DecimalPacked | None = DecimalPacked.packed#
field decimal_packed_check: DecimalPackedCheck | None = DecimalPackedCheck.check#
field decimal_packed_sign_position: DecimalPackedSignPosition | None = DecimalPackedSignPosition.trailing#
field decimal_packed_signed: DecimalPackedSigned | None = DecimalPackedSigned.signed#
field decimal_sep_value: str | None = ''#
field decimal_separator: DecimalSeparator | None = DecimalSeparator.period#
field delim_value: str | None = ''#
field delimiter: Delimiter | None = Delimiter.comma (alias 'delim')#
field delimiter_string: str | None = '' (alias 'delim_string')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field diskpool: str | None = None#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field export_ebcdic_as_ascii: ExportEbcdicAsAscii | None = ExportEbcdicAsAscii.export_ebcdic_as_ascii#
field field_defaults_options: FieldOption | None = FieldOption.delimiter (alias 'fieldOption')#
field field_max_width: int | None = None (alias 'max_width')#
field field_width: int | None = None (alias 'width')#
field file_name_column: str | None = None (alias 'sourceNameField')#
field file_prefix: str | None = None (alias 'filePrefix')#
field file_set: str = None (alias 'fileset')#
field file_set_schema_policy: OmitSchemaWriteSchema | None = OmitSchemaWriteSchema.writeSchema (alias 'omitSchema-writeSchema')#
field file_set_update_policy: CreateReplaceDiscardRecordsDiscardSchemaAndRecords | None = CreateReplaceDiscardRecordsDiscardSchemaAndRecords.replace (alias 'create-replace-discard_records-discard_schema_and_records')#
field file_suffix: str | None = None (alias 'suffix')#
field fill_char: FillChar | None = FillChar.null (alias 'fill')#
field fill_char_value: str | None = ''#
field filter: str | None = None#
field final_delim_value: str | None = ''#
field final_delimiter: FinalDelimiter | None = FinalDelimiter.end (alias 'final_delim')#
field final_delimiter_string: str | None = '' (alias 'final_delim_string')#
field flow_dirty: str | None = 'false'#
field general_options: GeneralOption | None = [] (alias 'generalOption')#
field hide: bool | None = False#
field import_ascii_as_ebcdic: ImportAsciiAsEbcdic | None = ImportAsciiAsEbcdic.import_ascii_as_ebcdic#
field in_format: str | None = ''#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field intact: str | None = ''#
field is_julian: IsJulian | None = IsJulian.julian#
field is_midnight_seconds: IsMidnightSeconds | None = IsMidnightSeconds.midnight_seconds#
field keep_file_partitions: KeepPartitions | None = KeepPartitions.false (alias 'keepPartitions')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field map_name: MapName | None = MapName.UTF_8 (alias 'nls_map_name')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_file_size: str = None (alias 'maxFileSize')#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field null_field_length: int | None = None (alias 'null_length')#
field null_field_sep_value: str | None = ''#
field null_field_value: str | None = 'NULL' (alias 'null_field')#
field null_field_value_separator: NullFieldValueSeparator | None = NullFieldValueSeparator.comma (alias 'null_field_sep')#
field numeric_options: NumericOption | None = [] (alias 'numericOption')#
field out_format: str | None = ''#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field pad_char: PadChar | None = PadChar.false_ (alias 'padchar')#
field padchar_value: str | None = ''#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field precision: int | None = None#
field prefix_bytes: PrefixBytes | None = PrefixBytes.one (alias 'prefix')#
field preserve_partitioning: PreservePartitioning | None = PreservePartitioning.default_clear (alias 'preserve')#
field print_field: PrintField | None = PrintField.print_field#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field quote: Quote | None = Quote.double#
field quote_value: str | None = ''#
field record_delim_value: str | None = ''#
field record_delimiter: RecordDelimiter | None = RecordDelimiter.newline (alias 'record_delim')#
field record_delimiter_string: str | None = '' (alias 'record_delim_string')#
field record_len_value: int | None = 0#
field record_length: RecordLength | None = RecordLength.fixed#
field record_level_options: RecLevelOption | None = [] (alias 'recLevelOption')#
field record_prefix: RecordPrefix | None = RecordPrefix.one#
field record_type: RecordType | None = RecordType.type_implicit (alias 'record_format')#
field reject_mode: RejectMode | None = RejectMode.cont (alias 'rejects')#
field reject_reason_column: str | None = None (alias 'rejectReasonField')#
field report_progress: ReportProgress | None = ReportProgress.yes (alias 'reportProgress')#
field rounding: Rounding | None = Rounding.trunc_zero (alias 'round')#
field row_number_column: str | None = None (alias 'recordNumberField')#
field runtime_column_propagation: bool | None = None#
field scale: int | None = None#
field schema_file: str | None = None (alias 'schemafile')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field single_file_per_partition: SingleFilePerPartition | None = SingleFilePerPartition.false (alias 'singleFilePerPartition')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field string_options: StringOption | None = [] (alias 'stringOption')#
field strip_bom: StripBom | None = StripBom.false (alias 'stripbom')#
field time_format: str | None = ''#
field time_options: TimeOption | None = TimeOption.none (alias 'timeOption')#
field timestamp_format: str | None = ''#
field timestamp_options: TimestampOption | None = TimestampOption.none (alias 'timestampOption')#
field unique: bool | None = None (alias 'part_unique')#
field use_schema_defined_in_file_set: UseSchemaDefinedInFileSet | None = UseSchemaDefinedInFileSet.true (alias 'schemafileset')#
field vector_prefix: VectorPrefix | None = VectorPrefix.one#
field whether_check_intact: bool | None = False (alias 'check_intact_flag')#
field whether_specify_null_field_value_separator: bool | None = False (alias 'null_field_sep_flag')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxFileSet.svg'#
label: ClassVar[str] = 'File set'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'PxFileSet'#

This module defines configuration or the Filter stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.filter.filter#

Bases: BaseStage

Properties for the Filter stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field first: First | None = First.false#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field nulls: Nulls | None = Nulls.first#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field reject: Reject | None = Reject.false#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field where_properties: list | None = [] (alias 'whereProperties')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxFilter.svg'#
label: ClassVar[str] = 'Filter'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxFilter'#

This module defines configuration or the Funnel stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.funnel.funnel#

Bases: BaseStage

Properties for the Funnel stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field operator: Operator | None = Operator.funnel#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxFunnel.svg'#
label: ClassVar[str] = 'Funnel'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxFunnel'#

This module defines configuration or the Generic stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.generic.generic#

Bases: BaseStage

Properties for the Generic stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field name_properties: list | None = [] (alias 'nameProperties')#
field operator: str = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxGeneric.svg'#
label: ClassVar[str] = 'Generic'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxGeneric'#

This module defines configuration or the Head stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.head.head#

Bases: BaseStage

Properties for the Head stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field all: All | None = All.false#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field nrecs: int | None = 10#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part: list | None = []#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field period: int | None = None#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field selection: Selection | None = Selection.true#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field skip: int | None = None#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxHead.svg'#
label: ClassVar[str] = 'Head'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxHead'#

This module defines configuration or the Hierarchical Data stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.hierarchical_data.hierarchical_data#

Bases: BaseStage

Properties for the Hierarchical Data stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field classpath: str | None = '' (alias 'Classpath')#
field coll_type: CollType | None = CollType.auto#
field column_properties: str | None = ''#
field combinability: Combinability | None = Combinability.auto#
field debug_test_data: str | None = None#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field e2_assembly: str | None = None#
field e2_assembly_errors: int = 1#
field e2_assembly_file: str | None = None#
field e2_assembly_pipeline: str | None = ''#
field enable_disk_based: bool | None = True#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_logging: bool | None = True#
field enable_output_trigger_columns: bool | None = False#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field enable_split_data_into_batches: bool | None = False#
field execmode: Execmode | None = Execmode.default_seq#
field flow_dirty: str | None = 'false'#
field heap_size: int | None = 256#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field is_x_m_l_stage: bool | None = True (alias 'isXMLStage')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field last_modified: str | None = None#
field limit_output_rows: bool | None = False#
field log_level: LogLevel = LogLevel.warning#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field max_num_output_rows: int = None#
field maximum_stack_size: int | None = 256#
field num_assembly_threads: int = 1#
field open_sub_canvas: bool | None = False (alias 'openSubCanvas')#
field other_options: str | None = '' (alias 'OtherOptions')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field output_row_trigger_column: str | None = None#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field params: str | None = None#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field security_info: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field split_batch_key: str = None#
field split_batch_key_value_order: SplitBatchKeyValueOrder = SplitBatchKeyValueOrder.ascending#
field stage_description: list | None = '' (alias 'stageDescription')#
field stage_type: str = 'XMLStage'#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/XMLStagePX.svg'#
label: ClassVar[str] = 'Hierarchical Data'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'XMLStagePX'#

This module defines configuration or the Investigate stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.investigate.investigate#

Bases: BaseStage

Properties for the Investigate stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field alpha: bool | None = False#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: str | None = ''#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: str | None = ''#
field char_concat: str | None = '' (alias 'charConcat')#
field char_discrete: bool | None = False (alias 'charDiscrete')#
field choose_char_concate: bool | None = False (alias 'chooseCharConcate')#
field coll_type: CollType | None = CollType.auto#
field columns_invt: list | None = [] (alias 'columnsInvt')#
field combinability: Combinability | None = Combinability.auto#
field compress_unknown: bool | None = False (alias 'compressUnknown')#
field db2_instance_select: Db2InstanceSelect | None = Db2InstanceSelect.use_db2InstanceEnv (alias 'db2InstanceSelect')#
field db2_name_select: Db2NameSelect | None = Db2NameSelect.use_db2NameEnv (alias 'db2NameSelect')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: str | None = ''#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field field: list = []#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.Select_a_column (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field locale: str | None = None#
field mask_properties: list | None = [] (alias 'maskProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: str | None = ''#
field min_freq: int = 0 (alias 'minFreq')#
field numeric: bool | None = False#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_dbname: str | None = 'eg: SAMPLE'#
field part_server: str | None = 'eg: DB2INST'#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field pattern: list | None = []#
field pattern_rpt: bool | None = False (alias 'patternRpt')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field pttn_rpt: int | None = 0 (alias 'pttnRpt')#
field q_s_x_m_l: str = None (alias 'QSXML')#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: str | None = ''#
field ruleset: str | None = None#
field runtime_column_propagation: bool | None = None#
field schema_name_check: bool | None = None (alias 'schemaNameCheck')#
field seplist: str = 'XX !?%$,.;:()-/#&'#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = 'If sort keys are added, a sort will be performed.' (alias 'sortInstructions')#
field special: bool | None = False#
field stage_description: list | None = '' (alias 'stageDescription')#
field stasticsoptions: str | None = None (alias 'Stasticsoptions')#
field striplist: str = 'XX !?%$,.;:()-/#&'#
field token_edit: TokenEdit | None = TokenEdit.abbrev (alias 'tokenEdit')#
field tokn_rpt: bool | None = False (alias 'toknRpt')#
field trace_properties: list | None = [] (alias 'traceProperties')#
field word: Word | None = Word.true#
field workdir: str = ''#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/Investigate.svg'#
label: ClassVar[str] = 'Investigate'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'Investigate'#

This module defines configuration or the Java Integration stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.java_integration.java_integration#

Bases: BaseStage

Properties for the Java Integration stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field abort_after_percent: int | None = 0#
field asset_name: str | None = None#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field class_properties: list | None = []#
field classpath: str = None#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field custom_properties: str | None = None#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field function_name: str = None#
field heap_size: int | None = 256#
field hide: bool | None = False#
field in_column_mapping: str | None = None#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field is_reject_output: bool | None = False#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field lookup_type: LookupType | None = LookupType.empty#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field other_options: str | None = None#
field out_column_mapping: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field reject_condition_properties_option: RejectConditionPropertiesOption | None = None (alias 'rejectConditionProperties_option')#
field reject_condition_row_rejected: bool | None = False#
field reject_data_element_errorcode: bool | None = False#
field reject_data_element_errortext: bool | None = False#
field reject_number: int | None = None#
field reject_rows_properties_set: RejectRowsPropertiesSet | None = [] (alias 'rejectRowsPropertiesSet')#
field reject_threshold: int | None = None#
field reject_uses: RejectUses | None = RejectUses.Rows#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field user_class_name: str = None#
field user_defined_function: bool | None = False#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/JavaStagePX.svg'#
label: ClassVar[str] = 'Java Integration'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'JavaStagePX'#

This module defines configuration or the Join stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.join.join#

Bases: BaseStage

Properties for the Join stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field hide_case_sensitive_column: bool | None = None (alias 'hideCaseSensitiveColumn')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field operator: Operator | None = Operator.innerjoin#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxJoin.svg'#
label: ClassVar[str] = 'Join'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxJoin'#

This module defines configuration or the Lookup stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.lookup.lookup#

Bases: BaseStage

Properties for the Lookup stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field allow_dups: AllowDups | None = AllowDups.false#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field condition: str | None = None#
field condition_not_met: ConditionNotMet | None = ConditionNotMet.fail (alias 'conditionNotMet')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field diskpool: str | None = None#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field globalconstraints_items: GlobalconstraintsItems | None = GlobalconstraintsItems.custom (alias 'globalconstraintsItems')#
field has_key: bool | None = False (alias 'hasKey')#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field lookup_derivation: list[LookupDerivation] | None = [] (alias 'lookupDerivation')#
field lookup_fail: LookupFail | None = LookupFail.fail (alias 'lookupFail')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field primary_key_columns_items: list | None = [] (alias 'primaryKeyColumnsItems')#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field save: str | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxLookup.svg'#
label: ClassVar[str] = 'Lookup'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxLookup'#

This module defines configuration or the Lookup file set stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.lookupfileset.lookupfileset#

Bases: BaseStage

Properties for the Lookup file set stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field allow_dups: AllowDups | None = AllowDups.false#
field buf_free_run_ronly: int | None = 50#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field buffer_free_run_percent: int | None = 50 (alias 'buf_free_run')#
field buffering_mode: BufferingMode | None = BufferingMode.default (alias 'buf_mode')#
field collecting: Collecting | None = Collecting.auto (alias 'coll_type')#
field column_metadata_change_propagation: bool | None = None (alias 'auto_column_propagation')#
field combinability_mode: CombinabilityMode | None = CombinabilityMode.auto (alias 'combinability')#
field data_asset_name: str = None (alias 'dataAssetName')#
field db2_database_name: str | None = None (alias 'part_client_dbname')#
field db2_instance_name: str | None = None (alias 'part_client_instance')#
field db2_source_connection_required: str | None = '' (alias 'part_dbconnection')#
field db2_table_name: str | None = None (alias 'part_table')#
field disk_write_inc_ronly: int | None = 1048576#
field disk_write_increment_bytes: int | None = 1048576 (alias 'disk_write_inc')#
field diskpool: str | None = None#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execution_mode: ExecutionMode | None = ExecutionMode.default_par (alias 'execmode')#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_high: str | None = '' (alias 'keyHigh')#
field key_high_ci_cs: KeyHighCiCs | None = KeyHighCiCs.cs (alias 'keyHigh-ci-cs')#
field key_high_keep: KeyHighKeep | None = KeyHighKeep.false (alias 'keyHigh-keep')#
field key_low: str | None = '' (alias 'keyLow')#
field key_low_ci_cs: KeyLowCiCs | None = KeyLowCiCs.cs (alias 'keyLow-ci-cs')#
field key_low_keep: KeyLowKeep | None = KeyLowKeep.false (alias 'keyLow-keep')#
field key_ordered: str | None = '' (alias 'keyOrdered')#
field key_ordered_ci_cs: KeyOrderedCiCs | None = KeyOrderedCiCs.cs (alias 'keyOrdered-ci-cs')#
field key_ordered_keep: KeyOrderedKeep | None = KeyOrderedKeep.false (alias 'keyOrdered-keep')#
field key_properties: list | None = [] (alias 'keyProperties')#
field lookup_file_set: str = None (alias 'fileset')#
field max_mem_buf_size_ronly: int | None = 3145728#
field maximum_memory_buffer_size_bytes: int | None = 3145728 (alias 'max_mem_buf_size')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field partition_type: PartitionType | None = PartitionType.auto (alias 'part_type')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field queue_upper_bound_size_bytes: int | None = 0 (alias 'queue_upper_size')#
field queue_upper_size_ronly: int | None = 0#
field range: Range | None = Range.false#
field register_data_asset: bool | None = False (alias 'registerDataAsset')#
field runtime_column_propagation: bool | None = None#
field save: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sorting_key: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field stable: bool | None = None (alias 'part_stable')#
field stage_description: list | None = '' (alias 'stageDescription')#
field table: str = 'table'#
field unique: bool | None = None (alias 'part_unique')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxLookupFileSet.svg'#
label: ClassVar[str] = 'Lookup file set'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'PxLookupFileSet'#

This module defines configuration or the Make Subrecord stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.make_subrecord.make_subrecord#

Bases: BaseStage

Properties for the Make Subrecord stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field name: list = []#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field schema_name_check: bool | None = None (alias 'schemaNameCheck')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field subrecname: str = None#
field variable: Variable | None = Variable.false#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxMakeSubRec.svg'#
label: ClassVar[str] = 'Make Subrecord'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxMakeSubRec'#

This module defines configuration or the Make Vector stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.make_vector.make_vector#

Bases: BaseStage

Properties for the Make Vector stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field name: str = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxMakeVect.svg'#
label: ClassVar[str] = 'Make Vector'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxMakeVect'#

This module defines configuration or the Match Frequency stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.match_frequency.match_frequency#

Bases: BaseStage

Properties for the Match Frequency stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field abtype: Abtype | None = Abtype.Data#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: str | None = ''#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: str | None = ''#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field db2_instance_select: Db2InstanceSelect | None = Db2InstanceSelect.use_db2InstanceEnv (alias 'db2InstanceSelect')#
field db2_name_select: Db2NameSelect | None = Db2NameSelect.use_db2NameEnv (alias 'db2NameSelect')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: str | None = ''#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int = 1#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.Select_a_column (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field locale: str | None = None#
field match_type: str | None = 'Undup' (alias 'matchType')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: str | None = ''#
field maxfreq: int = 1#
field not_use_specfile: bool = False (alias 'notUseSpecfile')#
field output_count: int = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_dbname: str | None = 'eg: SAMPLE'#
field part_server: str | None = 'eg: DB2INST'#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field pgmtype: Pgmtype = Pgmtype.dependent#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: str | None = ''#
field runtime_column_propagation: bool | None = None#
field schema_name_check: bool | None = None (alias 'schemaNameCheck')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = 'If sort keys are added, a sort will be performed.' (alias 'sortInstructions')#
field specfile: str | None = False#
field stage_description: list | None = '' (alias 'stageDescription')#
field trace_properties: list | None = [] (alias 'traceProperties')#
field workdir: str = ''#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/MatchFrequency.svg'#
label: ClassVar[str] = 'Match Frequency'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'MatchFrequency'#

This module defines configuration or the Merge stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.merge.merge#

Bases: BaseStage

Properties for the Merge stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field drop_bad_masters_keep_bad_masters: DropBadMastersKeepBadMasters | None = DropBadMastersKeepBadMasters.keepBadMasters (alias 'dropBadMasters-keepBadMasters')#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field nowarn_bad_masters_warn_bad_masters: NowarnBadMastersWarnBadMasters | None = NowarnBadMastersWarnBadMasters.warnBadMasters (alias 'nowarnBadMasters-warnBadMasters')#
field nowarn_bad_updates_warn_bad_updates: NowarnBadUpdatesWarnBadUpdates | None = NowarnBadUpdatesWarnBadUpdates.warnBadUpdates (alias 'nowarnBadUpdates-warnBadUpdates')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxMerge.svg'#
label: ClassVar[str] = 'Merge'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxMerge'#

This module defines configuration or the Modify stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.modify.modify#

Bases: BaseStage

Properties for the Modify stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field endingquote: str | None = "'"#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field modifyspec_properties: list | None = [] (alias 'modifyspecProperties')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxModify.svg'#
label: ClassVar[str] = 'Modify'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxModify'#

This module defines configuration or the One-source Match stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.one_source_match.one_source_match#

Bases: BaseStage

Properties for the One-source Match stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run: int | None = 50#
field buf_free_run_ronly: str | None = ''#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: str | None = ''#
field clro: str | None = None#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field cutoffs_properties: list | None = [] (alias 'cutoffsProperties')#
field db2_instance_select: Db2InstanceSelect | None = Db2InstanceSelect.use_db2InstanceEnv (alias 'db2InstanceSelect')#
field db2_name_select: Db2NameSelect | None = Db2NameSelect.use_db2NameEnv (alias 'db2NameSelect')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: str | None = ''#
field dupa: str | None = None#
field enable_override: bool | None = False (alias 'enableOverride')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field group: str | None = None#
field hide: bool | None = False#
field input_count: int = 1#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.Select_a_column (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field locale: str | None = None#
field match_type: str | None = 'Undup' (alias 'matchType')#
field mato: str | None = None#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: str | None = ''#
field output_count: int = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field override: str | None = None#
field part_dbname: str | None = 'eg: SAMPLE'#
field part_server: str | None = 'eg: DB2INST'#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field pgmtype: Pgmtype | None = Pgmtype.dependent#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: str | None = ''#
field resa: str | None = None#
field resx: str | None = None#
field reuse_rec_id: str | None = None (alias 'reuseRecId')#
field runtime_column_propagation: bool | None = None#
field schema_name_check: bool | None = None (alias 'schemaNameCheck')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = 'If sort keys are added, a sort will be performed.' (alias 'sortInstructions')#
field spec_cutoff: str | None = '' (alias 'specCutoff')#
field specfile: str = None#
field stage_description: list | None = '' (alias 'stageDescription')#
field stats: str | None = None#
field trace_properties: list | None = [] (alias 'traceProperties')#
field workdir: str = ''#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/UnduplicateMatch.svg'#
label: ClassVar[str] = 'One-source Match'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'UnduplicateMatch'#

This module defines configuration or the Peek stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.peek.peek#

Bases: BaseStage

Properties for the Peek stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field all: All | None = All.false#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field columns: Columns | None = Columns.true#
field combinability: Combinability | None = Combinability.auto#
field dataset: Dataset | None = Dataset.false#
field delim: str | None = None#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field field: list = []#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field name: Name | None = Name.true#
field nrecs: int | None = 10#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part: list = []#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field period: int | None = None#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field selection: Selection | None = Selection.true#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field skip: int | None = None#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxPeek.svg'#
label: ClassVar[str] = 'Peek'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxPeek'#

This module defines configuration or the Pivot Enterprise stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.pivot.pivot#

Bases: BaseStage

Properties for the Pivot Enterprise stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field array_size: int | None = 1#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field pivot_custom_input_o_s_h: str | None = '' (alias 'PivotCustomInputOSH')#
field pivot_custom_o_s_h: str | None = '' (alias 'PivotCustomOSH')#
field pivot_index: bool | None = False (alias 'pivotIndex')#
field pivot_properties: list | None = [] (alias 'pivotProperties')#
field pivot_type: PivotType | None = PivotType.pivot#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field vertical_pivot_custom_o_s_h: str | None = '' (alias 'VerticalPivotCustomOSH')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxPivot.svg'#
label: ClassVar[str] = 'Pivot Enterprise'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxPivot'#

This module defines configuration or the Promote Subrecord stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.promote_subrecord.promote_subrecord#

Bases: BaseStage

Properties for the Promote Subrecord stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field schema_name_check: bool | None = None (alias 'schemaNameCheck')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field subrecname: str = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxPromoteSubRec.svg'#
label: ClassVar[str] = 'Promote Subrecord'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxPromoteSubRec'#

This module defines configuration or the Remove Duplicates stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.remove_duplicates.remove_duplicates#

Bases: BaseStage

Properties for the Remove Duplicates stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field hide_case_sensitive_column: bool | None = None (alias 'hideCaseSensitiveColumn')#
field hide_ebcdic_column: bool | None = None (alias 'hide_ebcdic_Column')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field keep: Keep | None = Keep.first#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxRemDup.svg'#
label: ClassVar[str] = 'Remove Duplicates'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxRemDup'#

This module defines configuration or the Rest stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.rest.rest#

Bases: BaseStage

Properties for the Rest stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field combinability: Combinability | None = Combinability.auto#
field execmode: Execmode | None = Execmode.default_seq#
field input_count: int | None = 0#
field log_level: LogLevel | None = LogLevel.warning#
field log_reject_error: bool | None = False#
field output_count: int | None = 0#
field parameters: str | None = ' '#
field part_coll: str | None = 'part_type'#
field preserve: Preserve | None = Preserve.default_propagate#
field reject_message_column: RejectMessageColumn | None = RejectMessageColumn.custom#
field requests: list = []#
field runtime_column_propagation: int | None = 1#
field variables: dict = []#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/REST.svg'#
label: ClassVar[str] = 'Rest'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxRest'#

This module defines configuration or the Row Generator stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.row_generator.row_generator#

Bases: BaseStage

Properties for the Row Generator stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_seq#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field records: int = 10#
field runtime_column_propagation: bool | None = None#
field schemafile: str | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxRowGenerator.svg'#
label: ClassVar[str] = 'Row Generator'#
node_type: ClassVar[str] = 'binding'#
op_name: ClassVar[str] = 'PxRowGenerator'#

This module defines configuration or the Sample stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sample.sample#

Bases: BaseStage

Properties for the Sample stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field maxoutputrows: int | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field percent_properties: list | None = [] (alias 'percentProperties')#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field sample: int = None#
field seed: int | None = None#
field selection: Selection | None = Selection.percent#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field totalmaxoutputrows: int | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxSample.svg'#
label: ClassVar[str] = 'Sample'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxSample'#

This module defines configuration or the Slowly Changing Dimension stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.slowly_changing_dimension.slowly_changing_dimension#

Bases: BaseStage

Properties for the Slowly Changing Dimension stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field account_name: str = None#
field advanced_hostname: str = None (alias 'advanced.hostname')#
field advanced_port: int = 50000 (alias 'advanced.port')#
field auto_column_propagation: bool | None = None#
field block_size: int | None = 1#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field client_dbname: str | None = None#
field client_instance: str | None = None#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field database: str = None#
field db2_cat: bool | None = None (alias 'db2Cat')#
field dbname: str | None = None#
field dbtype: Dbtype | None = Dbtype.db2#
field dim_update: list | None = None (alias 'dimUpdate')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field dsn_type: DsnType = DsnType.DB2#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field hostname: str = None#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field keyfile: str | None = None#
field keysourcetype: Keysourcetype | None = Keysourcetype.file#
field lookup_derivation: list | None = [] (alias 'lookupDerivation')#
field lookup_derivation_properties: list | None = [] (alias 'lookupDerivationProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field oracle_db_host: str = None#
field oracle_db_port: str = None#
field oracle_service_name: str = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field password: str | None = None#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field port: int = None#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field retrievekeys: Retrievekeys | None = Retrievekeys.in_blocks_of#
field role: str = None#
field runtime_column_propagation: bool | None = None#
field schema_: str = None (alias 'schema')#
field seedvalue: str | None = None#
field server: str | None = None#
field service_name: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field source_conn: str | None = '' (alias 'sourceConn')#
field stage_description: list | None = '' (alias 'stageDescription')#
field user: str | None = None#
field username: str = None#
field warehouse: str = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxSCD.svg'#
label: ClassVar[str] = 'Slowly Changing Dimension'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxSCD'#

This module defines configuration or the Sort stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.sort.sort#

Bases: BaseStage

Properties for the Sort stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flag_cluster: FlagCluster | None = FlagCluster.false (alias 'flagCluster')#
field flag_key: FlagKey | None = FlagKey.false (alias 'flagKey')#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field memory: int | None = 20#
field operator: Operator | None = Operator.tsort#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_set#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stable: Stable | None = Stable.stable#
field stage_description: list | None = '' (alias 'stageDescription')#
field stats: Stats | None = Stats.false#
field unique: Unique | None = Unique.false#
field workspace: str | None = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxSort.svg'#
label: ClassVar[str] = 'Sort'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxSort'#

This module defines configuration or the Split Subrecord stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.split_subrecord.split_subrecord#

Bases: BaseStage

Properties for the Split Subrecord stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field subrecname: str = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxSplitSubRec.svg'#
label: ClassVar[str] = 'Split Subrecord'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxSplitSubRec'#

This module defines configuration or the Split Vector stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.split_vector.split_vector#

Bases: BaseStage

Properties for the Split Vector stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field name: str = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxSplitVect.svg'#
label: ClassVar[str] = 'Split Vector'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxSplitVect'#

This module defines configuration or the Standardize stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.standardize.standardize#

Bases: BaseStage

Properties for the Standardize stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run: int | None = 50#
field buf_free_run_ronly: str | None = ''#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: str | None = ''#
field case: Case | None = Case.UPPERALL#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field db2_instance_select: Db2InstanceSelect | None = Db2InstanceSelect.use_db2InstanceEnv (alias 'db2InstanceSelect')#
field db2_name_select: Db2NameSelect | None = Db2NameSelect.use_db2NameEnv (alias 'db2NameSelect')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: str | None = ''#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.Select_a_column (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field locale: str | None = None#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: str | None = ''#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_dbname: str | None = 'eg: SAMPLE'#
field part_server: str | None = 'eg: DB2INST'#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: str | None = ''#
field reporting_output: bool | None = False (alias 'reportingOutput')#
field ruleset: str | None = ''#
field ruleset_properties: list | None = [] (alias 'rulesetProperties')#
field runtime_column_propagation: bool | None = None#
field schema_name_check: bool | None = None (alias 'schemaNameCheck')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = 'If sort keys are added, a sort will be performed.' (alias 'sortInstructions')#
field stage_description: list | None = '' (alias 'stageDescription')#
field stagecol_properties: list | None = [] (alias 'stagecolProperties')#
field trace_properties: list | None = [] (alias 'traceProperties')#
field workdir: str | None = ''#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/Standardize.svg'#
label: ClassVar[str] = 'Standardize'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'Standardize'#

This module defines configuration or the Surrogate Key Generator stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.surrogate_key_generator.surrogate_key_generator#

Bases: BaseStage

Properties for the Surrogate Key Generator stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field account_name: str = None#
field admin: Admin | None = Admin.create#
field advanced_hostname: str = None (alias 'advanced.hostname')#
field advanced_port: int = 50000 (alias 'advanced.port')#
field as_sequencer: AsSequencer | None = AsSequencer.false (alias 'asSequencer')#
field auto_column_propagation: bool | None = None#
field block_size: int | None = 2#
field block_size_properties: dict | None = None (alias 'blockSizeProperties')#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field client_dbname: str | None = None#
field client_instance: str | None = None#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field database: str = None#
field db2_cat: bool | None = None (alias 'db2Cat')#
field dbblocktype: Dbblocktype | None = Dbblocktype.adaptivesize#
field dbtype: Dbtype | None = Dbtype.db2#
field dbtype_properties: dict | None = None (alias 'dbtypeProperties')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field dsn_type: DsnType = DsnType.DB2#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field fileblocktype: Fileblocktype | None = Fileblocktype.adaptivesize#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field hostname: str = None#
field input_count: int | None = 0#
field input_key: str = None#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field keysourcetype: Keysourcetype = Keysourcetype.file#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field oracle_db_host: str = None#
field oracle_db_port: str = None#
field oracle_service_name: str = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field output_key: str = None#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field password: str = None#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field port: str = None#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field records: int | None = 10#
field role: str = None#
field runtime_column_propagation: bool | None = None#
field schema_: str = None (alias 'schema')#
field service_name: str = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sk_id: str = None#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field source_conn: str | None = '' (alias 'sourceConn')#
field source_name_check: bool | None = None (alias 'sourceNameCheck')#
field stage_description: list | None = '' (alias 'stageDescription')#
field start_value: int | None = 1#
field start_value_plus: int | None = 1 (alias 'start_value +')#
field update_action: UpdateAction | None = UpdateAction.admin_update#
field username: str = None#
field viewonly: Viewonly | None = Viewonly.no#
field warehouse: str = None#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxSurrogateKeyGeneratorN.svg'#
label: ClassVar[str] = 'Surrogate Key Generator'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxSurrogateKeyGeneratorN'#

This module defines configuration or the Survive stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.survive.survive#

Bases: BaseStage

Properties for the Survive stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run: int | None = 50#
field buf_free_run_ronly: str | None = ''#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: str | None = ''#
field cfdefn: str = ''#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field db2_instance_select: Db2InstanceSelect | None = Db2InstanceSelect.use_db2InstanceEnv (alias 'db2InstanceSelect')#
field db2_name_select: Db2NameSelect | None = Db2NameSelect.use_db2NameEnv (alias 'db2NameSelect')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: str | None = ''#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field f: str = 'Survive_1.tx'#
field flow_dirty: str | None = 'false'#
field group_column: str = 'Select a column' (alias 'groupColumn')#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field locale: str | None = None#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: str | None = ''#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_dbname: str | None = 'eg: SAMPLE'#
field part_server: str | None = 'eg: DB2INST'#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field q_s_x_m_l: str = '' (alias 'QSXML')#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: str | None = ''#
field rule_column_properties: list | None = [] (alias 'ruleColumnProperties')#
field runtime_column_propagation: bool | None = None#
field schema_name_check: bool | None = None (alias 'schemaNameCheck')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort: str | None = ''#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field trace_properties: list | None = [] (alias 'traceProperties')#
field workdir: str | None = ''#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/Survive.svg'#
label: ClassVar[str] = 'Survive'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'Survive'#

This module defines configuration or the Switch stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.switch.switch#

Bases: BaseStage

Properties for the Switch stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field case: list | None = []#
field ci_cs: CiCs | None = CiCs.cs (alias 'ci-cs')#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field discard: int | None = None#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hash_selector: str = 'hashSelector' (alias 'hashSelector')#
field hide: bool | None = False#
field if_not_found: IfNotFound | None = IfNotFound.fail (alias 'ifNotFound')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key: Key = Key.custom#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field selection: Selection | None = Selection.user#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxSwitch.svg'#
label: ClassVar[str] = 'Switch'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxSwitch'#

This module defines configuration or the Tail stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.tail.tail#

Bases: BaseStage

Properties for the Tail stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field nrecs: int | None = 10#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part: list | None = []#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field runtime_column_propagation: bool | None = None#
field selection: Selection | None = Selection.true#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxTail.svg'#
label: ClassVar[str] = 'Tail'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxTail'#

This module defines configuration or the Transformer stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.transformer.transformer#

Bases: BaseStage

Properties for the Transformer stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field abort_on_null: bool | None = None (alias 'AbortOnNull')#
field auto_column_propagation: bool | None = None#
field block_size: int | None = None (alias 'BlockSize')#
field block_size_selected_type: BlockSizeSelectedType | None = BlockSizeSelectedType.systemSelected (alias 'BlockSizeSelectedType')#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field collation_sequence: CollationSequence | None = CollationSequence.OFF#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field initial_value: str | None = None (alias 'InitialValue')#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field loop_variables: list | None = [] (alias 'LoopVariables')#
field loop_while: str | None = None (alias 'LoopWhile')#
field max_loop_iterations: int | None = 0 (alias 'MaxLoopIterations')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field max_reject_log_msg: int | None = 0 (alias 'MaxRejectLogMsg')#
field old_null_handling: bool | None = None (alias 'OldNullHandling')#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field reject: bool | None = None (alias 'Reject')#
field row_limit: int | None = 0 (alias 'RowLimit')#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sk_client_db_name: str | None = None (alias 'SKClientDBName')#
field sk_client_instance: str | None = None (alias 'SKClientInstance')#
field sk_db_name: str | None = None (alias 'SKDBName')#
field sk_db_type: str | None = None (alias 'SKDBType')#
field sk_key_source_type: SKKeySourceType | None = SKKeySourceType.file (alias 'SKKeySourceType')#
field sk_password: str | None = None (alias 'SKPassword')#
field sk_server: str | None = None (alias 'SKServer')#
field sk_user: str | None = None (alias 'SKUser')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field sort_order_preserved: bool | None = True (alias 'SortOrderPreserved')#
field stage_description: list | None = '' (alias 'stageDescription')#
field stage_variables: list | None = [] (alias 'StageVariables')#
field sur_key_state_file: str | None = None (alias 'SurKeyStateFile')#
field transformer_constraint: list | None = [] (alias 'TransformerConstraint')#
field triggers: list | None = [] (alias 'Triggers')#
field value_derivation: list | None = None (alias 'valueDerivation')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/CTransformerStage.svg'#
label: ClassVar[str] = 'Transformer'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'CTransformerStage'#

This module defines configuration or the Two-source Match stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.two_source_match.two_source_match#

Bases: BaseStage

Properties for the Two-source Match stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field buf_free_run: int | None = 50#
field buf_free_run_ronly: str | None = ''#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: str | None = ''#
field clro: str | None = None#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field cutoffs_properties: list | None = [] (alias 'cutoffsProperties')#
field db2_instance_select: Db2InstanceSelect | None = Db2InstanceSelect.use_db2InstanceEnv (alias 'db2InstanceSelect')#
field db2_name_select: Db2NameSelect | None = Db2NameSelect.use_db2NameEnv (alias 'db2NameSelect')#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: str | None = ''#
field dupa: str | None = None#
field dupb: str | None = None#
field enable_override: bool | None = False (alias 'enableOverride')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field freq_a: str = None (alias 'freqA')#
field freq_b: str = None (alias 'freqB')#
field group: str | None = None#
field hide: bool | None = False#
field input_count: int = 1#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.Select_a_column (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field locale: str | None = None#
field match_type: str | None = 'Match' (alias 'matchType')#
field mato: str | None = None#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: str | None = ''#
field output_count: int = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field override: str | None = None#
field part_dbname: str | None = 'eg: SAMPLE'#
field part_server: str | None = 'eg: DB2INST'#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field pgmtype: Pgmtype | None = Pgmtype.many_to_one#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: str | None = ''#
field resa: str | None = None#
field resb: str | None = None#
field reuse_rec_id: str | None = None (alias 'reuseRecId')#
field runtime_column_propagation: bool | None = None#
field schema_name_check: bool | None = None (alias 'schemaNameCheck')#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = 'If sort keys are added, a sort will be performed.' (alias 'sortInstructions')#
field spec_cutoff: str | None = '' (alias 'specCutoff')#
field specfile: str = None#
field stage_description: list | None = '' (alias 'stageDescription')#
field stats: str | None = None#
field trace_properties: list | None = [] (alias 'traceProperties')#
field workdir: str = ''#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/ReferenceMatch.svg'#
label: ClassVar[str] = 'Two-source Match'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'ReferenceMatch'#

This module defines configuration or the Wave Generator stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.wave_generator.wave_generator#

Bases: BaseStage

Properties for the Wave Generator stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field column_name: str = None#
field column_value: str = None#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_par#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field mode: Mode | None = Mode.count#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field record_count: int = None#
field runtime_column_propagation: bool | None = None#
field sequence: Sequence | None = Sequence.before#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxWaveGenerator.svg'#
label: ClassVar[str] = 'Wave Generator'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxWaveGenerator'#

This module defines configuration or the Web Service stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.web_service.web_service#

Bases: BaseStage

Properties for the Web Service stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field allow_column_mapping: bool | None = False#
field authentication_password: str | None = ''#
field authentication_user_name: str | None = ''#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field certificate_passwd: str | None = ''#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field error_handling: ErrorHandling | None = ErrorHandling.fatal#
field execmode: Execmode | None = Execmode.default_seq#
field field_delimiter: str | None = ' '#
field flow_dirty: str | None = 'false'#
field has_reference_output: bool | None = False (alias 'hasReferenceOutput')#
field hide: bool | None = False#
field inherit_from_stage: int | None = 1#
field input_col: list | None = None#
field input_count: int | None = 0#
field input_description: str | None = '0'#
field input_message_table_id: str | None = ''#
field input_name: str | None = ''#
field input_namespace_name: str | None = ''#
field input_namespace_value: str | None = ''#
field input_request: str | None = ''#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field is_authentication_required: bool | None = False#
field is_automatic_trust: bool | None = False#
field is_certificate_required: bool | None = False#
field is_input_header_xml_column: bool | None = False#
field is_input_xml_column: bool | None = False#
field is_log_reject_reason_disabled: bool | None = False#
field is_output_header_xml_column: bool | None = False#
field is_output_xml_column: bool | None = False#
field is_pass_through_disabled: bool | None = False#
field is_proxy_required: bool | None = False#
field is_reject_column: bool | None = False#
field jvm_options: str | None = ''#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field keystore_file: str | None = ''#
field log_response: LogResponse | None = LogResponse.zero#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field operation_timeout: int | None = 0#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_col: list | None = None#
field output_count: int | None = 0#
field output_message_table_id: str | None = ''#
field output_namespace_name: str | None = ''#
field output_namespace_value: str | None = ''#
field output_reject_column: str | None = ''#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_coll: str | None = 'part_type'#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_clear#
field proxy_host_name: str | None = ''#
field proxy_password: str | None = ''#
field proxy_port_number: str | None = ''#
field proxy_user_name: str | None = ''#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field request_mapping: str | None = ''#
field request_name: str | None = '' (alias 'requestName')#
field response_name: str | None = '' (alias 'responseName')#
field runtime_column_propagation: int | None = 1#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field ssl_certificate: str | None = ''#
field stage_description: list | None = '' (alias 'stageDescription')#
field truststore_type: TruststoreType | None = TruststoreType.JKS#
field user_classpath: str | None = ''#
field wsdl_document_uri: str | None = ''#
field wsdl_document_xml: str | None = ''#
field wsdl_input_message_namespace: str | None = ''#
field wsdl_operation_name: str | None = ''#
field wsdl_operation_style: str | None = ''#
field wsdl_port_address: str | None = ''#
field wsdl_port_name: str | None = ''#
field wsdl_service_name: str | None = ''#
field wsdl_soap_action: str | None = ''#
field xml_input_column: str | None = ''#
field xml_input_header_column: str | None = ''#
field xml_output_column: str | None = ''#
field xml_output_header_column: str | None = ''#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/WSTransformerPX.svg'#
label: ClassVar[str] = 'Web Service'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'WSTransformerPX'#

This module defines configuration or the Write Range Map stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.write_range_map.write_range_map#

Bases: BaseStage

Properties for the Write Range Map stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_seq#
field flow_dirty: str | None = 'false'#
field hide: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field key_properties: list | None = [] (alias 'keyProperties')#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field overwrite: Overwrite | None = Overwrite.false#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field rangemap: str = None#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/PxWriteRangeMap.svg'#
label: ClassVar[str] = 'Write Range Map'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxWriteRangeMap'#

This module defines configuration or the XML Input stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.xml_input.xml_input#

Bases: BaseStage

Properties for the XML Input stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field accept_nulls: bool | None = False#
field accept_nulls_output: bool | None = False#
field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field discard_empty_values: bool | None = False#
field discard_empty_values_output: bool | None = False#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_grammar_caching: bool | None = False#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_seq#
field flow_dirty: str | None = 'false'#
field format_serialized_chunks: bool | None = False#
field format_serialized_chunks_output: bool | None = False#
field heap_size: int | None = 256#
field hide: bool | None = False#
field include_namespaces: bool | None = False#
field include_namespaces_output: bool | None = False#
field inherit_properties: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field is_icp4d: bool | None = False (alias 'isIcp4d')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field log_reject_errors: bool | None = False#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field namespace_declaration: str | None = ''#
field namespace_declaration_output: str | None = ''#
field other_options: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field reject_message_column: RejectMessageColumn | None = RejectMessageColumn.custom#
field replace_nulls: bool | None = False#
field replace_nulls_output: bool | None = False#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field use_custom_xslt: bool | None = False#
field validate_input: bool | None = False#
field xml_source_column: XmlSourceColumn | None = XmlSourceColumn.false#
field xml_text: XmlText | None = XmlText.URL#
field xml_validation_level: XmlValidationLevel | None = XmlValidationLevel.default#
field xslt_error_mapping: XsltErrorMapping | None = XsltErrorMapping.DS_FATAL#
field xslt_fatal_mapping: XsltFatalMapping | None = XsltFatalMapping.DS_REJECT#
field xslt_source: XsltSource | None = XsltSource.property#
field xslt_stylesheet: str | None = ''#
field xslt_text: XsltText | None = XsltText.URL#
field xslt_warning_mapping: XsltWarningMapping | None = XsltWarningMapping.DS_WARNING#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/XMLInputPX.svg'#
label: ClassVar[str] = 'XML Input'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxXMLInput'#

This module defines configuration or the XML Output stage.

pydantic model ibm_watsonx_data_integration.services.datastage.models.stage_models.xml_output.xml_output#

Bases: BaseStage

Properties for the XML Output stage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field auto_column_propagation: bool | None = None#
field buf_free_run: int | None = 50#
field buf_free_run_ronly: int | None = 50#
field buf_mode: BufMode | None = BufMode.default#
field buf_mode_ronly: BufModeRonly | None = BufModeRonly.default#
field coll_type: CollType | None = CollType.auto#
field combinability: Combinability | None = Combinability.auto#
field discard_empty_values: bool | None = False#
field discard_empty_values_output: bool | None = False#
field disk_write_inc: int | None = 1048576#
field disk_write_inc_ronly: int | None = 1048576#
field document_type: str | None = ''#
field document_type_output: str | None = ''#
field document_type_text: str | None = 'text'#
field document_type_text_output: str | None = 'text'#
field empty_element_style: str | None = 'use single tag'#
field empty_element_style_output: str | None = 'use single tag'#
field enable_flow_acp_control: bool = True (alias 'enableFlowAcpControl')#
field enable_grammar_caching: bool | None = False#
field enable_schemaless_design: bool = False (alias 'enableSchemalessDesign')#
field execmode: Execmode | None = Execmode.default_seq#
field flow_dirty: str | None = 'false'#
field generate_comment: bool | None = True#
field generate_comment_output: bool | None = True#
field generate_full_document: bool | None = False#
field generate_full_document_output: bool | None = False#
field header: str | None = ''#
field header_output: str | None = ''#
field header_text: str | None = 'text'#
field header_text_output: str | None = 'text'#
field heap_size: int | None = 256#
field hide: bool | None = False#
field include_document: bool | None = False#
field include_document_output: bool | None = False#
field include_header: bool | None = False#
field include_header_output: bool | None = False#
field include_namespaces: bool | None = False#
field include_namespaces_output: bool | None = False#
field include_xml_chunk: bool | None = False#
field include_xml_chunk_output: bool | None = False#
field indentation_character: str | None = '\\s'#
field indentation_character_output: str | None = '\\s'#
field indentation_length: int | None = 2#
field indentation_length_output: int | None = 2#
field inherit_properties: bool | None = False#
field input_count: int | None = 0#
field inputcol_properties: list | None = [] (alias 'inputcolProperties')#
field is_icp4d: bool | None = False (alias 'isIcp4d')#
field is_remote_engine: bool | None = False (alias 'isRemoteEngine')#
field key_col_select: KeyColSelect | None = KeyColSelect.default (alias 'keyColSelect')#
field key_cols_coll: list | None = [] (alias 'keyColsColl')#
field key_cols_coll_ordered: list | None = [] (alias 'keyColsCollOrdered')#
field key_cols_coll_robin: list | None = [] (alias 'keyColsCollRobin')#
field key_cols_none: list | None = [] (alias 'keyColsNone')#
field key_cols_part: list | None = [] (alias 'keyColsPart')#
field log_reject_errors: bool | None = False#
field max_mem_buf_size: int | None = 3145728#
field max_mem_buf_size_ronly: int | None = 3145728#
field multi_line_output: bool | None = False#
field multi_line_output_output: bool | None = False#
field namespace_declaration: str | None = ''#
field namespace_declaration_output: str | None = ''#
field new_line_style: str | None = 'MAC'#
field new_line_style_output: str | None = 'MAC'#
field other_options: str | None = None#
field output_acp_should_hide: bool = True (alias 'outputAcpShouldHide')#
field output_count: int | None = 0#
field output_file: str | None = ''#
field output_file_output: str | None = ''#
field output_row_generation: str | None = 'use column'#
field output_row_generation_output: str | None = 'use column'#
field output_row_trigger_column: str | None = ''#
field output_row_trigger_column_output: str | None = ''#
field outputcol_properties: list | None = [] (alias 'outputcolProperties')#
field part_client_dbname: str | None = None#
field part_client_instance: str | None = None#
field part_dbconnection: str | None = ''#
field part_stable: bool | None = None#
field part_stable_coll: bool | None = False#
field part_stable_ordered: bool | None = False#
field part_stable_roundrobin_coll: bool | None = False#
field part_table: str | None = None#
field part_type: PartType | None = PartType.auto#
field part_unique: bool | None = None#
field part_unique_coll: bool | None = False#
field part_unique_ordered: bool | None = False#
field part_unique_roundrobin_coll: bool | None = False#
field perform_sort: bool | None = False#
field perform_sort_coll: bool | None = False#
field perform_sort_modulus: bool | None = False#
field preserve: Preserve | None = Preserve.default_propagate#
field queue_upper_size: int | None = 0#
field queue_upper_size_ronly: int | None = 0#
field reject_message_column: str | None = ''#
field replace_nulls: bool | None = False#
field replace_nulls_output: bool | None = False#
field runtime_column_propagation: bool | None = None#
field show_coll_type: int | None = 0 (alias 'showCollType')#
field show_part_type: int | None = 1 (alias 'showPartType')#
field show_sort_options: int | None = 0 (alias 'showSortOptions')#
field sort_instructions: str | None = '' (alias 'sortInstructions')#
field sort_instructions_text: str | None = '' (alias 'sortInstructionsText')#
field stage_description: list | None = '' (alias 'stageDescription')#
field validate_output: bool | None = False#
field write_to_file: bool | None = False#
field write_to_file_output: bool | None = False#
field xml_chunk: str | None = ''#
field xml_chunk_output: str | None = ''#
field xml_chunk_text: str | None = 'text'#
field xml_chunk_text_output: str | None = 'text'#
field xml_encoding: str | None = 'UTF-8'#
field xml_encoding_output: str | None = 'UTF-8'#
field xml_error_mapping: XmlErrorMapping | None = XmlErrorMapping.DS_FATAL#
field xml_fatal_mapping: XmlFatalMapping | None = XmlFatalMapping.DS_REJECT#
field xml_validation_level: XmlValidationLevel | None = XmlValidationLevel.default#
field xml_warning_mapping: XmlWarningMapping | None = XmlWarningMapping.DS_WARNING#
image: ClassVar[str] = '/data-intg/flows/graphics/palette/XMLOutputPX.svg'#
label: ClassVar[str] = 'XML Output'#
node_type: ClassVar[str] = 'execution_node'#
op_name: ClassVar[str] = 'PxXMLOutput'#

Batch Subflow#

Subflow.

class ibm_watsonx_data_integration.services.datastage.models.flow.subflow.Subflow(*, dag: DAG | None = None, name: str = 'unnamed_subflow', is_local: bool = False, subflow_id: str | None = None, description: str = '', project: Project, parameter_sets: list = [], parent_dag: DAG | None = None, label: str = '', pipeline_id: str | None = None, url: str | None = None, local_parameters: list[Parameter] = [], local_parameter_values: dict[str, str] = {}, response: dict | None = None, convert_data: bool = False, **kwargs)#

Bases: FlowComposer, SuperNode

Represents a batch subflow.

Initializer for subflow.

add_entry_node(label: str | None = None)#

Adds an entry node to the subflow.

Parameters:

label (str, optional) – Label for the entry node. Defaults to None.

Returns:

EntryNode object added to the subflow.

Return type:

EntryNode

add_exit_node(label: str | None = None)#

Adds an exit node to the subflow.

Parameters:

label (str, optional) – Label for the exit node. Defaults to None.

Returns:

ExitNode object added to the subflow.

Return type:

ExitNode

add_local_parameter(parameter_type: str, name: str, value: str | int = '', description: str = '')#

Adds a local parameter to the subflow. Default value is optional.

Return type:

Subflow

set_local_parameter(name: str, value: str)#

Set the value of a local parameter.

Return type:

Subflow

use_parameter_set(parameter_set: ParameterSet)#

Use parameter set in the subflow.

Return type:

Subflow

class ibm_watsonx_data_integration.services.datastage.models.flow.subflow.Subflows(project: Project)#

Bases: CollectionModel[Subflow]

Collection of Subflow objects.

Initialize the Subflows collection.

Parameters:

project (Project) – The Project object.

Configuration#

Module containing configuration class, useful for legacy+platform streamsets objects.

class ibm_watsonx_data_integration.services.streamsets.models.configuration.Configuration(configuration: dict | list[dict] | None = None, compatibility_map: dict | None = None, property_key: str = 'name', property_value: str = 'value', id_to_remap: dict | None = None)#

Bases: object

Abstraction for configurations.

This class enables easy access to and modification of data stored as a list of dictionaries. A Configuration is stored in the form:

[{"name" : "<name_1>","value" : "<value_1>"}, {"name" : "<name_2>", value" : "<value_2>"},...]

However, the passed in configuration parameter can be a list of Configurations such as:

[[{"name" : "<name_1>","value" : "<value_1>"}, {"name" : "<name_2>", value" : "<value_2>"},...],
[{"name" : "<name_3>","value" : "<value_3>"}, {"name" : "<name_4>", value" : "<value_4>"},...],...]

The __init__ for the configuration class.

Parameters:

  • configuration (Union[dict, list[dict], None], default: None) – The configuration to represent with this class.

  • compatibility_map (Optional[dict], default: None) – Any backwards compatibility map for older version of things

  • property_key (str, default: 'name') – The key on which a configuration’s key exists

  • property_value (str, default: 'value') – The key on which a configuration’s value exists

  • id_to_remap (Optional[dict], default: None) – If we want to change certain names of configurations

get(key: str, default: any | None = None)#

Return the value of key or, if not in the configuration, the default value.

Return type:

any

items()#

Gets the configuration’s items.

Return type:

Iterable[tuple]

update(configs: dict)#

Update instance with a collection of configurations.

Parameters:

configs (dict) – Dictionary of configurations to use.

Return type:

None

Connections#

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.Connection#

Bases: BaseModel, Generatable

The __init__ of the Connection class.

Parameters:

  • connection_json (dict) – The JSON for the Connection.

  • platform (Optional[Platform], default: None) – The Platform object.

  • project (Optional[Project], default: None) – The Project object.

field asset_category: str | None = None#

The asset category

field connection_id: str | None [Optional]#

Returns id of connection

field data_source_definition_asset_id: str | None = None#

The id of the data source definition asset related to connection. For example “cfdcb449-1204-44ba-baa6-9a8a878e6aa7”.

field data_source_definition_asset_name: str | None = None#

The name of the data source definition asset related to connection. For example “Data privacy profile for DB2”.

field datasource_type: str [Required]#

The id or the name of the data source type to connect to. For example “cfdcb449-1204-44ba-baa6-9a8a878e6aa7” or “db2”.

field description: str | None = None#

The description of the connection.

field gateway_id: str | None = None#

The id of the secure gateway to use with the connection. A Secure Gateway is needed when connecting to an on-premises data source. This is the id of the Secure Gateway created with the SecureGateway Service. Your Secure Gateway Client running on-premises must be connected to the gateway with this Id. For example, “E9oXGRIhv1e_prod_ng”.

field interaction_properties: ConnectionInteractionProperties | None = None#
field metadata: ConnectionMetadata | None = None#
field name: str [Required]#

The name of the connection.

field owner_id: str | None = None#

Owner or creator of connection. Provided when a service ID token is used to create connection.

field properties: dict | None = None#
field ref_asset_id: str | None = None#

The ID of the connection in reference catalog that this connection refers to for properties values.

field ref_catalog_id: str | None = None#

The ID of the catalog that this connection refers to for properties values.

field resource_key: str | None = None#

Resource key that should be set in connection asset metadata record instead of using a calculated one by the service.

field source_system: dict[str, dict[str, Any]] | None = None#
field source_system_history: list[dict[str, dict[str, Any]]] | None = None#
field tags: list[str] | None = None#
model_dump(*, by_alias: bool = True, exclude_unset: bool = True, **kwargs: dict)#

Changing default parameters of model_dump to make sure that serialized json math API response.

Parameters:

  • by_alias (bool, default: True) – Whether to use alias names in serialization.

  • exclude_unset (bool, default: True) – Whether to exclude unset fields from serialization.

  • **kwargs (dict) – Additional keyword arguments to pass to the model_dump method.

Return type:

dict

Returns:

A dictionary representation of the model.

property actions: list[DatasourceTypeAction]#

Get all supported actions for the connection.

Returns:

Retrieved list of Actions.

property origin: dict#

Returns origin model dump.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.ConnectionFile#

Bases: BaseModel

The __init__ of the Datasource class.

Parameters:

  • file_json (dict) – The JSON for the ConnectionFile.

  • platform (Optional[Platform], default: None) – The Platform object.

field created_at: str | None = None (alias 'createdAt')#

Date of creation of the file.

field digest: str | None = None#

Digest of the file.

field file_name: str | None = None (alias 'fileName')#

Name of the file to be used by connections.

field hash: str | None = None#

Hash of the file.

field url: str | None = None#

Signed URL of the file.

download(output: Path)#

Download a file.

Parameters:

output (Path) – destination file path.

Return type:

Response

Returns:

A HTTP response.

class ibm_watsonx_data_integration.cpd_models.connections_model.ConnectionFiles(platform: Platform)#

Bases: CollectionModel[ConnectionFile]

Collection of ConnectionFile instances.

The __init__ of the ConnectionFile class.

Parameters:

platform (Platform) – The Platform object.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.ConnectionInteractionProperties#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field source: list[DatasourceTypeProperty] | None = None#

The properties that can be set for a source interaction.

field target: list[DatasourceTypeProperty] | None = None#

The properties that can be set for a target interaction.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.ConnectionMetadata#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field asset_category: str | None = None#

The asset category (“USER” or “SYSTEM”)

field asset_id: str | None = None#

The ID of the asset

field asset_type: str | None = None#

The type of the asset

field catalog_id: str | None = None#

The ID of the catalog which contains the asset. catalog_id, project_id or spaceid is required.

field create_time: str | None = None#

The timestamp when the asset was created (in format YYYY-MM-DDTHH:mm:ssZ or YYYY-MM-DDTHH:mm:ss.sssZ, matching the date-time format as specified by RFC 3339)

field creator_id: str | None = None#

The IAM ID of the user that created the asset

field href: str | None = None#

URL that can be used to get the asset.

field owner_id: str | None = None#

The IAM ID of the user that owns the asset

field project_id: str | None = None#

The ID of the project which contains the asset. catalog_id, project_id or spaceid is required.

field resource_key: str | None = None#

Optional external unique key for assets that supports it

field space_id: str | None = None#

The ID of the space which contains the asset. catalog_id, project_id or spaceid is required.

field tags: list[str] | None = None#
class ibm_watsonx_data_integration.cpd_models.connections_model.Connections(platform: Platform, project: Project)#

Bases: CollectionModel[Connection]

Collection of Connection instances.

The __init__ of the Connection class.

Parameters:

  • platform (Platform) – The Platform object.

  • project (Project) – Instance of Project in which connection was created.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.ConnectionsServiceInfo#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field configuration: dict[str, Any] | None = None#
field failure_message: str | None = None#

A message indicating the cause if the service is not running correctly

field service_name: str | None = None#

The name of the service

field status: str | None = None#

An overall status indicating whether the service is running correctly

field timestamp: str | None = None#

The timestamp when the information was retrieved (in format YYYY-MM-DDTHH:mm:ssZ or YYYY-MM-DDTHH:mm:ss.sssZ, matching the date-time format as specified by RFC 3339)

field version: str | None = None#

The service version string

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.DatasourceType#

Bases: BaseModel

The __init__ of the Datasource class.

Parameters:

  • datasource_json (dict) – The JSON for the Datasource.

  • platform (Optional[Platform], default: None) – The Platform object.

field actions: list[DatasourceTypeAction] | None = None#

The actions supported for the data source.

field allowed_as_source: bool | None = None#

Whether the data source can be accessed as a source of data. That is, data can be read from the data source.

field allowed_as_target: bool | None = None#

Whether the data source can be accessed as a target. That is, data can be written to the data source.

field child_source_systems: list[dict[str, dict[str, Any]]] | None = None#
field description: str | None = None#

A localized, displayable description of the data source.

field label: str | None = None#

A localized, displayable label such as, “IBM dashDB”.

field metadata: ConnectionMetadata | None = None#
field name: str | None = None#

A unique name, such as “dashdb”.

field origin_country: str | None = None#

Country which data originated from. - ISO 3166 Country Codes.

field owner_id: str | None = None#

Owner or creator of connection. Provided when a service ID token is used to create connection.

field properties: DatasourceTypeProperties [Optional]#
field status: str | None = None#

The status of the data source.

field tags: list[str] | None = None#

Tags associated with a data source type.

model_dump(*, by_alias: bool = True, exclude_unset: bool = True, **kwargs: dict)#

Changing default parameters of model_dump to make sure that serialized json math API response.

Parameters:

  • by_alias (bool, default: True) – Whether to use alias names in serialization.

  • exclude_unset (bool, default: True) – Whether to exclude unset fields from serialization.

  • **kwargs (dict) – Additional keyword arguments to pass to the model_dump method.

Return type:

dict

Returns:

A dictionary representation of the model.

property datasource_id: str#

Returns id of datasource.

property required_connection_properties: list[DatasourceTypeProperty]#

Get all required connection properties.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.DatasourceTypeAction#

Bases: BaseModel

The __init__ of the DatasourceTypeAction class.

Parameters:

  • action_json (dict) – The JSON for the DatasourceTypeAction.

  • platform (Optional[Platform], default: None) – The Platform object.

  • project (Optional[Project], default: None) – The Project object.

  • connection (Optional[Connection], default: None) – The Connection object.

field description: str | None = None#

A description of the action.

field name: str | None = None#

The action name.

field properties: DatasourceTypeActionProperties | None = None#
execute(configuration: dict[str, Any] | None = None, connection: Connection | None = None)#

Execute DatasourceTypeAction.

Parameters:

  • configuration (Optional[dict[str, Any]], default: None) – action config parameters.

  • connection (Optional[Connection], default: None) – connection to be used.

Return type:

Response

Returns:

A HTTP response.

Raises:

ValueError – If no connection is provided.

model_dump(*, by_alias: bool = True, exclude_unset: bool = True, **kwargs: dict)#

Changing default parameters of model_dump to make sure that serialized json math API response.

Parameters:

  • by_alias (bool, default: True) – Whether to use alias names in serialization.

  • exclude_unset (bool, default: True) – Whether to exclude unset fields from serialization.

  • **kwargs (dict) – Additional keyword arguments to pass to the model_dump method.

Return type:

dict

Returns:

A dictionary representation of the model.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.DatasourceTypeActionProperties#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field input: list[DatasourceTypeProperty] | None = None#

The input properties.

field output: list[DatasourceTypeProperty] | None = None#

The properties of the action result.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.DatasourceTypeProperties#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field connection: list[DatasourceTypeProperty] [Optional]#

The connection properties.

field filter: list[DatasourceTypeProperty] | None = None#

The filter properties that can be set for a discovery interaction.

field source: list[DatasourceTypeProperty] | None = None#

The properties that can be set for a source interaction.

field target: list[DatasourceTypeProperty] | None = None#

The properties that can be set for a target interaction.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.DatasourceTypeProperty#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field default_value: str | None = None#

The default value for the property if the value is not otherwise specified.

field description: str | None = None#

The description for the property.

field group: str | None = None#

A classification group for the property.

field hidden: bool | None = None#

Whether the property should be displayed in a user interface.

field label: str | None = None#

The label for the property.

field masked: bool | None = None#

Whether the property should be masked. For example, when the property is a password.

field name: str | None = None#

The property name.

field required: bool | None = None#

Whether the property is required.

field type: str | None = None#

The type of the property.

field values: list[dict] | None = None#

If the property type is enum, the list of enumerated values that the property can take.

class ibm_watsonx_data_integration.cpd_models.connections_model.DatasourceTypes(platform: Platform)#

Bases: CollectionModel[DatasourceType]

Collection of DatasourceType instances.

The __init__ of the DatasourceType class.

Parameters:

platform (Platform) – The Platform object.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.DirectoryAsset#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field path: str | None = None#

Folder ID that the connection asset lives in.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.DiscoveredAsset#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field description: str | None = None#

A description of the asset.

field details: dict[str, dict[str, Any]] | None = None#
field has_children: bool | None = None#

True if it is known that the asset has children. False if it is known that the asset does not have children. If it is not known, or it is too expensive to determine this, then this property will not be returned.

field id: str | None = None#

An ID for the asset.

field name: str | None = None#

A name for the asset.

field path: str | None = None#

The path for the object which can be used to discover child assets.

field tags: list[str] | None = None#

Tags associated with the asset.

field type: str | None = None#

The type of the asset, such as SCHEMA, TABLE, FILE, or FOLDER.

pydantic model ibm_watsonx_data_integration.cpd_models.connections_model.FunctionalID#

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field description: str | None = None#

Description of this functional.

field entitlements: list[str] | None = None#

The list of names of relevant entitlements.

field functional_id: str | None = None#

the ID of function.

field label: str | None = None#

the label of this functional.

field tools: list[str] | None = None#

Allowed tools that consume this functional ID.

DAG#

class ibm_watsonx_data_integration.services.datastage.models.flow.dag.DAG#

Bases: object

DAG to represent the graph structure of a batch flow.

Initializes a DAG to represent the graph structure of a batch flow.

Add a link to the DAG.

Return type:

None

add_node(node: Node)#

Add a node to the DAG.

Return type:

Node

compute_metadata()#

Get metadata.

Return type:

None

get_connected_subgraphs()#

Get connected subgraphs.

Return type:

list

get_dag_from_nodes(nodes: set[Node])#

Creates a dag object only including the specified nodes.

Returns:

subgraph including the specified nodes

Return type:

DAG

Get all input links to dest node.

Return type:

list[Link]

Gets a link by name.

Return type:

Link | None

Get links between two nodes.

Return type:

list[Link]

Get all output links of src node.

Return type:

list[Link]

get_topological_ordering(*, stages_only: bool)#

Get topological ordering.

Return type:

list

Returns all links in the dag in any order.

Return type:

Iterator[Link]

Returns links using stable sort.

Return type:

Iterator[Link]

nodes()#

Return nodes in the dag.

Return type:

Iterator[Node]

Remove a link from DAG.

Return type:

list[Link]

Remove links from DAG.

Return type:

list[Link]

remove_node(node: Node)#

Remove a node from DAG.

Return type:

None

replace_node(old_node: Node, new_node: Node)#

Replace a node in the DAG.

Return type:

None

replace_nodes_with_supernode(supernode: Node, nodes: set[Node])#

Replaces references of nodes with a SuperNode (used for subflow replacement).

Parameters:

  • supernode (SuperNode) – super node to replace the set of nodes with

  • nodes (set[Node]) – set of nodes to replace

  • outgoing (tuple[Node, Node, list[Link]]) – outgoing connections from the subflow to main flow

Return type:

None

stages()#

Return stages in the DAG.

Return type:

Iterator[Node]

Engine#

Module containing Engine Models.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.engine_model.Engine#

Bases: BaseModel

The Model for Engine.

The __init__ of the Engine class.

Parameters:

  • platform (Optional[Platform], default: None) – The Platform object.

  • project (Optional[Project], default: None) – The Project object.

  • engine_json (dict) – The JSON for the Engine.

field engine_id: str [Optional]#
field engine_type: str [Required]#
field health: EngineHealthMetadata | None = None#
field last_startup_time: int = None#
field metadata: EngineMetadata [Required]#
field registration_status: str | None [Required]#
field registration_time: int [Required]#
field reported_build_sha: str = None#
field reported_build_time: int = None#
field reported_engine_version: str = None#
field reported_java_vendor: str | None = None#
field reported_java_version: str = None#
field reported_os_arch: str = None#
field reported_os_name: str = None#
field reported_os_version: str = None#
field streamsets_environment_asset_id: str [Required]#
field url: str = None#
property api_client: DataCollectorAPIClient#

The API Client connected directly to the engine.

property description: str#

Returns the description of the engine.

property library_definitions: LibraryDefinitions#

Library Definitions of the Engine.

property name: str#

Returns the name of the engine.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.engine_model.EngineHealthMetadata#

Bases: BaseModel

StreamSets Engine Health Metadata.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field last_heartbeat_time: int | None = None#
field last_registration_time: int | None = None#
field last_startup_time: int | None = None#
field last_status_updated_time: int | None = None#
field metrics: EngineMetrics | None = None#
field status: str [Required]#
pydantic model ibm_watsonx_data_integration.services.streamsets.models.engine_model.EngineMetadata#

Bases: BaseModel

The model for CPD StreamSets Engine Metadata.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field asset_attributes: list | None [Required]#
field asset_id: str [Required]#
field asset_state: str | None [Required]#
field asset_type: str | None [Required]#
field catalog_id: str [Required]#
field create_time: str | None = None#
field created: int [Required]#
field created_at: str [Required]#
field creator_id: str [Required]#
field description: str [Required]#
field name: str [Required]#
field origin_country: str | None = None#
field owner: str = None#
field owner_id: str [Required]#
field project_id: str [Required]#
field rating: int | None [Required]#
field rov: dict | None [Required]#
field sandbox_id: str [Required]#
field size: int | None [Required]#
field space_id: str = None#
field tags: list | None [Optional]#
field usage: dict | None [Required]#
field version: int | None [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

pydantic model ibm_watsonx_data_integration.services.streamsets.models.engine_model.EngineMetrics#

Bases: BaseModel

StreamSets Engine Metrics.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field cpu_load: float [Required]#
field job_count: int [Required]#
field memory_load: float [Required]#
class ibm_watsonx_data_integration.services.streamsets.models.engine_model.Engines(project: Project)#

Bases: CollectionModel[Engine]

Collection of Engine instances.

The __init__ of the Engines class.

Parameters:

project (Project) – The Project object.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.engine_model.LibraryDefinitions#

Bases: BaseModel

An engine’s library definition.

The __init__ for this class.

field category: Optional[Any] [Required]#
field category_label: str | None [Required] (alias 'categoryLabel')#
field el_catalog: Any [Required] (alias 'elCatalog')#
field event_definitions: dict [Required] (alias 'eventDefinitions')#
field executor_version: str | None [Required] (alias 'executorVersion')#
field legacy_stage_libs: list[dict] [Required] (alias 'legacyStageLibs')#
field pipeline: list[dict] [Required]#
field pipeline_fragment: list[dict] [Required] (alias 'pipelineFragment')#
field pipeline_rules: list[dict] [Required] (alias 'pipelineRules')#
field rules_el_metadata: dict [Required] (alias 'rulesElMetadata')#
field runtime_configs: list[Any] [Required] (alias 'runtimeConfigs')#
field schema_version: str [Required] (alias 'schemaVersion')#
field services: list[dict] [Required]#
field stage_definition_map: dict | None [Required] (alias 'stageDefinitionMap')#
field stage_definition_minimal_list: list[dict] | None [Required] (alias 'stageDefinitionMinimalList')#
field stage_icons: Any [Required] (alias 'stageIcons')#
field stages: list[dict] [Required]#
field version: int | None [Required]#
classmethod for_engine_version(platform: Platform, engine_version: str)#

Returns the library definitions for a particular engine version, includes all possible stage libraries.

Parameters:

  • platform (Platform) – Instance of platform to get the definitions from.

  • engine_version (str) – The engine version for which library definitions need to be fetched.

Return type:

LibraryDefinitions

Returns:

An instance of the LibraryDefinitions class.

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

static process_library_definitions(library_definitions: dict)#

Make library definitions easier to parse.

This entire function essentially expands on stageDefinitionMap and stageDefinitionMinimalList to make the entire library definitions.

This is only required for library definitions gotten from CPD and not from an engine as it is compressed to contain all possible stage libraries.

Return type:

dict

Engine Version#

Module containing Engine Version Models.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.engine_version.StreamingEngineStage#

Bases: BaseModel

Simple Stage definition available in a StreamingEngineStageLib.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field description: str [Required]#
field label: str [Required]#
field name: str [Required]#
field subtype: str [Required]#
field type: str [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

pydantic model ibm_watsonx_data_integration.services.streamsets.models.engine_version.StreamingEngineStageLib#

Bases: BaseModel

Stage lib available in an engine version.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field image_location: str [Required]#
field label: str [Required]#
field stage_lib_id: str [Required]#
field stages: list[StreamingEngineStage] [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

pydantic model ibm_watsonx_data_integration.services.streamsets.models.engine_version.StreamingEngineVersion#

Bases: BaseModel

Represents data for an engine version available on platform.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field disabled: bool = True#
field engine_type: str [Required]#
field engine_version_id: str [Required]#
field image_tag: str [Required]#
field release: bool = False#
field stage_libs: list[StreamingEngineStageLib] | None = None#
field tunneling_supported: bool = False#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

class ibm_watsonx_data_integration.services.streamsets.models.engine_version.StreamingEngineVersions(platform: Platform)#

Bases: CollectionModel[StreamingEngineVersion]

Collection of streaming engine versions.

The __init__ for StreamingEngineVersions.

Parameters:

platform (Platform) – The Platform object.

Environment#

Module containing Environment Models.

class ibm_watsonx_data_integration.services.streamsets.models.environment_model.Engines(project: Project, environment: Environment)#

Bases: CollectionModel[Engine]

Collection of Engine instances.

The __init__ of the Engines class.

Parameters:

  • project (Project) – The Project object.

  • environment (Environment) – The Environment object.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.environment_model.Environment#

Bases: BaseModel

The model for CPD StreamSets Environment.

The __init__ of the Environment class.

Parameters:

  • env_json (dict) – The JSON for the Service.

  • platform (Optional[Platform], default: None) – The Platform object.

  • project (Project) – The Project object.

field cpus_to_allocate: float | None = None#
field custom_cert: str | None = None#
field engine_heartbeat_interval: int | None = None#
field engine_properties: dict[str, Any] [Optional]#
field engine_statuses: dict[str, str] [Optional]#
field engine_type: str [Required]#
field engine_version: str [Required]#
field environment_id: str | None [Optional]#
field external_resource_asset: dict | None [Optional]#
field href: str | None = None#
field jvm_options: list[str] | None [Optional]#
field log4j2_properties: dict[str, str] [Optional]#
field max_cpu_load: float | None = None#
field max_jobs_running: int | None = None#
field max_memory_used: float | None = None#
field metadata: EnvironmentMetadata | None = None#
field stage_libs: list[str] | None [Optional]#
add_stage_libraries(stage_libs: list)#

Allows to add stage libraries to the environment.

Parameters:

stage_libs (list) – List of stage libraries to add.

Return type:

None

get_installation_command(pretty: bool = True, foreground: bool = False)#

Returns the installation command for the environment.

Return type:

str | None

model_dump(*, by_alias: bool = True, exclude_unset: bool = True, **kwargs: dict)#

Changing default parameters of model_dump to make sure that serialized json math API response.

Parameters:

  • by_alias (bool, default: True) – Whether to use alias names in serialization.

  • exclude_unset (bool, default: True) – Whether to exclude unset fields from serialization.

  • **kwargs (dict) – Additional keyword arguments to pass to the model_dump method.

Return type:

dict

Returns:

A dictionary representation of the model.

property description: str#

Returns the description of the environment.

property engines: Engines#

Returns the engines associated with the Environment.

property name: str#

Returns the name of the environment.

property origin: dict#

Returns origin model dump.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.environment_model.EnvironmentMetadata#

Bases: BaseModel

The model for CPD StreamSets Environment Metadata.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field asset_attributes: list | None [Required]#
field asset_id: Union[Annotated[UUID], str] [Required]#
field asset_state: str | None [Required]#
field asset_type: str | None [Required]#
field catalog_id: Union[Annotated[UUID], str, None] [Required]#
field create_time: str [Required]#
field description: str = ''#
field name: str [Required]#
field owner_id: str [Required]#
field project_id: Union[Annotated[UUID], str] [Required]#
field version: float | None [Required]#
model_dump(*, by_alias: bool = True, exclude_unset: bool = True, **kwargs: dict)#

Changing default parameters of model_dump to make sure that serialized json math API response.

Parameters:

  • by_alias (bool, default: True) – Whether to use alias names in serialization.

  • exclude_unset (bool, default: True) – Whether to exclude unset fields from serialization.

  • **kwargs (dict) – Additional keyword arguments to pass to the model_dump method.

Return type:

dict

Returns:

A dictionary representation of the model.

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

class ibm_watsonx_data_integration.services.streamsets.models.environment_model.Environments(platform: Platform, project: Project)#

Bases: CollectionModel[Environment]

Collection of Environments instances.

The __init__ of the Environment class.

Parameters:

  • platform (Platform) – The Platform object.

  • project (Project) – The Project object.

HTTP Retry#

HTTPError Retry logic.

exception ibm_watsonx_data_integration.common.retry.HTTPRetryError(message: str, last_exception: Exception | None = None, last_response: Any | None = None, attempts_made: int = 0, total_time: float = 0.0)#

Bases: HTTPError

Exception raised when max retries are exceeded.

Setting members of HTTPRetryError class.

class ibm_watsonx_data_integration.common.retry.RetryConfig#

Bases: object

Global retry configuration - class-only interface.

Available Operations:

enable() - Enable retries (previous config or defaults) disable() - Disable all retries temporarily reset() - Reset to factory defaults

set() - Replace entire configuration add() - Add status codes or configurations remove() - Remove status codes or configurations

show() - Display current configuration get_current_config() - Get current configuration as dict

Raises RuntimeError since RetryConfig should not be instantiated.

classmethod add(cls, status_configs: dict[int, RetrySettings] | None = None, retryable_status_codes: set[int] | None = None)#

Add status codes and/or configurations to existing retry settings.

Parameters:

  • status_configs (Optional[dict[int, RetrySettings]], default: None) – Additional status-specific retry configurations to add/update

  • retryable_status_codes (Optional[set[int]], default: None) – Additional status codes to mark as retryable

Return type:

None

Example

# Add new retryable status codes
>>> RetryConfig.add(retryable_status_codes={404, 405})

# Add specific configuration for a status code
>>> RetryConfig.add(
...     status_configs={
...         404: RetrySettings(max_attempts=5, init_delay=2.0)
...     }
... )
classmethod disable(cls)#

Disable all retries, saves current configuration for later restore via enable().

Return type:

None

classmethod enable(cls, use_defaults: bool = False)#

Enable retries.

Parameters:

use_defaults (bool, default: False) – If True, use default configuration, If False use previous enabled configuration

Return type:

None

classmethod get_current_config()#

Get current configuration as a dictionary.

Return type:

dict[str, Any]

classmethod get_settings_for_status(status_code: int)#

Get retry settings for a specific HTTP status code.

Return type:

RetrySettings

classmethod is_retryable_status(status_code: int)#

Check if status code should be retried.

Return type:

bool

classmethod remove(cls, status_configs: set[int] | None = None, retryable_status_codes: set[int] | None = None)#

Remove status codes and/or configurations from retry settings.

Parameters:

  • status_configs (Optional[set[int]], default: None) – Set of status codes to remove from status-specific configurations

  • retryable_status_codes (Optional[set[int]], default: None) – Set of status codes to remove from retryable list

Return type:

None

Example

# Remove status codes from retryable list
>>> RetryConfig.remove(retryable_status_codes={404, 405})

# Remove specific configuration (will fall back to default)
>>> RetryConfig.remove(status_configs={429})

Note

  • Removing from status_configs makes that code use default settings

  • Removing from retryable_status_codes makes that code non-retryable (unless it’s still in status_configs)

classmethod reset(cls)#

Reset to default configuration.

Return type:

None

classmethod set(cls, default: RetrySettings | None = None, status_configs: dict[int, RetrySettings] | None = None, retryable_status_codes: set[int] | None = None)#

Replace current configuration (overwrites existing settings).

Parameters:

  • default (Optional[RetrySettings], default: None) – New default retry settings (replaces current default)

  • status_configs (Optional[dict[int, RetrySettings]], default: None) – New status-specific configurations (replaces all existing configs)

  • retryable_status_codes (Optional[set[int]], default: None) – New set of retryable status codes (replaces existing set)

Return type:

None

Warning

This method REPLACES configurations, not adds to them. Use add() to add to existing configuration without removing others.

Example

# Replace default settings
>>> RetryConfig.set(default=RetrySettings(max_attempts=10))

# Replace ALL status-specific configs (removes previous ones)
>>> RetryConfig.set(
...     status_configs={
...         429: RetrySettings(max_attempts=20),
...         500: RetrySettings(max_attempts=5)
...     }
... )

# Replace ALL retryable status codes (removes previous ones)
>>> RetryConfig.set(retryable_status_codes={429, 500, 503})

# Replace everything at once
>>> RetryConfig.set(
...     default=RetrySettings(max_attempts=5),
...     status_configs={429: RetrySettings(max_attempts=15)},
...     retryable_status_codes={429, 500, 502, 503, 504}
... )
classmethod show()#

Display current retry configuration in human-readable format.

Return type:

None

Example

>>> RetryConfig.show()
======================================================================
RETRY CONFIGURATION
STATUS: RETRIES ENABLED
======================================================================

Default Settings:
Applied to status codes: [500, 502, 503, 504]
  Max Attempts:    3
  Max Time:        None (max allowed: 3600s)
  Initial Delay:   1.0s
  Exp Factor:      2.0x
  Jitter:          Enabled
class ibm_watsonx_data_integration.common.retry.RetryConfigMeta#

Bases: type

Metaclass for RetryConfig, needed to create __repr__ for RetryConfig.

class ibm_watsonx_data_integration.common.retry.RetrySettings(max_attempts: int = 3, max_time: float | None = None, init_delay: float = 2.0, exp_factor: float = 2, jitter: bool = True)#

Bases: object

Configuration for retry behavior.

ibm_watsonx_data_integration.common.retry.get_current_retry_override()#

Get current retry override for this thread.

Return type:

dict[str, Any] | None

ibm_watsonx_data_integration.common.retry.http_retry()#

Decorator for HTTP retry logic with jitter and exponential backoff.

Uses global RetryConfig settings or context manager overrides. When retries are disabled (max_attempts=1), behaves as if decorator was not applied.

Return type:

Callable

ibm_watsonx_data_integration.common.retry.no_retry_on_http_error()#

Context manager to disable retry for the current thread.

Return type:

Iterator[None]

ibm_watsonx_data_integration.common.retry.retry_on_http_error(settings: RetrySettings | None = None, only_status_codes: set[int] | None = None, skip_status_codes: set[int] | None = None)#

Context manager for temporary retry override with error filtering.

Parameters:

  • settings (Optional[RetrySettings], default: None) – Retry settings to use

  • only_status_codes (Optional[set[int]], default: None) – If specified, ONLY retry these status codes

  • skip_status_codes (Optional[set[int]], default: None) – If specified, retry ALL status codes EXCEPT these

Return type:

Iterator[None]

Logic:

  • only_status_codes={429} = retry ONLY 429

  • skip_status_codes={500} = retry ALL 4xx and 5xx status codes EXCEPT 500

  • No parameters = retry ALL 4xx and 5xx status codes

Example

# Retry ONLY rate limiting errors
>>> with retry_on_http_error(RetrySettings(max_attempts=10), only_status_codes={429}):
...    project.delete_job(job)  # Only retries on 429

# Retry everything except 500 errors
>>> with retry_on_http_error(RetrySettings(max_attempts=5), skip_status_codes={500}):
...    project.delete_job(job)  # Retries all errors except 500

# Retry everything (all 4xx and 5xx)
>>> with retry_on_http_error(RetrySettings(max_attempts=20)):
...    project.delete_job(job)  # Retries ALL 4xx and 5xx status codes

Job#

Module containing Job and Job Run Models.

pydantic model ibm_watsonx_data_integration.cpd_models.job_model.Job#

Bases: BaseModel

The model for CPD Job.

The __init__ of the Job class.

Parameters:

  • platform (Optional[Platform], default: None) – The Platform object.

  • project (Optional[Project], default: None) – The Project object.

  • job_json (dict) – The JSON for the Job.

field asset_ref: str | None = None#
field configuration: JobConfiguration [Required]#
field enable_notifications: bool [Required]#
field future_scheduled_runs: list [Required]#
field job_id: str [Optional]#
field job_parameters: list[JobParameter] | None = None#
field job_type: str [Required] (alias 'asset_ref_type')#
field last_run_status_timestamp: int [Required]#
field metadata: JobMetadata [Required]#
field notification_types: NotificationTypes | None = None#
field parameter_sets: list[ParameterSet] | None = None#
field project_name: str [Required]#
field retention_policy: RetentionPolicy | None = None#
field schedule: str | None = None#
field schedule_creator_id: str [Required]#
field schedule_id: str [Required]#
field schedule_info: ScheduleInfo | None = None#
delete_job_run(job_run: JobRun)#

Delete given run of job.

Parameters:

job_run (JobRun) – Instance of a Job Run to delete.

Return type:

Response

Returns:

A HTTP response. If it is 204, then the operation completed successfully. If the code is 202, then the operation is in progress.

edit_configuration(environment: str = <object object>, warn_limit: int = <object object>, retention_days: int = <object object>, retention_amount: int = <object object>, parameter_value_sets: list[tuple[str, str]] = <object object>, job_parameters: list[tuple[str, str]] = <object object>, schedule: ~ibm_watsonx_data_integration.cpd_models.job_model.Schedule = <object object>, notify_success: bool = <object object>, notify_warning: bool = <object object>, notify_failure: bool = <object object>)#

This method edits the configuration of a job.

It uses a sentinel object to make sure that the arguments the user explicitly change are the only ones that are affected.

Return type:

Response

model_dump(*, by_alias: bool = True, exclude_unset: bool = True, **kwargs: dict)#

Changing default parameters of model_dump to make sure that serialized json math API response.

Parameters:

  • by_alias (bool, default: True) – Whether to use alias names in serialization.

  • exclude_unset (bool, default: True) – Whether to exclude unset fields from serialization.

  • **kwargs (dict) – Additional keyword arguments to pass to the model_dump method.

Return type:

dict

Returns:

A dictionary representation of the model.

print_json(indent: int = 4)#

String JSON representation of Job.

Return type:

str

reset_offset()#

This method is intended to clear the current offset associated with a job.

Return type:

Response

Returns:

The HTTP response.

Raises:

TypeError – If the provided job type is not streaming.

start(name: str = 'job run', description: str = '', configuration: dict[str, Any] | None = None, job_parameters: dict[str, Any] | None = None, parameter_sets: list[dict[str, str]] | None = None)#

Create Job Run for given configuration.

Parameters:

  • name (str, default: 'job run') – Name for a Job Run.

  • description (str, default: '') – Description for a Job Run.

  • configuration (Optional[dict[str, Any]], default: None) – Environment variables.

  • job_parameters (Optional[dict[str, Any]], default: None) – Parameters use internally by a Job.

  • parameter_sets (Optional[list[dict[str, str]]], default: None) – Parameter sets for a Job Run.

Return type:

JobRun

Returns:

An instance of a Job Run.

property description: str#

Returns description of job.

property job_runs: JobRuns#

Returns a list of Job Runs of the job.

Returns:

A list of jobs runs for the given job.

property job_schedule: Schedule | None#

Compute Schedule object from job to improve user readability.

property name: str#

Returns name of job.

property origin: dict#

Returns origin model dump.

property version: str#

Returns version of job.

pydantic model ibm_watsonx_data_integration.cpd_models.job_model.JobConfiguration#

Bases: BaseModel

Holds configuration parameters which Job was run.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field deployment_job_definition_id: str = ''#
field env_id: str = ''#
field env_type: str = ''#
field env_variables: list[str] | None [Optional]#
field flow_limits: dict[str, int] = {'warn_limit': 0}#
field offset: dict[str, Any] | None = None#
field outputs: Outputs [Optional]#
field version: str = ''#
pydantic model ibm_watsonx_data_integration.cpd_models.job_model.JobMetadata#

Bases: BaseModel

Model representing metadata for a job.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field asset_id: str [Required]#
field description: str | None = ''#
field name: str [Required]#
field owner_id: str [Required]#
field version: int [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

pydantic model ibm_watsonx_data_integration.cpd_models.job_model.JobParameter#

Bases: BaseModel

Parameter used when running job.

Represents parameter used by Connectors and Stages to dynamically change value.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field name: str [Required]#
field value: Any [Required]#
pydantic model ibm_watsonx_data_integration.cpd_models.job_model.JobRun#

Bases: BaseModel

The model for CPD Job Run.

The __init__ of the Job Run class.

Parameters:

  • platform (Optional[Platform], default: None) – The Platform object.

  • project (Optional[Project], default: None) – The Project object.

  • job_run_json (dict) – The JSON for the Job Run.

field configuration: JobConfiguration [Required]#
field duration: int | None = None#
field execution_end: int | None = None#
field execution_start: int | None = None#
field is_scheduled_run: bool = False (alias 'isScheduledRun')#
field job_id: str [Required] (alias 'job_ref')#
field job_name: str [Required]#
field job_parameters: list[JobParameter] | None = None#
field job_run_id: str [Optional]#
field job_type: str [Required]#
field last_state_change_timestamp: str | None = None#
field metadata: JobRunMetadata [Required]#
field parameter_sets: list[ParameterSet] | None = None#
field project_name: str | None = None#
field queue_end: int | None = None#
field queue_start: int | None = None#
field resource_usage: float | None = None#
field runtime_job_id: str | None = None#
field state: JobRunState [Required]#
field total_rows_read: int | None = None#
field total_rows_written: int | None = None#
field total_stages: int | None = None#
cancel()#

Stop already started Job Run.

Return type:

Response

Returns:

A HTTP response. If it is 204, then the operation completed successfully.

model_dump(*, by_alias: bool = True, exclude_unset: bool = True, **kwargs: dict)#

Changing default parameters of model_dump to make sure that serialized json math API response.

Parameters:

  • by_alias (bool, default: True) – Whether to use alias names in serialization.

  • exclude_unset (bool, default: True) – Whether to exclude unset fields from serialization.

  • **kwargs (dict) – Additional keyword arguments to pass to the model_dump method.

Return type:

dict

Returns:

A dictionary representation of the model.

refresh_status()#

Updated status of a job.

Return type:

Response

property logs: list[str]#

Retrieves runtime logs for a job run.

Returns:

A list containing runtime log entries that describe the job run execution. Each entry is a single log line from the UI.

Raises:

TypeError – If the provided job type is streaming.

pydantic model ibm_watsonx_data_integration.cpd_models.job_model.JobRunMetadata#

Bases: BaseModel

Model representing metadata for a Job Run.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field asset_id: str [Required]#
field created: int [Required]#
field created_at: str [Required]#
field name: str [Required]#
field owner_id: str [Required]#
field usage: dict[str, Any] [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

class ibm_watsonx_data_integration.cpd_models.job_model.JobRunState(value)#

Bases: str, Enum

Available states for Job Run.

class ibm_watsonx_data_integration.cpd_models.job_model.JobRuns(platform: Platform, project: Project, job_id: str)#

Bases: CollectionModel[JobRun]

Collection of Job Run instances.

The __init__ of the JobRuns class.

Parameters:

  • platform (Platform) – The Platform object.

  • project (Project) – Instance of Project in which job run was created.

  • job_id (str) – ID of Job for which runs was stared.

class ibm_watsonx_data_integration.cpd_models.job_model.JobType(value)#

Bases: str, Enum

Internal enum for asset/job type constants to replace hardcoded strings.

class ibm_watsonx_data_integration.cpd_models.job_model.Jobs(platform: Platform, project: Project)#

Bases: CollectionModel[Job]

Collection of Job instances.

The __init__ of the Jobs class.

Parameters:

  • platform (Platform) – The Platform object.

  • project (Project) – Instance of Project in which job was created.

pydantic model ibm_watsonx_data_integration.cpd_models.job_model.NotificationTypes#

Bases: BaseModel

Notification Types model.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field failure: bool = False#
field success: bool = False#
field warning: bool = False#
pydantic model ibm_watsonx_data_integration.cpd_models.job_model.Outputs#

Bases: BaseModel

Holds job configuration output information.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field total_bytes_read: int = 0#
field total_bytes_written: int = 0#
field total_rows_read: int = 0#
field total_rows_written: int = 0#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

pydantic model ibm_watsonx_data_integration.cpd_models.job_model.ParameterSet#

Bases: BaseModel

Parameter sets.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field name: str [Required]#
field ref: str [Required]#
field value_set: str | None = None#
pydantic model ibm_watsonx_data_integration.cpd_models.job_model.RetentionPolicy#

Bases: BaseModel

Retention policy model.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field amount: int | None = None#
field days: int | None = None#
pydantic model ibm_watsonx_data_integration.cpd_models.job_model.Schedule#

Bases: BaseModel

Provide details about scheduling a job.

Parameters:

  • start_date – The scheduled job will be triggered after this timestamp. Format: yyyy-mm-dd hh-mm

  • repeat_mode – When to repeat the job. Options are minutely, hourly, daily, weekly, monthly.

  • repeat_value – Values to accompany repeat_mode. These values have different types depending on the repeat_mode. minutely: int. Repeats every repeat_value minutes hourly: int. Repeats at repeat_value minutes past hour daily: datetime.time. Repeats every day at repeat_value weekly: tuple. First value represents the day of the week, second value is the time. monthly: tuple. First value represents the day of the month, second value is the time.

  • exclude_days – Days to exclude. Only available when repeat_mode is minutely, hourly, or daily. 0-6 corresponds to Sunday-Saturday.

  • end_date – The date to end the schedule. Format: yyyy-mm-dd hh-mm

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field end_date: datetime | None = None#
field exclude_days: list[int] | None = None#
field repeat_mode: Optional[Literal['minutely', 'hourly', 'daily', 'weekly', 'monthly']] = None#
field repeat_value: int | time | tuple[int | str, time] | None = None#
field start_date: datetime | None = None#
pydantic model ibm_watsonx_data_integration.cpd_models.job_model.ScheduleInfo#

Bases: BaseModel

Represent schedule configuration for Job.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field end_on: int | None = None (alias 'endOn')#
field repeat: bool | None = None#
field start_on: int | None = None (alias 'startOn')#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

Models#

Model module.

pydantic model ibm_watsonx_data_integration.common.models.BaseModel#

Bases: BaseModel

BaseModel class to standardize objects .

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

validator entity_validation  »  all fields#

Flattens/Manipulates the data.

Parameters:

data (dict) – Data to be manipulated on

Return type:

dict

Returns:

A dict with manipulated data.

model_dump(by_alias: bool = True, **kwargs: any)#

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Return type:

dict

Returns:

A dictionary representation of the model.

classmethod model_json_schema(**kwargs: Any)#

Generate a JSON schema for the model.

This method overrides the Pydantic base method to provide a properly formatted docstring that avoids Sphinx/docutils warnings about bullet list formatting.

Parameters:

**kwargs (Any) – Additional keyword arguments passed to the parent method.

Return type:

dict

Returns:

A dictionary containing the JSON schema for the model.

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

serialize_model(handler: Any, info: SerializationInfo)#

Serializes the data, back into it’s original formatting.

Return type:

dict

Returns:

A dict with updated information

class ibm_watsonx_data_integration.common.models.CollectionModel(platform: Platform)#

Bases: Iterable[T]

Base class wrapper with abstractions for pagination.

The __init__ of the Project class.

Parameters:

platform (Platform) – The Platform object.

get(**kwargs)#

Used to get an instant result from the api.

Parameters:

**kwargs – Optional arguments to be passed to filter the results.

Return type:

TypeVar(T, bound= BaseModel)

Returns:

An inherited instance of streamsets.sdk.sch_models.BaseModel.

Raises:

ValueError – If instance is not in the list.

get_all(**kwargs: dict)#

Used to get multiple (all) results from api.

Parameters:

**kwargs (dict) – Optional other arguments to be passed to filter the results.

Return type:

SeekableList[TypeVar(T, bound= BaseModel)]

Returns:

A list of inherited instances of

streamsets.sdk.sch_models.BaseModel.

class ibm_watsonx_data_integration.common.models.CollectionModelResults(results, class_type, response_bookmark, request_bookmark, response_location, constructor_params)#

Bases: tuple

Create new instance of CollectionModelResults(results, class_type, response_bookmark, request_bookmark, response_location, constructor_params)

class_type#

Alias for field number 1

constructor_params#

Alias for field number 5

request_bookmark#

Alias for field number 3

response_bookmark#

Alias for field number 2

response_location#

Alias for field number 4

results#

Alias for field number 0

Parameter Set#

Module containing Parameter Set and Parameter Models.

pydantic model ibm_watsonx_data_integration.cpd_models.parameter_set_model.Parameter#

Bases: BaseModel

Class for a parameter in a parameter set.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field description: str | None = None#
field name: str [Required]#
field param_type: str [Required] (alias 'type')#
field prompt: str | None = None#
field valid_values: list | None = None#
field value: Any | None = None#
classmethod from_dict(data: dict[str, Any])#

Create Parameter from dict with validation.

Parameters:

data (dict[str, Any]) – Dictionary containing parameter data.

Return type:

Parameter

Returns:

A validated Parameter instance.

Raises:

ValueError – If parameter type is missing, invalid, or if enum type lacks valid_values.

validator validate_enum_has_valid_values  »  all fields#

Ensure enum type has valid_values.

Return type:

Parameter

pydantic model ibm_watsonx_data_integration.cpd_models.parameter_set_model.ParameterSet#

Bases: BaseModel

A parameter set consists of a name, a list of parameters, a list of value sets, and a description.

Parameters:

  • name – The name of the parameter set.

  • parameters – A list of Parameter objects.

  • description – An optional description of the parameter set.

  • value_sets – A list of ValueSet objects.

The __init__ of the ParameterSet class.

Parameters:

  • platform – The Platform object.

  • project (Optional[Project], default: None) – The Project object.

field description: str | None = ''#

The description of the parameter set

field metadata: ParameterSetMetadata | None = None#
field name: str [Required]#

The name of the parameter set.

field parameter_set_id: str | None [Optional]#

Returns id of parameter set

field parameters: list[Parameter] [Optional]#

The parameters in a parameter set

field value_sets: list[ValueSet] [Optional]#

The value sets of a parameter set

add_parameter(parameter_type: str, name: str, description: str | None = None, prompt: str | None = None, value: str | int | float | bool | list | None = None, valid_values: list | None = None)#

Adds a Parameter to the parameter set or updates it if it already exists.

Parameters:

  • parameter_type (str) – Parameter type.

  • name (str) – Parameter name.

  • description (Optional[str], default: None) – Parameter help text.

  • prompt (Optional[str], default: None) – Parameter prompt

  • value (Union[str, int, float, bool, list, None], default: None) – Parameter value

  • valid_values (Optional[list], default: None) – Valid values for List Parameter only.

Return type:

ParameterSet

Returns:

The ParameterSet object with an updated list of parameters.

add_value_set(value_set: ValueSet)#

Adds a ValueSet to the list of value sets.

Parameters:

value_set (ValueSet) – (Required) A ValueSet object.

Return type:

ParameterSet

Returns:

The ParameterSet object with an updated list of value_sets.

classmethod from_dict(parameter_set_data: dict[str, Any])#

Creates a new ParameterSet object from a dictionary of values.

Parameters:

parameter_set_data (dict[str, Any]) – A dictionary of values for a ParameterSet object.

Return type:

ParameterSet

Returns:

A new ParameterSet object with the existing parameter set details.

remove_parameter(parameter_name: str)#

Removes a parameter from the parameter set.

Parameters:

parameter_name (str) – Name of the parameter to remove

Return type:

ParameterSet

Returns:

The ParameterSet object with an updated list of parameters.

property origin: dict#

Returns origin model dump.

pydantic model ibm_watsonx_data_integration.cpd_models.parameter_set_model.ParameterSetMetadata#

Bases: BaseModel

Metadata for a parameter set.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field asset_category: str | None = None#

The asset category (“USER” or “SYSTEM”)

field asset_id: str | None = None#

The ID of the asset

field asset_type: str | None = None#

The type of the asset

field catalog_id: str | None = None#

The ID of the catalog which contains the asset. catalog_id, project_id or spaceid is required.

field create_time: str | None = None#

The timestamp when the asset was created

field creator_id: str | None = None#

The IAM ID of the user that created the asset

field href: str | None = None#

URL that can be used to get the asset.

field owner_id: str | None = None#

The IAM ID of the user that owns the asset

field project_id: str | None = None#

The ID of the project which contains the asset. catalog_id, project_id or spaceid is required.

field resource_key: str | None = None#

Optional external unique key for assets that supports it

field space_id: str | None = None#

The ID of the space which contains the asset. catalog_id, project_id or spaceid is required.

field tags: list[str] | None = None#
class ibm_watsonx_data_integration.cpd_models.parameter_set_model.ParameterSets(project: Project)#

Bases: CollectionModel[ParameterSet]

Collection of ParameterSet instances.

The __init__ of the ParameterSets class.

Parameters:

project (Project) – Instance of Project in which parameter set was created.

get_by_ids(ids: list[str])#

Fetch parameter sets by their IDs in bulk.

Parameters:

ids (list[str]) – List of parameter set IDs to fetch.

Return type:

list[ParameterSet]

Returns:

List of ParameterSet instances.

class ibm_watsonx_data_integration.cpd_models.parameter_set_model.ParameterType(value)#

Bases: str, Enum

Available types for parameter.

pydantic model ibm_watsonx_data_integration.cpd_models.parameter_set_model.ValueSet#

Bases: BaseModel

Class for a value set for a parameter set.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field name: str [Required]#
field values: list[dict[str, ParameterValue]] [Optional]#
add_value(name: str, value: str | int | float | bool | list)#

Add or replace value in ValueSet.

Return type:

ValueSet

remove_value(name: str)#

Remove value from ValueSet.

Return type:

ValueSet

Platform#

This module defines top-level abstractions for WatsonX Data Integration.

class ibm_watsonx_data_integration.platform.Platform(auth: BaseAuthenticator, base_url: str = 'https://cloud.ibm.com', base_api_url: str = 'https://api.dataplatform.cloud.ibm.com')#

Bases: object

Class to interact with IBM Cloud Pak for Data.

The __init__ for the Platform class.

Parameters:

  • auth (BaseAuthenticator) – The Authentication object.

  • base_url (str, default: 'https://cloud.ibm.com') – The Cloud Pak for Data URL.

  • base_api_url (str, default: 'https://api.dataplatform.cloud.ibm.com') – The Cloud Pak for Data API URL.

property access_groups: AccessGroups | AccessGroupsOnPrem#

Returns a collection of AccessGroup or AccessGroupsOnPrem objects.

Returns:

A collection of Access Group objects retrieved from the API.

property accounts: Accounts#

Returns a list of all accounts associated with the current IAM identity or API key.

Returns:

A list of account objects retrieved from the API.

add_member_to_multiple_access_groups(member: UserProfile | ServiceID | TrustedProfile, access_groups: list[AccessGroup])#

Adds a member to multiple Access Groups.

Parameters:
Return type:

Response

Returns:

A HTTP response.

property available_engine_versions: StreamingEngineVersions#

Lists all released Streaming Engine Versions.

Returns:

A list of Engine versions.

property connections_service_info: ConnectionsServiceInfo#

Retrieves information about connection service. Can be used as heartheat mechanism.

Returns:

A connection service information.

create_access_group(name: str, description: str | None = None)#

Creates an access group and returns the created access group object.

Parameters:

  • name (str) – Name of the Access Group

  • description (Optional[str], default: None) – Description of the Access Group

Return type:

AccessGroup

Returns:

An Access Group instance.

create_access_group_on_prem(name: str, roles: RoleOnPrem, description: str | None = '')#

Creates an access group and returns the created access group object.

Parameters:

  • name (str) – Name of the Access Group

  • roles (RoleOnPrem) – Role of the Access Group

  • description (str | None, default: '') – Description of the Access Group

Return type:

AccessGroupOnPrem

Returns:

An Access Group instance.

create_project(name: str, description: str = '', tags: list | None = None, public: bool = False, project_type: str = 'wx')#

Create a Project.

Parameters:

  • name (str) – The Project name.

  • description (str, default: '') – The name of the Project.

  • tags (Optional[list], default: None) – The tags of the Project.

  • public (bool, default: False) – Whether the Project is public.

  • project_type (str, default: 'wx') – Type of the Project - ‘cpd’ - IBM Cloud Pak for Data; ‘wx’ - IBM watsonx (and Data Fabric).

Return type:

Project

Returns:

A project instance.

create_role(name: str, display_name: str, service_name: str, actions: list, description: str | None = None)#

Creates a custom role for a specific service within the account.

Parameters:

  • display_name (str) – The display the name of the role that is shown in the console.

  • actions (list) – The actions of the role (list of strings).

  • name (str) – The name of the role that is used in the CRN. This must be alphanumeric and capitalized.

  • service_name (str) – The service name.

  • description (Optional[str], default: None) – The description of the role.

Return type:

Role

Returns:

A role instance.

create_role_on_prem(name: str, permissions: list[str], description: str = '')#

Creates an on-prem role.

Return type:

RoleOnPrem

Returns:

A role instance.

create_service_instance(instance_type: str, name: str, target: str | None = None, tags: list | None = None)#

Creates Service Instance.

Parameters:

  • instance_type (str) – The Instance Type.

  • name (str) – The name of the Instance.

  • target (Optional[str], default: None) – The target of the Instance.

  • tags (Optional[list], default: None) – The tags of the Instance.

Return type:

Service

Returns:

A Service instance.

property current_account: Account#

Gets the current (first) account from the list of accounts.

Returns:

An account object representing the first account retrieved from the accounts list.

Raises:

ValueError – If there are no accounts available for the provided IAM identity or API key.

property current_on_prem_user: UserProfileOnPrem#

Gets the current (first) user from the list of user.

Returns:

An user object representing the first user retrieved from the accounts list.

Raises:

ValueError – If there are no user available for the provided API key.

property datasources: DatasourceTypes#

Retrieves available datasource types.

Returns:

A list of available DatasourceTypes.

delete_access_group(access_group: AccessGroup | AccessGroupOnPrem)#

Deletes an access group and returns output of api call to delete.

Parameters:

access_group (AccessGroup | AccessGroupOnPrem) – Access Group to be deleted

Return type:

Response

Returns:

A HTTP response.

delete_file(file: ConnectionFile)#

Delete a file.

Return type:

Response

Returns:

A HTTP response.

delete_project(project: Project)#

Deletes a Project.

Parameters:

project (Project) – The Project to delete.

Return type:

Response

Returns:

A HTTP response.

delete_role(role: Role | RoleOnPrem)#

Delete a custom role.

Parameters:

role (Role | RoleOnPrem) – The role to delete.

Return type:

Response

Returns:

A HTTP response.

Raises:

TypeError – If you try to delete service_role or system_role.

delete_service_instance(service: Service, delete_keys: bool = True)#

Delete Resource Instance.

Parameters:

  • service (Service) – The Service to delete.

  • delete_keys (bool, default: True) – Whether to recursively delete resource keys.

Return type:

Response

Returns:

A HTTP response.

property files: ConnectionFiles#

Retrieves list of files.

Returns:

List of connection files.

property projects: Projects#

Returns a list of Project objects.

Returns:

An instance of ibm_watsonx_data_integration.cpd_models.Projects

remove_member_from_all_access_groups(member: UserProfile | ServiceID | TrustedProfile)#

Removes a member from all access groups under an account.

Parameters:

member (UserProfile | ServiceID | TrustedProfile) – The member object to remove.

Return type:

Response

Returns:

A HTTP response.

property roles: Roles | RolesOnPrem#

Returns a list of all roles.

Returns:

An instance of watsonx_di_sdk.role_model.Roles

property service_ids: ServiceIDs#

Returns a collection of ServiceID objects.

Returns:

A collection of ServiceID objects retrieved from the API.

property service_instances: Services#

Returns a list of Service Instances.

Returns:

An instance of ibm_watsonx_data_integration.cpd_models.Services

property trusted_profiles: TrustedProfiles#

Returns a collection of TrustedProfile objects.

Returns:

A collection of TrsutedProfile objects retrieved from the API.

update_access_group(access_group: AccessGroup | AccessGroupOnPrem)#

Updates an existing group and returns the output for api call to update.

Parameters:

access_group (AccessGroup | AccessGroupOnPrem) – Access Group to be updated

Return type:

Response

Returns:

A HTTP response.

update_project(project: Project)#

Updates a Project.

Parameters:

project (Project) – The Project to delete.

Return type:

Response

Returns:

A HTTP response.

update_role(role: Role | RoleOnPrem)#

Update a custom role.

A role administrator might want to update an existing custom role by updating the display name, description, or the actions that are mapped to the role. The name, account_id, and service_name can’t be changed.

Parameters:

role (Role | RoleOnPrem) – The role to update.

Return type:

Response

Returns:

A HTTP response.

Raises:

TypeError – If you try to modify service_role or system_role.

upload_file(name: str, file: Path)#

Uploads file.

Parameters:

  • name (str) – Name of the file to upload.

  • file (Path) – File to upload.

Return type:

Response

Returns:

A HTTP response.

Raises:

  • ValueError – If incorrect file path is provided.

  • FileNotFoundError – If file path does not exist.

property users: UserProfiles | UserProfilesOnPrem#

Retrieves collection of all the user profiles in the current account.

Returns:

An iterable collection of user profiles.

validate()#

Perform validation of Platform setup.

Raises:

  • requests.exceptions.ConnectionError – If base_api_url is invalid or dns cannot be resolved.

  • ibm_watsonx_data_integration.common.exceptions.IbmCloudApiException – If base_api_url can be resolved but is incorrect.

Return type:

None

Project#

Module containing Project Models.

pydantic model ibm_watsonx_data_integration.cpd_models.project_model.Project#

Bases: BaseModel

The Model for Projects.

The __init__ of the Project class.

Parameters:

  • platform (Platform) – The Platform object.

  • project_json (dict) – The JSON for the Service.

field creator: str | None [Required]#
field description: str | None = None#
field generator: str | None [Required]#
field metadata: ProjectMetadata [Required]#
field name: str [Required]#
field project_id: str [Optional]#
field public: bool | None [Required]#
field scope: Scope | None [Required]#
field storage: Storage | None [Required]#
field tags: list[str] | None [Optional]#
field type: str | None [Required]#
add_collaborator(collaborator: AccessGroup | ServiceID | TrustedProfile | UserProfile | AccessGroupOnPrem | UserProfileOnPrem, *, role: CollaboratorRole)#

Adds a collaborator to the project.

Parameters:
Return type:

ProjectCollaborator

Returns:

ProjectCollaborator object.

add_collaborators(collaborators: list[AccessGroup | ServiceID | TrustedProfile | UserProfile | AccessGroupOnPrem | UserProfileOnPrem], *, role: CollaboratorRole)#

Adds a list of collaborators to the project.

Args:t

collaborators: list of collaborators to add role: role to assign members

Return type:

list[ProjectCollaborator]

Returns:

List of collaborators added.

copy_connection(connection: Connection)#

Copy the Connection.

Parameters:

connection (Connection) – connection to copy

Return type:

Connection

Returns:

Copied Connection object.

create_connection(name: str, datasource_type: DatasourceType, description: str | None = None, properties: dict | None = None, test: bool = True)#

Create a Connection.

Parameters:

  • name (str) – name for the new connection.

  • description (Optional[str], default: None) – description for the new connection.

  • datasource_type (DatasourceType) – type of the datasource.

  • properties (Optional[dict], default: None) – properties of the new connection.

  • test (bool, default: True) – whether to test the connection before saving it. If true and validation cannot be estabilished, connection will not be saved.

Return type:

Connection

Returns:

Created Connection object.

create_environment(*, name: str, engine_version: StreamingEngineVersion | str | None = None, description: str | None = None, engine_type: str = 'data_collector', engine_properties: dict[str, Any] | None = None, log4j2_properties: dict[str, Any] | None = None, external_resource_asset: dict[str, Any] | None = None, stage_libs: list[str] | None = None, jvm_options: list[str] | None = None, max_cpu_load: float | None = None, max_memory_used: float | None = None, max_jobs_running: int | None = None, engine_heartbeat_interval: int | None = None, cpus_to_allocate: float | None = None, custom_cert: str | None = None)#

Allows to create a new Environment within project.

All of not set parameters will be skipped and set with default values provided by backed.

Parameters:

  • name (str) – Name of the environment.

  • description (Optional[str], default: None) – Description of the environment.

  • engine_type (str, default: 'data_collector') – Type of the engine.

  • engine_version (Union[StreamingEngineVersion, str, None], default: None) – Version of the engine. Default is the latest engine version.

  • engine_properties (Optional[dict[str, Any]], default: None) – Properties of the engine.

  • external_resource_asset (Optional[dict[str, Any]], default: None) – External resources.

  • log4j2_properties (Optional[dict[str, Any]], default: None) – Log4j2 properties.

  • stage_libs (Optional[list[str]], default: None) – Stage libraries.

  • jvm_options (Optional[list[str]], default: None) – JVM options.

  • max_cpu_load (Optional[float], default: None) – Maximum CPU load.

  • max_memory_used (Optional[float], default: None) – Maximum memory used.

  • max_jobs_running (Optional[int], default: None) – Maximum jobs running.

  • engine_heartbeat_interval (Optional[int], default: None) – Engine heartbeat interval.

  • cpus_to_allocate (Optional[float], default: None) – Number of CPU used.

  • custom_cert (Optional[str], default: None) – Custom cert to add to the engine truststore.

Return type:

Environment

Returns:

The created environment.

create_flow(name: str, environment: Environment | None = None, description: str = '', flow_type: str = 'streaming')#

Creates a Flow.

Parameters:

  • name (str) – The name of the flow.

  • environment (Optional[Environment], default: None) – The environment which will be used to run this flow.

  • description (str, default: '') – The description of the flow.

  • flow_type (str, default: 'streaming') – The type of flow (must be registered in Flow.flow_registry).

Return type:

Flow

Returns:

The created Flow subclass instance (StreamingFlow by default).

create_job(name: str, flow: Flow, configuration: dict[str, Any] | None = None, description: str | None = None, job_parameters: dict[str, Any] | None = None, retention_policy: dict[str, int] | None = None, notification_types: dict[str, bool] | None = None, parameter_sets: list[dict[str, str]] | None = None, schedule: str | None = None, schedule_info: dict[str, Any] | None = None)#

Create Job for given asset.

Parameters:

  • name (str) – Name for a Job.

  • flow (Flow) – A reference to the flow for which the job will be created.

  • configuration (Optional[dict[str, Any]], default: None) – Environment variables for a Job.

  • description (Optional[str], default: None) – Job description.

  • job_parameters (Optional[dict[str, Any]], default: None) – Parameters use internally by a Job.

  • retention_policy (Optional[dict[str, int]], default: None) – Retention policy for a Job.

  • notification_types (Optional[dict[str, bool]], default: None) – Notification types for a Job.

  • parameter_sets (Optional[list[dict[str, str]]], default: None) – Parameter sets for a Job.

  • schedule (Optional[str], default: None) – Crone string.

  • schedule_info (Optional[dict[str, Any]], default: None) – Schedule info for a Job.

Return type:

Job

Returns:

A Job instance.

Raises:

TypeError – If both asset_ref and asset_ref_type are provided, or if neither is provided

create_parameter_set(name: str, description: str = '', parameters: list[Parameter | dict] | None = None, value_sets: list | None = None)#

Create a Parameter Set.

Parameters:

  • name (str) – name for the new parameter set.

  • description (str, default: '') – description for the new parameter set.

  • parameters (Optional[list[Parameter | dict]], default: None) – parameters for the new parameter set.

  • value_sets (Optional[list], default: None) – value sets for the new parameter set.

Return type:

ParameterSet

Returns:

Created ParameterSet object.

create_subflow(name: str, description: str = '')#

Creates a Subflow component in the project.

Parameters:

  • name (str) – The name of the subflow.

  • description (str, default: '') – The description of the subflow.

Return type:

Subflow

Returns:

The created subflow object.

delete_connection(connection: Connection)#

Remove the Connection.

Parameters:

connection (Connection) – connection to delete

Return type:

Response

Returns:

A HTTP response.

delete_engine(engine: Engine)#

Allows to delete specified Engine within project.

Parameters:

engine (Engine) – Instance of an Engine to delete.

Return type:

Response

Returns:

A HTTP response.

delete_environment(environment: Environment)#

Allows to delete specified Environment within a Project.

Parameters:

environment (Environment) – Instance of an Environment to delete.

Return type:

Response

Returns:

A HTTP response.

delete_environments(*environments: Environment)#

Allows to delete multiple Environments within a Project.

Parameters:

environments (Environment) – Instances of an Environment to delete.

Return type:

Response

Returns:

A HTTP response.

delete_flow(flow: Flow)#

Delete a Flow.

Parameters:

flow (Flow) – The Flow object.

Return type:

Response

Returns:

A HTTP response.

delete_job(job: Job)#

Allows to delete specified Job within project.

Parameters:

job (Job) – Instance of a Job to delete.

Return type:

Response

Returns:

A HTTP response. If it is 204, then the operation completed successfully.

delete_parameter_set(parameter_set: ParameterSet)#

Delete a Parameter Set.

Parameters:

parameter_set (ParameterSet) – Parameter set to delete

Return type:

Response

Returns:

A HTTP response.

delete_subflow(subflow: Subflow)#

Delete a subflow.

Parameters:

subflow (Subflow) – The Subflow object.

Return type:

Response

Returns:

A HTTP response.

duplicate_flow(flow: Flow, name: str, description: str | None = None, number_of_copies: int = 1)#

Duplicate a Flow.

Parameters:

  • flow (Flow) – The Flow.

  • name (str) – The name of the flow.

  • description (Optional[str], default: None) – The description of the flow.

  • number_of_copies (int, default: 1) – The number of copies to make.

Return type:

Flow

Returns:

A copy of passed flow.

duplicate_parameter_set(parameter_set: ParameterSet)#

Duplicate a Parameter Set.

Parameters:

parameter_set (ParameterSet) – Parameter set to copy

Return type:

ParameterSet

Returns:

Duplicated ParameteSet Object

duplicate_subflow(subflow: Subflow, name: str, description: str = '')#

Duplicate a Flow.

Parameters:

  • subflow (Subflow) – The Subflow.

  • name (str) – The name of the subflow.

  • description (str, default: '') – The description of the subflow.

Return type:

Subflow

Returns:

A copy of passed subflow.

export_flow(flow: Flow, with_plain_text_credentials: bool = False, destination: str | Path = 'flows.zip', stream: bool = True)#

Exports a flow to a zip file.

Parameters:

  • flow (Flow) – A flow to export.

  • with_plain_text_credentials (bool, default: False) – A boolean to allow plain text credentials to be exported. Only applies

  • StreamingFlows. (when exporting)

  • destination (str | Path, default: 'flows.zip') – The path to which the zip file should be saved to. Defaults to “flows.zip”.

  • stream (bool, default: True) – Whether to stream the response in chunks.. Defaults to True.

Raises:

TypeError – if flow is not of type Flow

Returns:

path to the exported zip file.

Return type:

Path

export_flows(flows: list[BatchFlow] | list[StreamingFlow], with_plain_text_credentials: bool = False, destination: str | Path = 'flows.zip', stream: bool = True)#

Exports a list of flows to a zip file.

Parameters:

  • flows (list[BatchFlow] | list[StreamingFlow]) – A list of flows to export. Must be either all BatchFlows or StreamingFlows. There cannot be a mix.

  • with_plain_text_credentials (bool, default: False) – A boolean to allow plain text credentials to be exported. Only applies when

  • StreamingFlows. (exporting)

  • destination (str | Path, default: 'flows.zip') – The path to which the zip file should be saved to. Defaults to “flows.zip”.

  • stream (bool, default: True) – Whether to stream the response in chunks.. Defaults to True.

Raises:

TypeError – if flows is an empty list or is not either a list of BatchFlows or StreamingFlows

Returns:

path to the exported zip file.

Return type:

Path

get_batch_environment(display_name: str)#

Retrieve a batch environment by its display name.

Return type:

str

Returns:

Internal name of the environment with the display name.

get_engine(engine_id: str)#

Retrieve an engine by its engine_id.

Parameters:

engine_id (str) – The asset_id of the engine to retrieve.

Returns:

The retrieved engine.

Return type:

Engine

Raises:

HTTPError – If the request fails.

import_flows(flow_type: Literal['batch', 'streaming'], source: str | Path, *, on_failure: BatchImportOnFailure = BatchImportOnFailure.CONTINUE, conflict_resolution: str | BatchImportConflictResolution = BatchImportConflictResolution.RENAME, import_only: bool = False, include_dependencies: bool = True, replace_mode: BatchImportReplaceMode = BatchImportReplaceMode.HARD, import_binaries: bool = False, wait: int | None = -1)#

Imports batch or streaming flows.

Parameters:

  • flow_type (Literal['batch', 'streaming']) – ‘batch’ or ‘streaming’

  • source (str | Path) – The path to the zip file to import.

  • on_failure (BatchImportOnFailure, default: <BatchImportOnFailure.CONTINUE: 'continue'>) – Action when the first import failure occurs. The default action is “continue” which will continue importing the remaining data flows. The “stop” action will stop the import operation upon the first error.

  • conflict_resolution (str | BatchImportConflictResolution, default: <BatchImportConflictResolution.RENAME: 'rename'>) – Resolution when data flow to be imported has a name conflict with an existing data flow in the project or catalog. The default conflict resolution is “skip” will skip the data flow so that it will not be imported.

  • import_only (bool, default: False) – If set to true, will skip compiling the imported flows.

  • include_dependencies (bool, default: True) – If set to false, flow dependencies in the zip file will be skipped.

  • replace_mode (BatchImportReplaceMode, default: <BatchImportReplaceMode.HARD: 'hard'>) – This parameter takes effect when conflict_resolution is set to replace or skip. soft- merge the parameter set, add new parameter only, keep the old value sets hard- replace all

  • import_binaries (bool, default: False) – If set to true, will import binary files found in the zip file.

  • wait (int | None, default: -1) – Time (in seconds) to wait for the import to complete. If set to None, the function will return before completing. If set to -1, will wait indefinitely until all flows have been imported.

Return type:

list[StreamingFlow] | StreamingFlow | BatchImportResponse

Returns:

Response from importing the flows.

list_batch_environments()#

Lists batch environments within a project.

Return type:

list[str]

Returns:

List of internal names of environments.

remove_collaborator(collaborator: ProjectCollaborator)#

Removes a collaborator from the project.

Parameters:

collaborator (ProjectCollaborator) – collaborator to remove

Return type:

Response

Returns:

A HTTPResponse object.

remove_collaborators(collaborators: list[ProjectCollaborator])#

Removes collaborators from the project.

Parameters:

collaborators (list[ProjectCollaborator]) – collaborators to remove

Return type:

Response

Returns:

A HTTPResponse object.

update_collaborator(collaborator: ProjectCollaborator)#

Updates a collaborator in the project.

Parameters:

collaborator (ProjectCollaborator) – collaborator to update

Return type:

ProjectCollaborator

Returns:

An updated project collaborator.

update_collaborators(collaborators: list[ProjectCollaborator])#

Updates collaborators in the project.

Parameters:

collaborators (list[ProjectCollaborator]) – collaborators to update

Return type:

list[ProjectCollaborator]

Returns:

A list of updated members.

update_connection(connection: Connection, test: bool = True)#

Update the Connection.

Parameters:

  • connection (Connection) – connection to update

  • test (bool, default: True) – whether to test the connection before saving it. If true and validation cannot be estabilished, connection will not be saved.

Return type:

Response

Returns:

A HTTP response.

update_environment(environment: Environment)#

Allows to update specified Environment within a Project.

Parameters:

environment (Environment) – Instance of an Environment to update.

Return type:

Response

Returns:

A HTTP response.

update_flow(flow: Flow)#

Update a Flow.

Parameters:

flow (Flow) – The Flow object.

Return type:

Response

Returns:

A HTTP response.

update_job(job: Job)#

Allows to update specified Job within a project.

Parameters:

job (Job) – Instance of a Job to update.

Return type:

Response

Returns:

A HTTP response. If it is 200, then the operation completed successfully.

update_parameter_set(parameter_set: ParameterSet)#

Update a Parameter Set.

Parameters:

parameter_set (ParameterSet) – Parameter set to update

Return type:

Response

Returns:

A HTTP response.

update_subflow(subflow: Subflow)#

Update a subflow.

Parameters:

subflow (Subflow) – The Subflow object.

Return type:

Response

Returns:

A HTTP response.

validate_flow(flow: StreamingFlow)#

Validates a flow.

Parameters:

flow (StreamingFlow) – The Flow to validate.

Return type:

ValidationResult

Returns:

A list of FlowValidationError containing issues.

property collaborators: ProjectCollaborators#

Retrieves project members associated with the project.

Returns:

ProjectCollaborators object.

property connections: Connections#

Retrieves connections associated with the project.

Returns:

A Connections object.

property engines: Engines#

Returns the engines associated with the Project.

property environments: Environments#

Retrieves environments associated with the project.

Returns:

A list of Environments within the project.

property flows: Flows#

Returns Flows from the Project.

property jobs: Jobs#

Retrieves jobs associated with the project.

Returns:

A list of Jobs within the project.

property parameter_sets: ParameterSets#

Retrieves parameter sets associated with the project.

Returns:

A ParameterSets object.

property subflows: Subflows#

Returns Subflows from the Project.

pydantic model ibm_watsonx_data_integration.cpd_models.project_model.ProjectMetadata#

Bases: BaseModel

Model for metadata in a project.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field created_at: str | None [Required]#
field guid: str [Required]#
field updated_at: str | None [Required]#
field url: str [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

class ibm_watsonx_data_integration.cpd_models.project_model.Projects(platform: Platform)#

Bases: CollectionModel[Project]

Collection of Project instances.

The __init__ of the Projects class.

Parameters:

platform (Platform) – The Platform object.

pydantic model ibm_watsonx_data_integration.cpd_models.project_model.Scope#

Bases: BaseModel

Model for Scope details in a project.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field bss_account_id: str [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

pydantic model ibm_watsonx_data_integration.cpd_models.project_model.Storage#

Bases: BaseModel

Model for Storage details in a project.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field guid: str [Required]#
field properties: dict[str, Any] | None = None#
field type: str [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

Project Collaborator#

Module containing ProjectCollaborator and ProjectCollaborators models.

class ibm_watsonx_data_integration.cpd_models.project_collaborator_model.CollaboratorRole(value)#

Bases: str, Enum

Enum that defines different types of collaborator roles in a project.

class ibm_watsonx_data_integration.cpd_models.project_collaborator_model.CollaboratorState(value)#

Bases: str, Enum

Enum that defines different types of collaborator states in a project.

class ibm_watsonx_data_integration.cpd_models.project_collaborator_model.CollaboratorType(value)#

Bases: str, Enum

Enum that defines different types of collaborator types in a project.

class ibm_watsonx_data_integration.cpd_models.project_collaborator_model.ProjectCollaboratable#

Bases: ABC

Interface that indicates a class can be added as a project collaborator.

pydantic model ibm_watsonx_data_integration.cpd_models.project_collaborator_model.ProjectCollaborator#

Bases: BaseModel

Class for a collaborator in a project.

The __init__ of a ProjectCollaborator class.

Parameters:

  • project (Optional[Project], default: None) – The project for the collaborator

  • custom_collaborator_json (dict) – key word arguments to populate the collaborator object

field iam_id: str [Required] (alias 'id')#
Constraints:

  • min_length = 0

  • max_length = 100

field role: CollaboratorRole [Required]#
field state: CollaboratorState = CollaboratorState.ACTIVE#
field type: CollaboratorType = CollaboratorType.USER#
field user_name: str [Required]#
Constraints:

  • min_length = 1

  • max_length = 100

class ibm_watsonx_data_integration.cpd_models.project_collaborator_model.ProjectCollaborators(project: Project)#

Bases: CollectionModel[ProjectCollaborator]

Collection of ProjectCollaborator instances.

The __init__ of the ProjectCollaborators class.

Parameters:

project (Project) – the project object

Role#

Module containing Roles Models.

pydantic model ibm_watsonx_data_integration.cpd_models.role_model.Role#

Bases: BaseModel

The model for a role.

The __init__ of the CustomRole class.

Parameters:

  • platform (Optional[Platform], default: None) – The Platform object.

  • custom_role_json (dict) – The JSON for the CustomRole.

field account_id: str = None#
field actions: list[str] [Required]#
field created_at: str | None = ''#
field created_by_id: str | None = ''#
field crn: str | None = ''#
field description: str | None = ''#
field display_name: str [Required]#
field etag: str | None = ''#
field href: str | None = ''#
field last_modified_at: str | None = ''#
field last_modified_by_id: str | None = ''#
field name: str = None#
field role_id: str = '' (alias 'id')#
field role_type: RoleType [Required]#
field service_name: str = None#
property id: str#

Unique identifier for the Role object.

Deprecated, Use role_id instead

class ibm_watsonx_data_integration.cpd_models.role_model.RoleType(value)#

Bases: str, Enum

The model for RoleType enums.

class ibm_watsonx_data_integration.cpd_models.role_model.Roles(platform: Platform)#

Bases: CollectionModel[Role]

Collection of Roles instances.

The __init__ of the Role class.

Parameters:

platform (Platform) – The Platform object.

Role On Prem#

On-prem Roles models.

pydantic model ibm_watsonx_data_integration.cpd_models.role_model_on_prem.RoleOnPrem#

Bases: BaseModel

On-prem Role model.

__init__ for On-prem Role model.

field description: str | None = None#
field extension_id: str | None = None#
field extension_name: str | None = None#
field permissions: list[str] [Optional]#
field role_id: str [Required] (alias 'id')#
field role_name: str [Required]#
field updated_at: int | None = None#
class ibm_watsonx_data_integration.cpd_models.role_model_on_prem.RolesOnPrem(platform: Platform)#

Bases: CollectionModel[RoleOnPrem]

Collection of on-prem roles.

__init__ of RolesOnPrem.

Service#

Module containing Service Models.

pydantic model ibm_watsonx_data_integration.cpd_models.service_model.Service#

Bases: BaseModel

The Model for Services.

The __init__ of the Service.

Parameters:

  • service_json (dict) – The JSON for the Service.

  • platform (Optional[Platform], default: None) – The Platform object.

field account_id: str [Required]#
field allow_cleanup: bool [Required]#
field controlled_by: str [Required]#
field created_at: str [Required]#
field created_by: str [Required]#
field crn: str [Required]#
field dashboard_url: None | str = None#
field deleted_at: str | None [Required]#
field deleted_by: str [Required]#
field guid: str [Required]#
field id: str [Required]#
field last_operation: dict | None [Required]#
field locked: bool [Required]#
field migrated: bool [Required]#
field name: str [Required]#
field onetime_credentials: bool [Required]#
field plan_history: list | None [Required]#
field region_id: str [Required]#
field reseller_channel_id: str [Required]#
field resource_aliases_url: str [Required]#
field resource_bindings_url: str [Required]#
field resource_group_crn: str [Required]#
field resource_group_id: str [Required]#
field resource_id: str [Required]#
field resource_keys_url: str [Required]#
field resource_plan_id: str [Required]#
field restored_at: str | None [Required]#
field restored_by: str [Required]#
field scheduled_reclaim_at: str | None [Required]#
field scheduled_reclaim_by: str [Required]#
field service_instance_id: str [Optional]#
field state: str [Required]#
field target_crn: str [Required]#
field type: str [Required]#
field updated_at: str [Required]#
field updated_by: str [Required]#
field url: str [Required]#
class ibm_watsonx_data_integration.cpd_models.service_model.Services(platform: Platform)#

Bases: CollectionModel[Service]

Collection of Service instances.

The __init__ of the Services class.

Parameters:

platform (Platform) – The Platform object.

Service Id#

Module containing Service ID Model.

pydantic model ibm_watsonx_data_integration.cpd_models.service_id_model.ServiceID#

Bases: BaseModel, ProjectCollaboratable

Model representing a Service ID.

The __init__ of the ServiceID Wrapper class.

Parameters:

  • service_id_json (dict) – The JSON for the Service ID.

  • platform (Optional[Platform], default: None) – The Platform object. Default: None

field account_id: str [Required]#
field created_at: str [Required]#
field crn: str [Required]#
field entity_tag: str [Required]#
field iam_id: str [Required]#
field id: str [Required]#
field locked: bool [Required]#
field modified_at: str [Required]#
field name: str [Required]#
field service_id: str [Optional]#
property type: str#

This property returns the member type “service”.

Returns:

The member type.

class ibm_watsonx_data_integration.cpd_models.service_id_model.ServiceIDs(platform: Platform | None = None)#

Bases: CollectionModel[ServiceID]

Collection of ServiceID instances.

The __init__ of the ServiceIDs class.

Parameters:

platform (Optional[Platform], default: None) – The Platform object.

Streaming Flow#

Module containing Streaming Flow Models.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.FlowValidationError#

Bases: BaseModel

Streaming validation error object.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field errorCode: str [Required]#
field humanReadableMessage: str [Required]#
field instanceName: str | None = None#
field level: str [Required]#
field message: str [Required]#
field technicalMessage: str [Required]#
field type: str [Required]#
classmethod construct(_fields_set: set[str] | None = None, **values: Any)#
Return type:

Self

copy(*, include: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, exclude: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, update: Dict[str, Any] | None = None, deep: bool = False)#

Returns a copy of the model.

!!! warning “Deprecated”

This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`python {test="skip" lint="skip"} data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

  • include (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to include in the copied model.

  • exclude (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to exclude in the copied model.

  • update (Optional[Dict[str, Any]], default: None) – Optional dictionary of field-value pairs to override field values in the copied model.

  • deep (bool, default: False) – If True, the values of fields that are Pydantic models will be deep-copied.

Return type:

Self

Returns:

A copy of the model with included, excluded and updated fields as specified.

dict(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False)#
Return type:

Dict[str, Any]

validator entity_validation  »  all fields#

Flattens/Manipulates the data.

Parameters:

data (dict) – Data to be manipulated on

Return type:

dict

Returns:

A dict with manipulated data.

classmethod from_orm(obj: Any)#
Return type:

Self

json(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Callable[[Any], Any] | None = PydanticUndefined, models_as_dict: bool = PydanticUndefined, **dumps_kwargs: Any)#
Return type:

str

classmethod model_construct(_fields_set: set[str] | None = None, **values: Any)#

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed.

!!! note

model_construct() generally respects the model_config.extra setting on the provided model. That is, if model_config.extra == ‘allow’, then all extra passed values are added to the model instance’s __dict__ and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored. Because no validation is performed with a call to model_construct(), having model_config.extra == ‘forbid’ does not result in an error if extra values are passed, but they will be ignored.

Parameters:

  • _fields_set (Optional[set[str]], default: None) – A set of field names that were originally explicitly set during instantiation. If provided, this is directly used for the [model_fields_set][pydantic.BaseModel.model_fields_set] attribute. Otherwise, the field names from the values argument will be used.

  • values (Any) – Trusted or pre-validated data dictionary.

Return type:

Self

Returns:

A new instance of the Model class with validated data.

model_copy(*, update: Mapping[str, Any] | None = None, deep: bool = False)#
!!! abstract “Usage Documentation”

[model_copy](../concepts/models.md#model-copy)

Returns a copy of the model.

!!! note

The underlying instance’s [__dict__][object.__dict__] attribute is copied. This might have unexpected side effects if you store anything in it, on top of the model fields (e.g. the value of [cached properties][functools.cached_property]).

Parameters:

  • update (Optional[Mapping[str, Any]], default: None) – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.

  • deep (bool, default: False) – Set to True to make a deep copy of the model.

Return type:

Self

Returns:

New model instance.

model_dump(by_alias: bool = True, **kwargs: any)#

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Return type:

dict

Returns:

A dictionary representation of the model.

model_dump_json(*, indent: int | None = None, ensure_ascii: bool = False, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, context: Any | None = None, by_alias: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, exclude_computed_fields: bool = False, round_trip: bool = False, warnings: bool | Literal['none', 'warn', 'error'] = True, fallback: Callable[[Any], Any] | None = None, serialize_as_any: bool = False)#
!!! abstract “Usage Documentation”

[model_dump_json](../concepts/serialization.md#json-mode)

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

  • indent (Optional[int], default: None) – Indentation to use in the JSON output. If None is passed, the output will be compact.

  • ensure_ascii (bool, default: False) – If True, the output is guaranteed to have all incoming non-ASCII characters escaped. If False (the default), these characters will be output as-is.

  • include (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to include in the JSON output.

  • exclude (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to exclude from the JSON output.

  • context (Optional[Any], default: None) – Additional context to pass to the serializer.

  • by_alias (Optional[bool], default: None) – Whether to serialize using field aliases.

  • exclude_unset (bool, default: False) – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults (bool, default: False) – Whether to exclude fields that are set to their default value.

  • exclude_none (bool, default: False) – Whether to exclude fields that have a value of None.

  • exclude_computed_fields (bool, default: False) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.

  • round_trip (bool, default: False) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings (Union[bool, Literal['none', 'warn', 'error']], default: True) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].

  • fallback (Optional[Callable[[Any], Any]], default: None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.

  • serialize_as_any (bool, default: False) – Whether to serialize fields with duck-typing serialization behavior.

Return type:

str

Returns:

A JSON string representation of the model.

classmethod model_json_schema(**kwargs: Any)#

Generate a JSON schema for the model.

This method overrides the Pydantic base method to provide a properly formatted docstring that avoids Sphinx/docutils warnings about bullet list formatting.

Parameters:

**kwargs (Any) – Additional keyword arguments passed to the parent method.

Return type:

dict

Returns:

A dictionary containing the JSON schema for the model.

classmethod model_parametrized_name(params: tuple[type[Any], ...])#

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:

params (tuple[type[Any], ...]) – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.

Return type:

str

Returns:

String representing the new class where params are passed to cls as type variables.

Raises:

TypeError – Raised when trying to generate concrete names for non-generic models.

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

classmethod model_rebuild(*, force: bool = False, raise_errors: bool = True, _parent_namespace_depth: int = 2, _types_namespace: Mapping[str, Any] | None = None)#

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

  • force (bool, default: False) – Whether to force the rebuilding of the model schema, defaults to False.

  • raise_errors (bool, default: True) – Whether to raise errors, defaults to True.

  • _parent_namespace_depth (int, default: 2) – The depth level of the parent namespace, defaults to 2.

  • _types_namespace (Optional[Mapping[str, Any]], default: None) – The types namespace, defaults to None.

Return type:

bool | None

Returns:

Returns None if the schema is already “complete” and rebuilding was not required. If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False.

classmethod model_validate(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, from_attributes: bool | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate a pydantic model instance.

Parameters:

  • obj (Any) – The object to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • from_attributes (Optional[bool], default: None) – Whether to extract data from object attributes.

  • context (Optional[Any], default: None) – Additional context to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Raises:

ValidationError – If the object could not be validated.

Return type:

Self

Returns:

The validated model instance.

classmethod model_validate_json(json_data: str | bytes | bytearray, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#
!!! abstract “Usage Documentation”

[JSON Parsing](../concepts/json.md#json-parsing)

Validate the given JSON data against the Pydantic model.

Parameters:

  • json_data (str | bytes | bytearray) – The JSON data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

Raises:

ValidationError – If json_data is not a JSON string or the object could not be validated.

classmethod model_validate_strings(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate the given object with string data against the Pydantic model.

Parameters:

  • obj (Any) – The object containing string data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

classmethod parse_file(path: str | Path, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod parse_obj(obj: Any)#
Return type:

Self

classmethod parse_raw(b: str | bytes, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}')#
Return type:

Dict[str, Any]

classmethod schema_json(*, by_alias: bool = True, ref_template: str = '#/$defs/{model}', **dumps_kwargs: Any)#
Return type:

str

serialize_model(handler: Any, info: SerializationInfo)#

Serializes the data, back into it’s original formatting.

Return type:

dict

Returns:

A dict with updated information

classmethod update_forward_refs(**localns: Any)#
Return type:

None

classmethod validate(value: Any)#
Return type:

Self

property model_extra: dict[str, Any] | None#

Get extra fields set during validation.

Returns:

A dictionary of extra fields, or None if config.extra is not set to “allow”.

property model_fields_set: set[str]#

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,

i.e. that were not filled from defaults.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.PipelineDefinition#

Bases: BaseModel

Pipeline Definition of a flow in an engine.

The __init__ for the pipeline definition.

Parameters:

  • flow (StreamingFlow) – The parent flow object

  • pipeline_definition_json (any) – Pipeline definition from the engine.

field configuration_data: list[dict] [Required] (alias 'configuration')#
field description: str [Required]#
field error_stage_data: dict | None = None (alias 'errorStage')#
field fragments: list [Required]#
field info: dict [Required]#
field issues: dict [Required]#
field metadata: dict | None = None#
field pipeline_id: str [Required] (alias 'pipelineId')#
field previewable: bool [Required]#
field schema_version: int [Required] (alias 'schemaVersion')#
field stages_data: list[dict] [Required] (alias 'stages')#
field start_event_stages_data: list[dict] | None = None (alias 'startEventStages')#
field stats_aggregator_stage_data: dict | None = None (alias 'statsAggregatorStage')#
field stop_event_stages_data: list[dict] | None = None (alias 'stopEventStages')#
field test_origin_stages_data: dict | None = None (alias 'testOriginStage')#
field title: str [Required]#
field ui_info: dict [Required] (alias 'uiInfo')#
field uuid: str [Required]#
field valid: bool [Required]#
field version: int [Required]#
add_stage(label: str | None = None, name: str | None = None, type: str | None = None, library: str | None = None)#

Add a new stage to the pipeline.

Parameters:

  • label (Optional[str], default: None) – Label of the stage

  • name (Optional[str], default: None) – Name of the stage

  • type (Optional[str], default: None) – Type of stage

  • library (Optional[str], default: None) – Library the stage belongs to

Return type:

Stage | StageWithPredicates

classmethod construct(_fields_set: set[str] | None = None, **values: Any)#
Return type:

Self

copy(*, include: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, exclude: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, update: Dict[str, Any] | None = None, deep: bool = False)#

Returns a copy of the model.

!!! warning “Deprecated”

This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`python {test="skip" lint="skip"} data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

  • include (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to include in the copied model.

  • exclude (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to exclude in the copied model.

  • update (Optional[Dict[str, Any]], default: None) – Optional dictionary of field-value pairs to override field values in the copied model.

  • deep (bool, default: False) – If True, the values of fields that are Pydantic models will be deep-copied.

Return type:

Self

Returns:

A copy of the model with included, excluded and updated fields as specified.

dict(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False)#
Return type:

Dict[str, Any]

duplicate_stage(stage: Stage)#

Duplicate a stage.

Parameters:

stage (Stage) – Stage to duplicate.

Return type:

Stage

validator entity_validation  »  all fields#

Flattens/Manipulates the data.

Parameters:

data (dict) – Data to be manipulated on

Return type:

dict

Returns:

A dict with manipulated data.

classmethod from_orm(obj: Any)#
Return type:

Self

static get_attribute(config_definition: dict)#

Gets the attribute name for a configuration using its definition in a human-readable format.

Return type:

tuple[str, str]

static get_color_icon_from_stage_definition(stage_definition: dict, stage_types: dict)#

Create colorIcon value from stage definition.

Parameters:

  • stage_definition (dict) – stage definition from pipeline definitions

  • stage_types (dict) – The PipelineDefinition._NODE_TYPES

Return type:

str

json(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Callable[[Any], Any] | None = PydanticUndefined, models_as_dict: bool = PydanticUndefined, **dumps_kwargs: Any)#
Return type:

str

classmethod model_construct(_fields_set: set[str] | None = None, **values: Any)#

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed.

!!! note

model_construct() generally respects the model_config.extra setting on the provided model. That is, if model_config.extra == ‘allow’, then all extra passed values are added to the model instance’s __dict__ and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored. Because no validation is performed with a call to model_construct(), having model_config.extra == ‘forbid’ does not result in an error if extra values are passed, but they will be ignored.

Parameters:

  • _fields_set (Optional[set[str]], default: None) – A set of field names that were originally explicitly set during instantiation. If provided, this is directly used for the [model_fields_set][pydantic.BaseModel.model_fields_set] attribute. Otherwise, the field names from the values argument will be used.

  • values (Any) – Trusted or pre-validated data dictionary.

Return type:

Self

Returns:

A new instance of the Model class with validated data.

model_copy(*, update: Mapping[str, Any] | None = None, deep: bool = False)#
!!! abstract “Usage Documentation”

[model_copy](../concepts/models.md#model-copy)

Returns a copy of the model.

!!! note

The underlying instance’s [__dict__][object.__dict__] attribute is copied. This might have unexpected side effects if you store anything in it, on top of the model fields (e.g. the value of [cached properties][functools.cached_property]).

Parameters:

  • update (Optional[Mapping[str, Any]], default: None) – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.

  • deep (bool, default: False) – Set to True to make a deep copy of the model.

Return type:

Self

Returns:

New model instance.

model_dump(by_alias: bool = True, **kwargs: any)#

Ensure stages are properly arranged when pipeline definition is stringified.

Return type:

dict

model_dump_json(*, indent: int | None = None, ensure_ascii: bool = False, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, context: Any | None = None, by_alias: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, exclude_computed_fields: bool = False, round_trip: bool = False, warnings: bool | Literal['none', 'warn', 'error'] = True, fallback: Callable[[Any], Any] | None = None, serialize_as_any: bool = False)#
!!! abstract “Usage Documentation”

[model_dump_json](../concepts/serialization.md#json-mode)

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

  • indent (Optional[int], default: None) – Indentation to use in the JSON output. If None is passed, the output will be compact.

  • ensure_ascii (bool, default: False) – If True, the output is guaranteed to have all incoming non-ASCII characters escaped. If False (the default), these characters will be output as-is.

  • include (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to include in the JSON output.

  • exclude (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to exclude from the JSON output.

  • context (Optional[Any], default: None) – Additional context to pass to the serializer.

  • by_alias (Optional[bool], default: None) – Whether to serialize using field aliases.

  • exclude_unset (bool, default: False) – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults (bool, default: False) – Whether to exclude fields that are set to their default value.

  • exclude_none (bool, default: False) – Whether to exclude fields that have a value of None.

  • exclude_computed_fields (bool, default: False) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.

  • round_trip (bool, default: False) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings (Union[bool, Literal['none', 'warn', 'error']], default: True) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].

  • fallback (Optional[Callable[[Any], Any]], default: None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.

  • serialize_as_any (bool, default: False) – Whether to serialize fields with duck-typing serialization behavior.

Return type:

str

Returns:

A JSON string representation of the model.

classmethod model_json_schema(**kwargs: Any)#

Generate a JSON schema for the model.

This method overrides the Pydantic base method to provide a properly formatted docstring that avoids Sphinx/docutils warnings about bullet list formatting.

Parameters:

**kwargs (Any) – Additional keyword arguments passed to the parent method.

Return type:

dict

Returns:

A dictionary containing the JSON schema for the model.

classmethod model_parametrized_name(params: tuple[type[Any], ...])#

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:

params (tuple[type[Any], ...]) – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.

Return type:

str

Returns:

String representing the new class where params are passed to cls as type variables.

Raises:

TypeError – Raised when trying to generate concrete names for non-generic models.

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

classmethod model_rebuild(*, force: bool = False, raise_errors: bool = True, _parent_namespace_depth: int = 2, _types_namespace: Mapping[str, Any] | None = None)#

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

  • force (bool, default: False) – Whether to force the rebuilding of the model schema, defaults to False.

  • raise_errors (bool, default: True) – Whether to raise errors, defaults to True.

  • _parent_namespace_depth (int, default: 2) – The depth level of the parent namespace, defaults to 2.

  • _types_namespace (Optional[Mapping[str, Any]], default: None) – The types namespace, defaults to None.

Return type:

bool | None

Returns:

Returns None if the schema is already “complete” and rebuilding was not required. If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False.

classmethod model_validate(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, from_attributes: bool | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate a pydantic model instance.

Parameters:

  • obj (Any) – The object to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • from_attributes (Optional[bool], default: None) – Whether to extract data from object attributes.

  • context (Optional[Any], default: None) – Additional context to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Raises:

ValidationError – If the object could not be validated.

Return type:

Self

Returns:

The validated model instance.

classmethod model_validate_json(json_data: str | bytes | bytearray, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#
!!! abstract “Usage Documentation”

[JSON Parsing](../concepts/json.md#json-parsing)

Validate the given JSON data against the Pydantic model.

Parameters:

  • json_data (str | bytes | bytearray) – The JSON data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

Raises:

ValidationError – If json_data is not a JSON string or the object could not be validated.

classmethod model_validate_strings(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate the given object with string data against the Pydantic model.

Parameters:

  • obj (Any) – The object containing string data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

classmethod parse_file(path: str | Path, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod parse_obj(obj: Any)#
Return type:

Self

classmethod parse_raw(b: str | bytes, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

remove_stage(stage: Stage)#

Removed a stage from the flow.

Parameters:

stage (Stage) – Stage to remove.

Return type:

None

classmethod schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}')#
Return type:

Dict[str, Any]

classmethod schema_json(*, by_alias: bool = True, ref_template: str = '#/$defs/{model}', **dumps_kwargs: Any)#
Return type:

str

serialize_model(handler: Any, info: SerializationInfo)#

Serializes the data, back into it’s original formatting.

Return type:

dict

Returns:

A dict with updated information

set_error_stage(label: str | None = None, name: str | None = None, library: str | None = None)#

Set the error stage of the flow.

Parameters:

  • label (Optional[str], default: None) – Label of the stage.

  • name (Optional[str], default: None) – Name of the stage.

  • library (Optional[str], default: None) – Library the stage belongs to.

Return type:

Stage

set_start_event_stage(label: str | None = None, name: str | None = None, library: str | None = None)#

Set the start event stage of the flow.

Parameters:

  • label (Optional[str], default: None) – Label of the stage.

  • name (Optional[str], default: None) – Name of the stage.

  • library (Optional[str], default: None) – Library the stage belongs to.

Return type:

Stage

set_stats_aggregator_stage(label: str | None = None, name: str | None = None, library: str | None = None)#

Set the stats aggregator stage of the flow.

Parameters:

  • label (Optional[str], default: None) – Label of the stage.

  • name (Optional[str], default: None) – Name of the stage.

  • library (Optional[str], default: None) – Library the stage belongs to.

Return type:

Stage

set_stop_event_stage(label: str | None = None, name: str | None = None, library: str | None = None)#

Set the stop event stage of the flow.

Parameters:

  • label (Optional[str], default: None) – Label of the stage.

  • name (Optional[str], default: None) – Name of the stage.

  • library (Optional[str], default: None) – Library the stage belongs to.

Return type:

Stage

set_test_origin_stage(label: str | None = None, name: str | None = None, library: str | None = None)#

Set the test origin stage of the flow.

Parameters:

  • label (Optional[str], default: None) – Label of the stage.

  • name (Optional[str], default: None) – Name of the stage.

  • library (Optional[str], default: None) – Library the stage belongs to.

Return type:

Stage

static stage_configuration_name(stage_json: dict)#

Gets a stages configuration name, the name it has in a flow’s configuration.

Return type:

str

classmethod update_forward_refs(**localns: Any)#
Return type:

None

classmethod validate(value: Any)#
Return type:

Self

property configuration: Configuration#

The configuration for the pipeline.

property error_stage: Stage | None#

The error stage of the flow.

property model_extra: dict[str, Any] | None#

Get extra fields set during validation.

Returns:

A dictionary of extra fields, or None if config.extra is not set to “allow”.

property model_fields_set: set[str]#

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,

i.e. that were not filled from defaults.

property stages: Stages#

The stages belonging to the flow.

property start_event: Stage | None#

The start event stage of the flow.

property stats_aggregator_stage: Stage | None#

The stats aggregator stage of the flow.

property stop_event: Stage | None#

The stop event stage of the flow.

property test_origin: Stage | None#

The test origin stage of the flow.

class ibm_watsonx_data_integration.services.streamsets.models.flow_model.PreviewRunStage(value)#

Bases: Enum

Enum for preview run stage options.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.PreviewStage#

Bases: BaseModel

Class for a stage preview.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field input: list | None [Required]#
field instance_name: str [Required] (alias 'instanceName')#
field output: list | None [Required]#
field stage_name: str [Required] (alias 'stageName')#
classmethod construct(_fields_set: set[str] | None = None, **values: Any)#
Return type:

Self

copy(*, include: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, exclude: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, update: Dict[str, Any] | None = None, deep: bool = False)#

Returns a copy of the model.

!!! warning “Deprecated”

This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`python {test="skip" lint="skip"} data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

  • include (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to include in the copied model.

  • exclude (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to exclude in the copied model.

  • update (Optional[Dict[str, Any]], default: None) – Optional dictionary of field-value pairs to override field values in the copied model.

  • deep (bool, default: False) – If True, the values of fields that are Pydantic models will be deep-copied.

Return type:

Self

Returns:

A copy of the model with included, excluded and updated fields as specified.

dict(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False)#
Return type:

Dict[str, Any]

validator entity_validation  »  all fields#

Flattens/Manipulates the data.

Parameters:

data (dict) – Data to be manipulated on

Return type:

dict

Returns:

A dict with manipulated data.

classmethod from_orm(obj: Any)#
Return type:

Self

json(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Callable[[Any], Any] | None = PydanticUndefined, models_as_dict: bool = PydanticUndefined, **dumps_kwargs: Any)#
Return type:

str

classmethod model_construct(_fields_set: set[str] | None = None, **values: Any)#

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed.

!!! note

model_construct() generally respects the model_config.extra setting on the provided model. That is, if model_config.extra == ‘allow’, then all extra passed values are added to the model instance’s __dict__ and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored. Because no validation is performed with a call to model_construct(), having model_config.extra == ‘forbid’ does not result in an error if extra values are passed, but they will be ignored.

Parameters:

  • _fields_set (Optional[set[str]], default: None) – A set of field names that were originally explicitly set during instantiation. If provided, this is directly used for the [model_fields_set][pydantic.BaseModel.model_fields_set] attribute. Otherwise, the field names from the values argument will be used.

  • values (Any) – Trusted or pre-validated data dictionary.

Return type:

Self

Returns:

A new instance of the Model class with validated data.

model_copy(*, update: Mapping[str, Any] | None = None, deep: bool = False)#
!!! abstract “Usage Documentation”

[model_copy](../concepts/models.md#model-copy)

Returns a copy of the model.

!!! note

The underlying instance’s [__dict__][object.__dict__] attribute is copied. This might have unexpected side effects if you store anything in it, on top of the model fields (e.g. the value of [cached properties][functools.cached_property]).

Parameters:

  • update (Optional[Mapping[str, Any]], default: None) – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.

  • deep (bool, default: False) – Set to True to make a deep copy of the model.

Return type:

Self

Returns:

New model instance.

model_dump(by_alias: bool = True, **kwargs: any)#

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Return type:

dict

Returns:

A dictionary representation of the model.

model_dump_json(*, indent: int | None = None, ensure_ascii: bool = False, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, context: Any | None = None, by_alias: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, exclude_computed_fields: bool = False, round_trip: bool = False, warnings: bool | Literal['none', 'warn', 'error'] = True, fallback: Callable[[Any], Any] | None = None, serialize_as_any: bool = False)#
!!! abstract “Usage Documentation”

[model_dump_json](../concepts/serialization.md#json-mode)

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

  • indent (Optional[int], default: None) – Indentation to use in the JSON output. If None is passed, the output will be compact.

  • ensure_ascii (bool, default: False) – If True, the output is guaranteed to have all incoming non-ASCII characters escaped. If False (the default), these characters will be output as-is.

  • include (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to include in the JSON output.

  • exclude (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to exclude from the JSON output.

  • context (Optional[Any], default: None) – Additional context to pass to the serializer.

  • by_alias (Optional[bool], default: None) – Whether to serialize using field aliases.

  • exclude_unset (bool, default: False) – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults (bool, default: False) – Whether to exclude fields that are set to their default value.

  • exclude_none (bool, default: False) – Whether to exclude fields that have a value of None.

  • exclude_computed_fields (bool, default: False) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.

  • round_trip (bool, default: False) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings (Union[bool, Literal['none', 'warn', 'error']], default: True) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].

  • fallback (Optional[Callable[[Any], Any]], default: None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.

  • serialize_as_any (bool, default: False) – Whether to serialize fields with duck-typing serialization behavior.

Return type:

str

Returns:

A JSON string representation of the model.

classmethod model_json_schema(**kwargs: Any)#

Generate a JSON schema for the model.

This method overrides the Pydantic base method to provide a properly formatted docstring that avoids Sphinx/docutils warnings about bullet list formatting.

Parameters:

**kwargs (Any) – Additional keyword arguments passed to the parent method.

Return type:

dict

Returns:

A dictionary containing the JSON schema for the model.

classmethod model_parametrized_name(params: tuple[type[Any], ...])#

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:

params (tuple[type[Any], ...]) – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.

Return type:

str

Returns:

String representing the new class where params are passed to cls as type variables.

Raises:

TypeError – Raised when trying to generate concrete names for non-generic models.

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

classmethod model_rebuild(*, force: bool = False, raise_errors: bool = True, _parent_namespace_depth: int = 2, _types_namespace: Mapping[str, Any] | None = None)#

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

  • force (bool, default: False) – Whether to force the rebuilding of the model schema, defaults to False.

  • raise_errors (bool, default: True) – Whether to raise errors, defaults to True.

  • _parent_namespace_depth (int, default: 2) – The depth level of the parent namespace, defaults to 2.

  • _types_namespace (Optional[Mapping[str, Any]], default: None) – The types namespace, defaults to None.

Return type:

bool | None

Returns:

Returns None if the schema is already “complete” and rebuilding was not required. If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False.

classmethod model_validate(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, from_attributes: bool | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate a pydantic model instance.

Parameters:

  • obj (Any) – The object to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • from_attributes (Optional[bool], default: None) – Whether to extract data from object attributes.

  • context (Optional[Any], default: None) – Additional context to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Raises:

ValidationError – If the object could not be validated.

Return type:

Self

Returns:

The validated model instance.

classmethod model_validate_json(json_data: str | bytes | bytearray, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#
!!! abstract “Usage Documentation”

[JSON Parsing](../concepts/json.md#json-parsing)

Validate the given JSON data against the Pydantic model.

Parameters:

  • json_data (str | bytes | bytearray) – The JSON data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

Raises:

ValidationError – If json_data is not a JSON string or the object could not be validated.

classmethod model_validate_strings(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate the given object with string data against the Pydantic model.

Parameters:

  • obj (Any) – The object containing string data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

classmethod parse_file(path: str | Path, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod parse_obj(obj: Any)#
Return type:

Self

classmethod parse_raw(b: str | bytes, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}')#
Return type:

Dict[str, Any]

classmethod schema_json(*, by_alias: bool = True, ref_template: str = '#/$defs/{model}', **dumps_kwargs: Any)#
Return type:

str

serialize_model(handler: Any, info: SerializationInfo)#

Serializes the data, back into it’s original formatting.

Return type:

dict

Returns:

A dict with updated information

classmethod update_forward_refs(**localns: Any)#
Return type:

None

classmethod validate(value: Any)#
Return type:

Self

property model_extra: dict[str, Any] | None#

Get extra fields set during validation.

Returns:

A dictionary of extra fields, or None if config.extra is not set to “allow”.

property model_fields_set: set[str]#

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,

i.e. that were not filled from defaults.

class ibm_watsonx_data_integration.services.streamsets.models.flow_model.PreviewTimezone(value)#

Bases: Enum

Enum for preview timezone options.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.Stage#

Bases: BaseModel

Class for a stage.

The __init__ for the class.

Parameters:

  • pipeline_definition (PipelineDefinition) – Pipeline Definition of the flow this stage belongs to

  • output_streams (int, default: 0) – The number of output streams for the stage, if 0 then it is inferred from output lanes

  • supported_connection_types (Optional[list[str]], default: None) – The connection types supported by the stage.

  • attributes (Optional[dict], default: None) – Any attributes specific to the stage.

  • stage_json (any) – All other properties used by BaseModel

field configuration_data: list[dict] [Required] (alias 'configuration')#
field event_lanes: list[str] = [] (alias 'eventLanes')#
field input_lanes: list[str] = [] (alias 'inputLanes')#
field instance_name: str [Required] (alias 'instanceName')#
field library: str [Required]#
field output_lanes: list[str] = [] (alias 'outputLanes')#
field services_data: list[dict] [Required] (alias 'services')#
field stage_id: str [Required] (alias 'stageName')#
field stage_version: str [Required] (alias 'stageVersion')#
field ui_info: dict [Required] (alias 'uiInfo')#
connect_event_to(*stages: Stage)#

Connect other stages to the event output of this stage.

Parameters:

stages (Stage) – Stages to connect to this stage

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

connect_input_to(*stages: Stage, predicate: dict | None = None)#

Connect other stages to the input of this stage.

Parameters:

  • stages (Stage) – Stages to connect to this stage

  • predicate (Optional[dict], default: None) – Whether to use any specific predicate when connecting (only applicable to some stages).

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

connect_output_to(*stages: Stage, predicate: dict | None = None)#

Connect other stages to the output of this stage.

Parameters:

  • stages (Stage) – Stages to connect to this stage

  • predicate (Optional[dict], default: None) – Whether to use any specific predicate when connecting (only applicable to some stages).

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

classmethod construct(_fields_set: set[str] | None = None, **values: Any)#
Return type:

Self

copy(*, include: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, exclude: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, update: Dict[str, Any] | None = None, deep: bool = False)#

Returns a copy of the model.

!!! warning “Deprecated”

This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`python {test="skip" lint="skip"} data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

  • include (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to include in the copied model.

  • exclude (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to exclude in the copied model.

  • update (Optional[Dict[str, Any]], default: None) – Optional dictionary of field-value pairs to override field values in the copied model.

  • deep (bool, default: False) – If True, the values of fields that are Pydantic models will be deep-copied.

Return type:

Self

Returns:

A copy of the model with included, excluded and updated fields as specified.

dict(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False)#
Return type:

Dict[str, Any]

disconnect_event_from(*stages: Stage)#

Disconnect the event output of this stage from the provided stages.

Parameters:

stages (Stage) – Stages to disconnect from this stage.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

disconnect_input_from(*stages: Stage)#

Disconnect the input of this stage from the output of the provided stages.

Parameters:

stages (Stage) – Stages to disconnect from this stage.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

disconnect_output_from(*stages: Stage)#

Disconnect the output of this stage from the provided stages.

Parameters:

stages (Stage) – Stages to disconnect from this stage.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

validator entity_validation  »  all fields#

Flattens/Manipulates the data.

Parameters:

data (dict) – Data to be manipulated on

Return type:

dict

Returns:

A dict with manipulated data.

classmethod from_orm(obj: Any)#
Return type:

Self

json(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Callable[[Any], Any] | None = PydanticUndefined, models_as_dict: bool = PydanticUndefined, **dumps_kwargs: Any)#
Return type:

str

classmethod model_construct(_fields_set: set[str] | None = None, **values: Any)#

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed.

!!! note

model_construct() generally respects the model_config.extra setting on the provided model. That is, if model_config.extra == ‘allow’, then all extra passed values are added to the model instance’s __dict__ and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored. Because no validation is performed with a call to model_construct(), having model_config.extra == ‘forbid’ does not result in an error if extra values are passed, but they will be ignored.

Parameters:

  • _fields_set (Optional[set[str]], default: None) – A set of field names that were originally explicitly set during instantiation. If provided, this is directly used for the [model_fields_set][pydantic.BaseModel.model_fields_set] attribute. Otherwise, the field names from the values argument will be used.

  • values (Any) – Trusted or pre-validated data dictionary.

Return type:

Self

Returns:

A new instance of the Model class with validated data.

model_copy(*, update: Mapping[str, Any] | None = None, deep: bool = False)#
!!! abstract “Usage Documentation”

[model_copy](../concepts/models.md#model-copy)

Returns a copy of the model.

!!! note

The underlying instance’s [__dict__][object.__dict__] attribute is copied. This might have unexpected side effects if you store anything in it, on top of the model fields (e.g. the value of [cached properties][functools.cached_property]).

Parameters:

  • update (Optional[Mapping[str, Any]], default: None) – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.

  • deep (bool, default: False) – Set to True to make a deep copy of the model.

Return type:

Self

Returns:

New model instance.

model_dump(by_alias: bool = True, **kwargs: any)#

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Return type:

dict

Returns:

A dictionary representation of the model.

model_dump_json(*, indent: int | None = None, ensure_ascii: bool = False, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, context: Any | None = None, by_alias: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, exclude_computed_fields: bool = False, round_trip: bool = False, warnings: bool | Literal['none', 'warn', 'error'] = True, fallback: Callable[[Any], Any] | None = None, serialize_as_any: bool = False)#
!!! abstract “Usage Documentation”

[model_dump_json](../concepts/serialization.md#json-mode)

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

  • indent (Optional[int], default: None) – Indentation to use in the JSON output. If None is passed, the output will be compact.

  • ensure_ascii (bool, default: False) – If True, the output is guaranteed to have all incoming non-ASCII characters escaped. If False (the default), these characters will be output as-is.

  • include (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to include in the JSON output.

  • exclude (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to exclude from the JSON output.

  • context (Optional[Any], default: None) – Additional context to pass to the serializer.

  • by_alias (Optional[bool], default: None) – Whether to serialize using field aliases.

  • exclude_unset (bool, default: False) – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults (bool, default: False) – Whether to exclude fields that are set to their default value.

  • exclude_none (bool, default: False) – Whether to exclude fields that have a value of None.

  • exclude_computed_fields (bool, default: False) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.

  • round_trip (bool, default: False) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings (Union[bool, Literal['none', 'warn', 'error']], default: True) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].

  • fallback (Optional[Callable[[Any], Any]], default: None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.

  • serialize_as_any (bool, default: False) – Whether to serialize fields with duck-typing serialization behavior.

Return type:

str

Returns:

A JSON string representation of the model.

classmethod model_json_schema(**kwargs: Any)#

Generate a JSON schema for the model.

This method overrides the Pydantic base method to provide a properly formatted docstring that avoids Sphinx/docutils warnings about bullet list formatting.

Parameters:

**kwargs (Any) – Additional keyword arguments passed to the parent method.

Return type:

dict

Returns:

A dictionary containing the JSON schema for the model.

classmethod model_parametrized_name(params: tuple[type[Any], ...])#

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:

params (tuple[type[Any], ...]) – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.

Return type:

str

Returns:

String representing the new class where params are passed to cls as type variables.

Raises:

TypeError – Raised when trying to generate concrete names for non-generic models.

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

classmethod model_rebuild(*, force: bool = False, raise_errors: bool = True, _parent_namespace_depth: int = 2, _types_namespace: Mapping[str, Any] | None = None)#

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

  • force (bool, default: False) – Whether to force the rebuilding of the model schema, defaults to False.

  • raise_errors (bool, default: True) – Whether to raise errors, defaults to True.

  • _parent_namespace_depth (int, default: 2) – The depth level of the parent namespace, defaults to 2.

  • _types_namespace (Optional[Mapping[str, Any]], default: None) – The types namespace, defaults to None.

Return type:

bool | None

Returns:

Returns None if the schema is already “complete” and rebuilding was not required. If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False.

classmethod model_validate(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, from_attributes: bool | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate a pydantic model instance.

Parameters:

  • obj (Any) – The object to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • from_attributes (Optional[bool], default: None) – Whether to extract data from object attributes.

  • context (Optional[Any], default: None) – Additional context to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Raises:

ValidationError – If the object could not be validated.

Return type:

Self

Returns:

The validated model instance.

classmethod model_validate_json(json_data: str | bytes | bytearray, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#
!!! abstract “Usage Documentation”

[JSON Parsing](../concepts/json.md#json-parsing)

Validate the given JSON data against the Pydantic model.

Parameters:

  • json_data (str | bytes | bytearray) – The JSON data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

Raises:

ValidationError – If json_data is not a JSON string or the object could not be validated.

classmethod model_validate_strings(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate the given object with string data against the Pydantic model.

Parameters:

  • obj (Any) – The object containing string data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

classmethod parse_file(path: str | Path, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod parse_obj(obj: Any)#
Return type:

Self

classmethod parse_raw(b: str | bytes, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}')#
Return type:

Dict[str, Any]

classmethod schema_json(*, by_alias: bool = True, ref_template: str = '#/$defs/{model}', **dumps_kwargs: Any)#
Return type:

str

serialize_model(handler: Any, info: SerializationInfo)#

Serializes the data, back into it’s original formatting.

Return type:

dict

Returns:

A dict with updated information

classmethod update_forward_refs(**localns: Any)#
Return type:

None

classmethod validate(value: Any)#
Return type:

Self

property configuration: Configuration#

The stage’s configurations.

property events: Stages#

All the stages connected to this stages event output.

property inputs: Stages#

All the stages connected to this stages input.

property model_extra: dict[str, Any] | None#

Get extra fields set during validation.

Returns:

A dictionary of extra fields, or None if config.extra is not set to “allow”.

property model_fields_set: set[str]#

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,

i.e. that were not filled from defaults.

property outputs: Stages#

All the stages connected to this stages output.

property stage_name: str#

Getting the stage instance’s name.

property type: str#

The type of this stage.

ibm_watsonx_data_integration.services.streamsets.models.flow_model.StageClassDefault#

alias of StageClassDefaults

class ibm_watsonx_data_integration.services.streamsets.models.flow_model.StageConfigurationProperty(config_name)#

Bases: tuple

Create new instance of StageConfigurationProperty(config_name,)

config_name#

Alias for field number 0

count(value, /)#

Return number of occurrences of value.

index(value, start=0, stop=9223372036854775807, /)#

Return first index of value.

Raises ValueError if the value is not present.

class ibm_watsonx_data_integration.services.streamsets.models.flow_model.StageData(definition, instance)#

Bases: tuple

Create new instance of StageData(definition, instance)

count(value, /)#

Return number of occurrences of value.

definition#

Alias for field number 0

index(value, start=0, stop=9223372036854775807, /)#

Return first index of value.

Raises ValueError if the value is not present.

instance#

Alias for field number 1

class ibm_watsonx_data_integration.services.streamsets.models.flow_model.StageServiceProperty(service_name, config_name)#

Bases: tuple

Create new instance of StageServiceProperty(service_name, config_name)

config_name#

Alias for field number 1

count(value, /)#

Return number of occurrences of value.

index(value, start=0, stop=9223372036854775807, /)#

Return first index of value.

Raises ValueError if the value is not present.

service_name#

Alias for field number 0

pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.StageWithPredicates#

Bases: Stage

Class for a stage with predicates.

The __init__ for stages with predicates.

Parameters:

  • variable_output_drive (str) – The configuration key for where the predicates are located.

  • args (any) – Arguments for initializing a Stage.

  • kwargs (any) – Keyword arguments for initializing a Stage.

field configuration_data: list[dict] [Required] (alias 'configuration')#
field event_lanes: list[str] = [] (alias 'eventLanes')#
field input_lanes: list[str] = [] (alias 'inputLanes')#
field instance_name: str [Required] (alias 'instanceName')#
field library: str [Required]#
field output_lanes: list[str] = [] (alias 'outputLanes')#
field services_data: list[dict] [Required] (alias 'services')#
field stage_id: str [Required] (alias 'stageName')#
field stage_version: str [Required] (alias 'stageVersion')#
field ui_info: dict [Required] (alias 'uiInfo')#
add_predicates(predicates: list[dict | str])#

Add a predicate.

Example

stage.add_predicates([‘>0’]) stage.add_predicates([{‘predicate’:’>0’, ‘outputLane’:’lane1’}]) stage.add_predicates([‘>0’ ,’=0’])

Parameters:

predicates (list[dict | str]) – The list of predicates to add.

Raises:

ValueError – If predicates is not a list.

Return type:

None

connect_event_to(*stages: Stage)#

Connect other stages to the event output of this stage.

Parameters:

stages (Stage) – Stages to connect to this stage

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

connect_input_to(*stages: Stage, predicate: dict | None = None)#

Connect other stages to the input of this stage.

Parameters:

  • stages (Stage) – Stages to connect to this stage

  • predicate (Optional[dict], default: None) – Whether to use any specific predicate when connecting (only applicable to some stages).

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

connect_output_to(*stages: Stage, predicate: dict | None = None)#

Connect other stages to the output of this stage.

Parameters:

  • stages (Stage) – Stages to connect to this stage

  • predicate (Optional[dict], default: None) – Whether to use any specific predicate when connecting (only applicable to some stages).

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

classmethod construct(_fields_set: set[str] | None = None, **values: Any)#
Return type:

Self

copy(*, include: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, exclude: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, update: Dict[str, Any] | None = None, deep: bool = False)#

Returns a copy of the model.

!!! warning “Deprecated”

This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`python {test="skip" lint="skip"} data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

  • include (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to include in the copied model.

  • exclude (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to exclude in the copied model.

  • update (Optional[Dict[str, Any]], default: None) – Optional dictionary of field-value pairs to override field values in the copied model.

  • deep (bool, default: False) – If True, the values of fields that are Pydantic models will be deep-copied.

Return type:

Self

Returns:

A copy of the model with included, excluded and updated fields as specified.

dict(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False)#
Return type:

Dict[str, Any]

disconnect_event_from(*stages: Stage)#

Disconnect the event output of this stage from the provided stages.

Parameters:

stages (Stage) – Stages to disconnect from this stage.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

disconnect_input_from(*stages: Stage)#

Disconnect the input of this stage from the output of the provided stages.

Parameters:

stages (Stage) – Stages to disconnect from this stage.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

disconnect_output_from(*stages: Stage)#

Disconnect the output of this stage from the provided stages.

Parameters:

stages (Stage) – Stages to disconnect from this stage.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

validator entity_validation  »  all fields#

Flattens/Manipulates the data.

Parameters:

data (dict) – Data to be manipulated on

Return type:

dict

Returns:

A dict with manipulated data.

classmethod from_orm(obj: Any)#
Return type:

Self

json(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Callable[[Any], Any] | None = PydanticUndefined, models_as_dict: bool = PydanticUndefined, **dumps_kwargs: Any)#
Return type:

str

classmethod model_construct(_fields_set: set[str] | None = None, **values: Any)#

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed.

!!! note

model_construct() generally respects the model_config.extra setting on the provided model. That is, if model_config.extra == ‘allow’, then all extra passed values are added to the model instance’s __dict__ and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored. Because no validation is performed with a call to model_construct(), having model_config.extra == ‘forbid’ does not result in an error if extra values are passed, but they will be ignored.

Parameters:

  • _fields_set (Optional[set[str]], default: None) – A set of field names that were originally explicitly set during instantiation. If provided, this is directly used for the [model_fields_set][pydantic.BaseModel.model_fields_set] attribute. Otherwise, the field names from the values argument will be used.

  • values (Any) – Trusted or pre-validated data dictionary.

Return type:

Self

Returns:

A new instance of the Model class with validated data.

model_copy(*, update: Mapping[str, Any] | None = None, deep: bool = False)#
!!! abstract “Usage Documentation”

[model_copy](../concepts/models.md#model-copy)

Returns a copy of the model.

!!! note

The underlying instance’s [__dict__][object.__dict__] attribute is copied. This might have unexpected side effects if you store anything in it, on top of the model fields (e.g. the value of [cached properties][functools.cached_property]).

Parameters:

  • update (Optional[Mapping[str, Any]], default: None) – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.

  • deep (bool, default: False) – Set to True to make a deep copy of the model.

Return type:

Self

Returns:

New model instance.

model_dump(by_alias: bool = True, **kwargs: any)#

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Return type:

dict

Returns:

A dictionary representation of the model.

model_dump_json(*, indent: int | None = None, ensure_ascii: bool = False, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, context: Any | None = None, by_alias: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, exclude_computed_fields: bool = False, round_trip: bool = False, warnings: bool | Literal['none', 'warn', 'error'] = True, fallback: Callable[[Any], Any] | None = None, serialize_as_any: bool = False)#
!!! abstract “Usage Documentation”

[model_dump_json](../concepts/serialization.md#json-mode)

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

  • indent (Optional[int], default: None) – Indentation to use in the JSON output. If None is passed, the output will be compact.

  • ensure_ascii (bool, default: False) – If True, the output is guaranteed to have all incoming non-ASCII characters escaped. If False (the default), these characters will be output as-is.

  • include (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to include in the JSON output.

  • exclude (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to exclude from the JSON output.

  • context (Optional[Any], default: None) – Additional context to pass to the serializer.

  • by_alias (Optional[bool], default: None) – Whether to serialize using field aliases.

  • exclude_unset (bool, default: False) – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults (bool, default: False) – Whether to exclude fields that are set to their default value.

  • exclude_none (bool, default: False) – Whether to exclude fields that have a value of None.

  • exclude_computed_fields (bool, default: False) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.

  • round_trip (bool, default: False) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings (Union[bool, Literal['none', 'warn', 'error']], default: True) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].

  • fallback (Optional[Callable[[Any], Any]], default: None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.

  • serialize_as_any (bool, default: False) – Whether to serialize fields with duck-typing serialization behavior.

Return type:

str

Returns:

A JSON string representation of the model.

classmethod model_json_schema(**kwargs: Any)#

Generate a JSON schema for the model.

This method overrides the Pydantic base method to provide a properly formatted docstring that avoids Sphinx/docutils warnings about bullet list formatting.

Parameters:

**kwargs (Any) – Additional keyword arguments passed to the parent method.

Return type:

dict

Returns:

A dictionary containing the JSON schema for the model.

classmethod model_parametrized_name(params: tuple[type[Any], ...])#

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:

params (tuple[type[Any], ...]) – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.

Return type:

str

Returns:

String representing the new class where params are passed to cls as type variables.

Raises:

TypeError – Raised when trying to generate concrete names for non-generic models.

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

classmethod model_rebuild(*, force: bool = False, raise_errors: bool = True, _parent_namespace_depth: int = 2, _types_namespace: Mapping[str, Any] | None = None)#

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

  • force (bool, default: False) – Whether to force the rebuilding of the model schema, defaults to False.

  • raise_errors (bool, default: True) – Whether to raise errors, defaults to True.

  • _parent_namespace_depth (int, default: 2) – The depth level of the parent namespace, defaults to 2.

  • _types_namespace (Optional[Mapping[str, Any]], default: None) – The types namespace, defaults to None.

Return type:

bool | None

Returns:

Returns None if the schema is already “complete” and rebuilding was not required. If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False.

classmethod model_validate(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, from_attributes: bool | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate a pydantic model instance.

Parameters:

  • obj (Any) – The object to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • from_attributes (Optional[bool], default: None) – Whether to extract data from object attributes.

  • context (Optional[Any], default: None) – Additional context to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Raises:

ValidationError – If the object could not be validated.

Return type:

Self

Returns:

The validated model instance.

classmethod model_validate_json(json_data: str | bytes | bytearray, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#
!!! abstract “Usage Documentation”

[JSON Parsing](../concepts/json.md#json-parsing)

Validate the given JSON data against the Pydantic model.

Parameters:

  • json_data (str | bytes | bytearray) – The JSON data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

Raises:

ValidationError – If json_data is not a JSON string or the object could not be validated.

classmethod model_validate_strings(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate the given object with string data against the Pydantic model.

Parameters:

  • obj (Any) – The object containing string data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

classmethod parse_file(path: str | Path, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod parse_obj(obj: Any)#
Return type:

Self

classmethod parse_raw(b: str | bytes, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

remove_predicate(predicate: dict)#

Remove a predicate.

Example

stage.remove_predicate(stage.predicates[0]) stage.remove_predicate({‘predicate’:’>0’, ‘outputLane’:’lane1’})

Parameters:

predicate (dict) – The predicate to delete as a dictionary including the outputLane.

Raises:

ValueError – If predicates is not specified or can’t find its target.

Return type:

None

classmethod schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}')#
Return type:

Dict[str, Any]

classmethod schema_json(*, by_alias: bool = True, ref_template: str = '#/$defs/{model}', **dumps_kwargs: Any)#
Return type:

str

serialize_model(handler: Any, info: SerializationInfo)#

Serializes the data, back into it’s original formatting.

Return type:

dict

Returns:

A dict with updated information

classmethod update_forward_refs(**localns: Any)#
Return type:

None

classmethod validate(value: Any)#
Return type:

Self

property configuration: Configuration#

The stage’s configurations.

property events: Stages#

All the stages connected to this stages event output.

property inputs: Stages#

All the stages connected to this stages input.

property model_extra: dict[str, Any] | None#

Get extra fields set during validation.

Returns:

A dictionary of extra fields, or None if config.extra is not set to “allow”.

property model_fields_set: set[str]#

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,

i.e. that were not filled from defaults.

property outputs: Stages#

All the stages connected to this stages output.

property predicates: list[dict]#

Get the predicate list for this stage.

property stage_name: str#

Getting the stage instance’s name.

property type: str#

The type of this stage.

class ibm_watsonx_data_integration.services.streamsets.models.flow_model.Stages(iterable=(), /)#

Bases: SeekableList

Seekable List of Stages.

append(object, /)#

Append object to the end of the list.

clear()#

Remove all items from list.

connect_event_to(*stages: Stage)#

Connect other stages to the event output of this stage list - all combinations.

Parameters:

stages (Stage) – Stages to connect to this stage list.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

connect_input_to(*stages: Stage, predicate: dict | None = None)#

Connect other stages to the input of this stage list - all combinations.

Parameters:

  • stages (Stage) – Stages to connect to this stage list.

  • predicate (Optional[dict], default: None) – Whether to use any specific predicate when connecting (only applicable to some stages). Will be used for all stages.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

connect_output_to(*stages: Stage, predicate: dict | None = None)#

Connect other stages to the output of this stage list - all combinations.

Parameters:

  • stages (Stage) – Stages to connect to this stage list.

  • predicate (Optional[dict], default: None) – Whether to use any specific predicate when connecting (only applicable to some stages). Will be used for all stages.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

convert_from(*stages: Stage)#

Converts stages to Stages object.

Return type:

Stages

copy()#

Return a shallow copy of the list.

count(value, /)#

Return number of occurrences of value.

disconnect_event_from(*stages: Stage)#

Disconnect the event outputs of all the stages in this stage list from the provided stages.

Parameters:

stages (Stage) – Stages to disconnect from this stage list.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

disconnect_input_from(*stages: Stage)#

Disconnect the inputs of all the stages in this stage list from the output of the provided stages.

Parameters:

stages (Stage) – Stages to disconnect from this stage list.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

disconnect_output_from(*stages: Stage)#

Disconnect the outputs of all the stages in this stage list from the provided stages.

Parameters:

stages (Stage) – Stages to disconnect from this stage list.

Return type:

Union[Stage, StageWithPredicates, Stages]

Returns:

List stages parameter wrapped in Stages object or Stage/StageWithPredicates if only one stage was used.

extend(iterable, /)#

Extend list by appending elements from the iterable.

get(**kwargs: dict)#

Retrieve the first instance that matches the supplied arguments.

Parameters:

**kwargs (dict) – Optional arguments to be passed to filter the results offline.

Return type:

TypeVar(T, bound= BaseModel)

Returns:

The first instance from the group that matches the supplied arguments.

get_all(**kwargs: dict)#

Retrieve all instances that match the supplied arguments.

Parameters:

**kwargs (dict) – Optional arguments to be passed to filter the results offline.

Return type:

SeekableList[BaseModel]

Returns:

A ibm_watsonx_data_integration.common.utils.SeekableList of results that match the supplied arguments.

index(value, start=0, stop=9223372036854775807, /)#

Return first index of value.

Raises ValueError if the value is not present.

insert(index, object, /)#

Insert object before index.

pop(index=-1, /)#

Remove and return item at index (default last).

Raises IndexError if list is empty or index is out of range.

remove(value, /)#

Remove first occurrence of value.

Raises ValueError if the value is not present.

reverse()#

Reverse IN PLACE.

sort(*, key=None, reverse=False)#

Sort the list in ascending order and return None.

The sort is in-place (i.e. the list itself is modified) and stable (i.e. the order of two equal elements is maintained).

If a key function is given, apply it once to each list item and sort them, ascending or descending, according to their function values.

The reverse flag can be set to sort in descending order.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.StreamingConnection#

Bases: BaseModel

Streaming Connection object.

The __init__ of the StreamingConnection class.

Parameters:

  • connection_json (dict) – The JSON for the StreamingConnection.

  • platform (Optional[Platform], default: None) – The Platform object.

  • project (Optional[Project], default: None) – The Project object.

field datasource_type: str | None = None#

Datasource type

field metadata: StreamingConnectionMetadata [Required]#
field name: str | None = None#

Connection name

field properties: dict | None = None#
classmethod construct(_fields_set: set[str] | None = None, **values: Any)#
Return type:

Self

copy(*, include: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, exclude: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, update: Dict[str, Any] | None = None, deep: bool = False)#

Returns a copy of the model.

!!! warning “Deprecated”

This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`python {test="skip" lint="skip"} data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

  • include (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to include in the copied model.

  • exclude (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to exclude in the copied model.

  • update (Optional[Dict[str, Any]], default: None) – Optional dictionary of field-value pairs to override field values in the copied model.

  • deep (bool, default: False) – If True, the values of fields that are Pydantic models will be deep-copied.

Return type:

Self

Returns:

A copy of the model with included, excluded and updated fields as specified.

dict(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False)#
Return type:

Dict[str, Any]

validator entity_validation  »  all fields#

Flattens/Manipulates the data.

Parameters:

data (dict) – Data to be manipulated on

Return type:

dict

Returns:

A dict with manipulated data.

classmethod from_orm(obj: Any)#
Return type:

Self

json(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Callable[[Any], Any] | None = PydanticUndefined, models_as_dict: bool = PydanticUndefined, **dumps_kwargs: Any)#
Return type:

str

classmethod model_construct(_fields_set: set[str] | None = None, **values: Any)#

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed.

!!! note

model_construct() generally respects the model_config.extra setting on the provided model. That is, if model_config.extra == ‘allow’, then all extra passed values are added to the model instance’s __dict__ and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored. Because no validation is performed with a call to model_construct(), having model_config.extra == ‘forbid’ does not result in an error if extra values are passed, but they will be ignored.

Parameters:

  • _fields_set (Optional[set[str]], default: None) – A set of field names that were originally explicitly set during instantiation. If provided, this is directly used for the [model_fields_set][pydantic.BaseModel.model_fields_set] attribute. Otherwise, the field names from the values argument will be used.

  • values (Any) – Trusted or pre-validated data dictionary.

Return type:

Self

Returns:

A new instance of the Model class with validated data.

model_copy(*, update: Mapping[str, Any] | None = None, deep: bool = False)#
!!! abstract “Usage Documentation”

[model_copy](../concepts/models.md#model-copy)

Returns a copy of the model.

!!! note

The underlying instance’s [__dict__][object.__dict__] attribute is copied. This might have unexpected side effects if you store anything in it, on top of the model fields (e.g. the value of [cached properties][functools.cached_property]).

Parameters:

  • update (Optional[Mapping[str, Any]], default: None) – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.

  • deep (bool, default: False) – Set to True to make a deep copy of the model.

Return type:

Self

Returns:

New model instance.

model_dump(by_alias: bool = True, **kwargs: any)#

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Return type:

dict

Returns:

A dictionary representation of the model.

model_dump_json(*, indent: int | None = None, ensure_ascii: bool = False, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, context: Any | None = None, by_alias: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, exclude_computed_fields: bool = False, round_trip: bool = False, warnings: bool | Literal['none', 'warn', 'error'] = True, fallback: Callable[[Any], Any] | None = None, serialize_as_any: bool = False)#
!!! abstract “Usage Documentation”

[model_dump_json](../concepts/serialization.md#json-mode)

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

  • indent (Optional[int], default: None) – Indentation to use in the JSON output. If None is passed, the output will be compact.

  • ensure_ascii (bool, default: False) – If True, the output is guaranteed to have all incoming non-ASCII characters escaped. If False (the default), these characters will be output as-is.

  • include (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to include in the JSON output.

  • exclude (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to exclude from the JSON output.

  • context (Optional[Any], default: None) – Additional context to pass to the serializer.

  • by_alias (Optional[bool], default: None) – Whether to serialize using field aliases.

  • exclude_unset (bool, default: False) – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults (bool, default: False) – Whether to exclude fields that are set to their default value.

  • exclude_none (bool, default: False) – Whether to exclude fields that have a value of None.

  • exclude_computed_fields (bool, default: False) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.

  • round_trip (bool, default: False) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings (Union[bool, Literal['none', 'warn', 'error']], default: True) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].

  • fallback (Optional[Callable[[Any], Any]], default: None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.

  • serialize_as_any (bool, default: False) – Whether to serialize fields with duck-typing serialization behavior.

Return type:

str

Returns:

A JSON string representation of the model.

classmethod model_json_schema(**kwargs: Any)#

Generate a JSON schema for the model.

This method overrides the Pydantic base method to provide a properly formatted docstring that avoids Sphinx/docutils warnings about bullet list formatting.

Parameters:

**kwargs (Any) – Additional keyword arguments passed to the parent method.

Return type:

dict

Returns:

A dictionary containing the JSON schema for the model.

classmethod model_parametrized_name(params: tuple[type[Any], ...])#

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:

params (tuple[type[Any], ...]) – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.

Return type:

str

Returns:

String representing the new class where params are passed to cls as type variables.

Raises:

TypeError – Raised when trying to generate concrete names for non-generic models.

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

classmethod model_rebuild(*, force: bool = False, raise_errors: bool = True, _parent_namespace_depth: int = 2, _types_namespace: Mapping[str, Any] | None = None)#

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

  • force (bool, default: False) – Whether to force the rebuilding of the model schema, defaults to False.

  • raise_errors (bool, default: True) – Whether to raise errors, defaults to True.

  • _parent_namespace_depth (int, default: 2) – The depth level of the parent namespace, defaults to 2.

  • _types_namespace (Optional[Mapping[str, Any]], default: None) – The types namespace, defaults to None.

Return type:

bool | None

Returns:

Returns None if the schema is already “complete” and rebuilding was not required. If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False.

classmethod model_validate(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, from_attributes: bool | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate a pydantic model instance.

Parameters:

  • obj (Any) – The object to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • from_attributes (Optional[bool], default: None) – Whether to extract data from object attributes.

  • context (Optional[Any], default: None) – Additional context to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Raises:

ValidationError – If the object could not be validated.

Return type:

Self

Returns:

The validated model instance.

classmethod model_validate_json(json_data: str | bytes | bytearray, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#
!!! abstract “Usage Documentation”

[JSON Parsing](../concepts/json.md#json-parsing)

Validate the given JSON data against the Pydantic model.

Parameters:

  • json_data (str | bytes | bytearray) – The JSON data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

Raises:

ValidationError – If json_data is not a JSON string or the object could not be validated.

classmethod model_validate_strings(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate the given object with string data against the Pydantic model.

Parameters:

  • obj (Any) – The object containing string data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

classmethod parse_file(path: str | Path, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod parse_obj(obj: Any)#
Return type:

Self

classmethod parse_raw(b: str | bytes, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}')#
Return type:

Dict[str, Any]

classmethod schema_json(*, by_alias: bool = True, ref_template: str = '#/$defs/{model}', **dumps_kwargs: Any)#
Return type:

str

serialize_model(handler: Any, info: SerializationInfo)#

Serializes the data, back into it’s original formatting.

Return type:

dict

Returns:

A dict with updated information

classmethod update_forward_refs(**localns: Any)#
Return type:

None

classmethod validate(value: Any)#
Return type:

Self

property model_extra: dict[str, Any] | None#

Get extra fields set during validation.

Returns:

A dictionary of extra fields, or None if config.extra is not set to “allow”.

property model_fields_set: set[str]#

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,

i.e. that were not filled from defaults.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.StreamingConnectionMetadata#

Bases: BaseModel

Streaming Connection Metadata object.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field asset_category: str | None = None#

Category of the asset

field asset_id: str [Required]#

Asset identifier

field asset_type: str | None = None#

Type of the asset

field create_time: str | None = None#

Creation time

field creator_id: str | None = None#

Asset creator identifier

field project_id: str | None = None#

Project identifier

classmethod construct(_fields_set: set[str] | None = None, **values: Any)#
Return type:

Self

copy(*, include: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, exclude: Set[int] | Set[str] | Mapping[int, Any] | Mapping[str, Any] | None = None, update: Dict[str, Any] | None = None, deep: bool = False)#

Returns a copy of the model.

!!! warning “Deprecated”

This method is now deprecated; use model_copy instead.

If you need include or exclude, use:

`python {test="skip" lint="skip"} data = self.model_dump(include=include, exclude=exclude, round_trip=True) data = {**data, **(update or {})} copied = self.model_validate(data) `

Parameters:

  • include (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to include in the copied model.

  • exclude (Union[Set[int], Set[str], Mapping[int, Any], Mapping[str, Any], None], default: None) – Optional set or mapping specifying which fields to exclude in the copied model.

  • update (Optional[Dict[str, Any]], default: None) – Optional dictionary of field-value pairs to override field values in the copied model.

  • deep (bool, default: False) – If True, the values of fields that are Pydantic models will be deep-copied.

Return type:

Self

Returns:

A copy of the model with included, excluded and updated fields as specified.

dict(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False)#
Return type:

Dict[str, Any]

validator entity_validation  »  all fields#

Flattens/Manipulates the data.

Parameters:

data (dict) – Data to be manipulated on

Return type:

dict

Returns:

A dict with manipulated data.

classmethod from_orm(obj: Any)#
Return type:

Self

json(*, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, by_alias: bool = False, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Callable[[Any], Any] | None = PydanticUndefined, models_as_dict: bool = PydanticUndefined, **dumps_kwargs: Any)#
Return type:

str

classmethod model_construct(_fields_set: set[str] | None = None, **values: Any)#

Creates a new instance of the Model class with validated data.

Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed.

!!! note

model_construct() generally respects the model_config.extra setting on the provided model. That is, if model_config.extra == ‘allow’, then all extra passed values are added to the model instance’s __dict__ and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored. Because no validation is performed with a call to model_construct(), having model_config.extra == ‘forbid’ does not result in an error if extra values are passed, but they will be ignored.

Parameters:

  • _fields_set (Optional[set[str]], default: None) – A set of field names that were originally explicitly set during instantiation. If provided, this is directly used for the [model_fields_set][pydantic.BaseModel.model_fields_set] attribute. Otherwise, the field names from the values argument will be used.

  • values (Any) – Trusted or pre-validated data dictionary.

Return type:

Self

Returns:

A new instance of the Model class with validated data.

model_copy(*, update: Mapping[str, Any] | None = None, deep: bool = False)#
!!! abstract “Usage Documentation”

[model_copy](../concepts/models.md#model-copy)

Returns a copy of the model.

!!! note

The underlying instance’s [__dict__][object.__dict__] attribute is copied. This might have unexpected side effects if you store anything in it, on top of the model fields (e.g. the value of [cached properties][functools.cached_property]).

Parameters:

  • update (Optional[Mapping[str, Any]], default: None) – Values to change/add in the new model. Note: the data is not validated before creating the new model. You should trust this data.

  • deep (bool, default: False) – Set to True to make a deep copy of the model.

Return type:

Self

Returns:

New model instance.

model_dump(by_alias: bool = True, **kwargs: any)#

Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.

Return type:

dict

Returns:

A dictionary representation of the model.

model_dump_json(*, indent: int | None = None, ensure_ascii: bool = False, include: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, exclude: set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None = None, context: Any | None = None, by_alias: bool | None = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, exclude_computed_fields: bool = False, round_trip: bool = False, warnings: bool | Literal['none', 'warn', 'error'] = True, fallback: Callable[[Any], Any] | None = None, serialize_as_any: bool = False)#
!!! abstract “Usage Documentation”

[model_dump_json](../concepts/serialization.md#json-mode)

Generates a JSON representation of the model using Pydantic’s to_json method.

Parameters:

  • indent (Optional[int], default: None) – Indentation to use in the JSON output. If None is passed, the output will be compact.

  • ensure_ascii (bool, default: False) – If True, the output is guaranteed to have all incoming non-ASCII characters escaped. If False (the default), these characters will be output as-is.

  • include (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to include in the JSON output.

  • exclude (Union[set[int], set[str], Mapping[int, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], Mapping[str, Union[set[int], set[str], Mapping[int, Union[IncEx, bool]], Mapping[str, Union[IncEx, bool]], bool]], None], default: None) – Field(s) to exclude from the JSON output.

  • context (Optional[Any], default: None) – Additional context to pass to the serializer.

  • by_alias (Optional[bool], default: None) – Whether to serialize using field aliases.

  • exclude_unset (bool, default: False) – Whether to exclude fields that have not been explicitly set.

  • exclude_defaults (bool, default: False) – Whether to exclude fields that are set to their default value.

  • exclude_none (bool, default: False) – Whether to exclude fields that have a value of None.

  • exclude_computed_fields (bool, default: False) – Whether to exclude computed fields. While this can be useful for round-tripping, it is usually recommended to use the dedicated round_trip parameter instead.

  • round_trip (bool, default: False) – If True, dumped values should be valid as input for non-idempotent types such as Json[T].

  • warnings (Union[bool, Literal['none', 'warn', 'error']], default: True) – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors, “error” raises a [PydanticSerializationError][pydantic_core.PydanticSerializationError].

  • fallback (Optional[Callable[[Any], Any]], default: None) – A function to call when an unknown value is encountered. If not provided, a [PydanticSerializationError][pydantic_core.PydanticSerializationError] error is raised.

  • serialize_as_any (bool, default: False) – Whether to serialize fields with duck-typing serialization behavior.

Return type:

str

Returns:

A JSON string representation of the model.

classmethod model_json_schema(**kwargs: Any)#

Generate a JSON schema for the model.

This method overrides the Pydantic base method to provide a properly formatted docstring that avoids Sphinx/docutils warnings about bullet list formatting.

Parameters:

**kwargs (Any) – Additional keyword arguments passed to the parent method.

Return type:

dict

Returns:

A dictionary containing the JSON schema for the model.

classmethod model_parametrized_name(params: tuple[type[Any], ...])#

Compute the class name for parametrizations of generic classes.

This method can be overridden to achieve a custom naming scheme for generic BaseModels.

Parameters:

params (tuple[type[Any], ...]) – Tuple of types of the class. Given a generic class Model with 2 type variables and a concrete model Model[str, int], the value (str, int) would be passed to params.

Return type:

str

Returns:

String representing the new class where params are passed to cls as type variables.

Raises:

TypeError – Raised when trying to generate concrete names for non-generic models.

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

classmethod model_rebuild(*, force: bool = False, raise_errors: bool = True, _parent_namespace_depth: int = 2, _types_namespace: Mapping[str, Any] | None = None)#

Try to rebuild the pydantic-core schema for the model.

This may be necessary when one of the annotations is a ForwardRef which could not be resolved during the initial attempt to build the schema, and automatic rebuilding fails.

Parameters:

  • force (bool, default: False) – Whether to force the rebuilding of the model schema, defaults to False.

  • raise_errors (bool, default: True) – Whether to raise errors, defaults to True.

  • _parent_namespace_depth (int, default: 2) – The depth level of the parent namespace, defaults to 2.

  • _types_namespace (Optional[Mapping[str, Any]], default: None) – The types namespace, defaults to None.

Return type:

bool | None

Returns:

Returns None if the schema is already “complete” and rebuilding was not required. If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False.

classmethod model_validate(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, from_attributes: bool | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate a pydantic model instance.

Parameters:

  • obj (Any) – The object to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • from_attributes (Optional[bool], default: None) – Whether to extract data from object attributes.

  • context (Optional[Any], default: None) – Additional context to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Raises:

ValidationError – If the object could not be validated.

Return type:

Self

Returns:

The validated model instance.

classmethod model_validate_json(json_data: str | bytes | bytearray, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#
!!! abstract “Usage Documentation”

[JSON Parsing](../concepts/json.md#json-parsing)

Validate the given JSON data against the Pydantic model.

Parameters:

  • json_data (str | bytes | bytearray) – The JSON data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

Raises:

ValidationError – If json_data is not a JSON string or the object could not be validated.

classmethod model_validate_strings(obj: Any, *, strict: bool | None = None, extra: Literal['allow', 'ignore', 'forbid'] | None = None, context: Any | None = None, by_alias: bool | None = None, by_name: bool | None = None)#

Validate the given object with string data against the Pydantic model.

Parameters:

  • obj (Any) – The object containing string data to validate.

  • strict (Optional[bool], default: None) – Whether to enforce types strictly.

  • extra (Optional[Literal['allow', 'ignore', 'forbid']], default: None) – Whether to ignore, allow, or forbid extra data during model validation. See the [extra configuration value][pydantic.ConfigDict.extra] for details.

  • context (Optional[Any], default: None) – Extra variables to pass to the validator.

  • by_alias (Optional[bool], default: None) – Whether to use the field’s alias when validating against the provided input data.

  • by_name (Optional[bool], default: None) – Whether to use the field’s name when validating against the provided input data.

Return type:

Self

Returns:

The validated Pydantic model.

classmethod parse_file(path: str | Path, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod parse_obj(obj: Any)#
Return type:

Self

classmethod parse_raw(b: str | bytes, *, content_type: str | None = None, encoding: str = 'utf8', proto: Protocol | None = None, allow_pickle: bool = False)#
Return type:

Self

classmethod schema(by_alias: bool = True, ref_template: str = '#/$defs/{model}')#
Return type:

Dict[str, Any]

classmethod schema_json(*, by_alias: bool = True, ref_template: str = '#/$defs/{model}', **dumps_kwargs: Any)#
Return type:

str

serialize_model(handler: Any, info: SerializationInfo)#

Serializes the data, back into it’s original formatting.

Return type:

dict

Returns:

A dict with updated information

classmethod update_forward_refs(**localns: Any)#
Return type:

None

classmethod validate(value: Any)#
Return type:

Self

property model_extra: dict[str, Any] | None#

Get extra fields set during validation.

Returns:

A dictionary of extra fields, or None if config.extra is not set to “allow”.

property model_fields_set: set[str]#

Returns the set of fields that have been explicitly set on this model instance.

Returns:

A set of strings representing the fields that have been set,

i.e. that were not filled from defaults.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.StreamingFlow#

Bases: Flow, Generatable

The Model for StreamingFlow.

The __init__ of the StreamingFlow class.

Parameters:

  • project (Project) – The Project object.

  • pipeline_definition (Optional[dict], default: None) – Pipeline definition for the flow

  • flow_json (dict) – The JSON for the flow.

field connection_ids: list[str] = []#
field engine_version: str [Required]#
field environment_id: str = ''#
field flow_id: str | None [Optional]#
field fragment_ids: list[str] [Optional]#
field metadata: StreamingFlowMetadata [Required]#
add_stage(label: str | None = None, name: str | None = None, type: str | None = None, library: str | None = None)#

Add a stage to the flow.

Parameters:

  • label (Optional[str], default: None) – Label of the stage to add.

  • name (Optional[str], default: None) – Name of the stage to add.

  • type (Optional[str], default: None) – Type of the stage to add.

  • library (Optional[str], default: None) – Library that stage to add belongs to.

Return type:

Stage | StageWithPredicates

duplicate_stage(stage: Stage)#

Duplicate a stage.

Parameters:

stage (Stage) – The stage to duplicate.

Return type:

None

static get_remapped_stages(old_pipeline_definition: PipelineDefinition, new_pipeline_definition: PipelineDefinition)#

Reinitialize stages from the old pipeline_definition to the old one.

We need to ‘remap’ old stages to new ones to be able to use same object referentec to stages.

Return type:

list[Stage | StageWithPredicates]

preview(batch_size: int = 10, timeout_ms: int = 120000, run_through_stage: str = 'all', time_zone: str = 'BROWSER', write_to_destination: bool = False, execute_lifecycle_events: bool = False, show_field_type: bool = True, test_origin: bool = False, push_limit_down: bool = True, remote: bool = True)#

Runs a flow preview.

Parameters:

  • batch_size (int, default: 10) – Preview batch size (default 10).

  • timeout_ms (int, default: 120000) – Preview timeout in milliseconds (default 120000).

  • write_to_destination (bool, default: False) – Whether to write to targets and executors during preview (default False).

  • execute_lifecycle_events (bool, default: False) – Whether to execute pipeline lifecycle events during preview (default False).

  • show_field_type (bool, default: True) – Show field type in preview (default True).

  • run_through_stage (str, default: 'all') – Run preview through stage. Use “all” to run all stages, or specify a stage instance name to stop at that stage (default “all”).

  • time_zone (str, default: 'BROWSER') – Timezone to use for preview. Use “BROWSER” for browser timezone, or specify an IANA timezone identifier like “America/New_York” (default “BROWSER”).

  • test_origin (bool, default: False) – Whether to test origin connectivity before preview (default False).

  • push_limit_down (bool, default: True) – Whether to push limit down optimization (default True).

  • remote (bool, default: True) – Whether to run preview remotely (default True).

Return type:

list[PreviewStage]

Returns:

List of PreviewStages containing input/output data for each stage.

Raises:

  • ValueError – If flow is not associated with an environment.

  • NoEnginesInstalledError – If no engine is available in the associated environment.

  • FlowPreviewError – If the preview encounters an error during execution.

remove_stage(stage: Stage)#

Removes a stage from the flow.

Parameters:

stage (Stage) – The stage to remove.

Return type:

None

set_error_stage(label: str | None = None, name: str | None = None, library: str | None = None)#

Set the error stage of a flow.

Return type:

Stage

set_start_event_stage(label: str | None = None, name: str | None = None, library: str | None = None)#

Set the start event stage of a flow.

Return type:

Stage

set_stats_aggregator_stage(label: str | None = None, name: str | None = None, library: str | None = None)#

Set the stats aggregator stage of a flow.

Return type:

Stage

set_stop_event_stage(label: str | None = None, name: str | None = None, library: str | None = None)#

Set the stop event stage of a flow.

Return type:

Stage

set_test_origin_stage(label: str | None = None, name: str | None = None, library: str | None = None)#

Set the test origin stage of a flow.

Return type:

Stage

property configuration: Configuration#

The configuration of a flow.

property description: str#

Returns description of the flow.

property engine: Engine | None#

The engine in which this flow will be run.

property environment: Environment#

The environment this flow belongs to.

property error_stage: Stage | None#

The error stage of the flow.

property executor_id: str#

Get a flow’s id in an executor.

property name: str#

Returns name of the flow.

property pipeline_definition: dict#

A flow’s definition in an engine.

property stages: Stages[Stage | StageWithPredicates]#

The stages in the flow.

property start_event: Stage | None#

The start event stage of the flow.

property stats_aggregator_stage: Stage | None#

The stats aggregator stage of the flow.

property stop_event: Stage | None#

The stop event stage of the flow.

property test_origin: Stage | None#

The test origin stage of the flow.

pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.StreamingFlowMetadata#

Bases: BaseModel

The Model for StreamingFlowMetadata.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field asset_attributes: list | None = None#
field asset_category: str | None = None#
field asset_id: str [Required]#
field asset_state: str | None = None#
field asset_type: str | None = None#
field catalog_id: str | None = None#
field create_time: str | None = None#
field created: int | None = None#
field created_at: str | None = None#
field creator_id: str | None = None#
field description: str [Required]#
field is_linked_with_sub_container: bool | None = None#
field name: str [Required]#
field owner_id: str | None = None#
field project_id: str | None = None#
field rating: int | None = None#
field rov: StreamingFlowROV | None = None#
field sandbox_id: str | None = None#
field size: int | None = None#
field tags: list | None = None#
field total_ratings: int | None = None#
field usage: StreamingFlowUsage | None = None#
field version: float | None = None#
class ibm_watsonx_data_integration.services.streamsets.models.flow_model.StreamingFlowPayloadExtender#

Bases: PayloadExtender

Streaming flow extender setup also streamsets_env_id.

extend(payload: dict[str, Any], flow: Flow)#

Entry point for starting pre-check and extending create job payload.

Return type:

dict[str, Any]

prepare_payload(payload: dict[str, Any], flow: Flow)#

Here we should modify and return payload for job creation.

Return type:

dict[str, Any]

run_pre_start_operations(flow: StreamingFlow)#

Here we should perform all flow prerequisite operations like validation, compilation etc.

When pre-checks fail we should raise FlowPreStartOperationError exception to carry out this information to user.

Raises:

FlowPreStartOperationError – If any pre-check operation failed.

Return type:

None

pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.StreamingFlowROV#

Bases: BaseModel

The Model for StreamingFlowROV.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field collaborator_ids: dict | None [Required]#
field member_roles: dict | None [Required]#
field mode: int | None [Required]#
pydantic model ibm_watsonx_data_integration.services.streamsets.models.flow_model.StreamingFlowUsage#

Bases: BaseModel

The Model for StreamingFlowUsage.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field access_count: int | None [Required]#
field last_access_time: str | None [Required]#
field last_accessed_at: str | None [Required]#
field last_accessor_id: str | None [Required]#
field last_update_time: str | None [Required]#
field last_updated_at: str | None [Required]#
field last_updater_id: str | None [Required]#
class ibm_watsonx_data_integration.services.streamsets.models.flow_model.StreamingFlows(project: Project)#

Bases: CollectionModel[StreamingFlow]

Collection of StreamingFlows.

The __init__ of the StreamingFlow class.

Parameters:

project (Project) – The Project object.

get(**kwargs)#

Used to get an instant result from the api.

Parameters:

**kwargs – Optional arguments to be passed to filter the results.

Return type:

TypeVar(T, bound= BaseModel)

Returns:

An inherited instance of streamsets.sdk.sch_models.BaseModel.

Raises:

ValueError – If instance is not in the list.

get_all(**kwargs: dict)#

Used to get multiple (all) results from api.

Parameters:

**kwargs (dict) – Optional other arguments to be passed to filter the results.

Return type:

SeekableList[TypeVar(T, bound= BaseModel)]

Returns:

A list of inherited instances of

streamsets.sdk.sch_models.BaseModel.

class ibm_watsonx_data_integration.services.streamsets.models.flow_model.ValidationResult(success: bool, issues: list, message: str)#

Bases: object

The Model for ValidationResult.

Streaming Generator#

This module contains PythonGenerator class.

class ibm_watsonx_data_integration.codegen.generator.Generatable#

Bases: ABC

Interface indicating an object can be gnerated by the PythonGenerator.

class ibm_watsonx_data_integration.codegen.generator.PythonGenerator(source: Generatable, destination: str | Path, auth: BaseAuthenticator, mask_credentials: bool = True, persist_topology: bool = False, **kwargs: str | object)#

Bases: object

Flow code generation entrypoint.

The __init__ of the PythonGenerator class.

Parameters:

  • source (Generatable) – Location from Flow definition will be loaded.

  • destination (str | Path) – Location where save generated script.

  • auth (BaseAuthenticator) – Reference to authenticator object.

  • mask_credentials (bool, default: True) – Whether or not to mask sensitive authentication credentials. Defaults to true.

  • persist_topology (bool, default: False) – Whether to preserve stage coordinates in generated code. Defaults to false.

  • kwargs (str | object) – Additional configuration values.

save()#

Run code generation then save it to destination.

Return type:

Path

Returns:

Path to the location where the generated script was saved.

Trusted Profile#

Module containing Trusted Profile Model.

pydantic model ibm_watsonx_data_integration.cpd_models.trusted_profile_model.TrustedProfile#

Bases: BaseModel, ProjectCollaboratable

Model representing a Service ID.

The __init__ of the TrustedProfile Wrapper class.

Parameters:

  • trusted_profile_json (dict) – The JSON for the Trusted Profile.

  • platform (Optional[Platform], default: None) – The Platform object. Default: None

field account_id: str [Required]#
field created_at: str [Required]#
field crn: str [Required]#
field entity_tag: str [Required]#
field iam_id: str [Required]#
field id: str [Required]#
field modified_at: str [Required]#
field name: str [Required]#
field trusted_profile_id: str [Optional]#
property type: str#

This property returns the member type “profile”.

Returns:

The member type.

class ibm_watsonx_data_integration.cpd_models.trusted_profile_model.TrustedProfiles(platform: Platform | None = None)#

Bases: CollectionModel[TrustedProfile]

Collection of TrustedProfile instances.

The __init__ of the TrustedProfiles class.

Parameters:

platform (Optional[Platform], default: None) – The Platform object.

User#

Modules containing UserProfile Model and UserSettings Model.

pydantic model ibm_watsonx_data_integration.cpd_models.user_model.UserProfile#

Bases: BaseModel, ProjectCollaboratable

Model representing a user profile.

The __init__ of the User Profile.

Parameters:

  • platform (Optional[Platform], default: None) – The Platform object.

  • user_profile_json (dict) – The JSON for the User Profile.

field account_id: str [Required]#

Unique identifier of the account the user belongs to.

field added_on: str [Required]#

Timestamp of when the user was added.

field alt_phone: str | None = None (alias 'altphonenumber')#

An alternative phone number for the user.

field email: str [Required]#

The email address of the user.

field first_name: str | None = None (alias 'firstname')#

The first name (given name) of the user.

field iam_id: str [Required]#

The user’s IAM ID.

field id: str [Required]#

Unique identifier of the user.

field invited_on: str | None = None (alias 'invitedOn')#

Timestamp of when the user was invited.

field last_name: str | None = None (alias 'lastname')#

The last name (family name) of the user.

field phone: str | None = None (alias 'phonenumber')#

The primary phone number of the user.

field photo: str | None = None#

A URL or reference to the user’s photo.

field realm: str [Required]#

The realm to which the user belongs.

field state: UserState [Required]#

The current state of the user’s account (e.g. PROCESSING, PENDING, ACTIVE, DISABLED_CLASSIC_INFRASTRUCTURE, VPN_ONLY).

field sub_state: str | None = None#

The substate of the user’s account, if applicable.

field user_id: str [Required]#

The user’s identifier (often their email address).

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

update_user_settings(user_settings: UserSettings)#

Update the settings for a specific user in an account.

Parameters:

user_settings (UserSettings) – Instance of a UserSettings to update.

Return type:

Response

Returns:

A HTTP response.

property settings: UserSettings#

Returns the user settings associated with the current IAM identity and the current account.

Returns:

A User Settings instance retrieved from the API.

property type: str#

This property returns the member type “user”.

Returns:

The member type.

class ibm_watsonx_data_integration.cpd_models.user_model.UserProfiles(platform: Platform)#

Bases: CollectionModel[UserProfile]

Collection of UserProfile instances.

The __init__ of the UserProfiles class.

Parameters:

platform (Platform) – The Platform object.

pydantic model ibm_watsonx_data_integration.cpd_models.user_model.UserSettings#

Bases: BaseModel

Model representing user settings preferences.

The __init__ of the UserSettings model.

Parameters:

  • platform (Optional[Platform], default: None) – The Platform object.

  • settings_json (dict) – The JSON for the user settings.

field allowed_ip_addresses: str = ''#

A string representing allowed IP addresses (e.g., comma-separated). This field may be empty.

field language: str = ''#

The user’s preferred language. This field may be empty.

field notification_language: str = ''#

The preferred language for notifications. This field may be empty.

field security_questions_required: bool = False#

Flag indicating if security questions are required for the user.

field security_questions_setup: bool = False#

Flag that indicates whether the user has set up their security questions.

field self_manage: bool = False#

Flag indicating if the user has permission to manage their settings.

field two_fa: bool = False (alias '2FA')#

Flag that indicates whether two-factor authentication is enabled. This field is mapped from the JSON key “2FA”

model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

class ibm_watsonx_data_integration.cpd_models.user_model.UserState(value)#

Bases: Enum

An enumeration representing the possible states of a user’s account.

User On Prem#

Models for User, UserSettings, and Users collection.

class ibm_watsonx_data_integration.cpd_models.user_model_on_prem.AccountStatus(value)#

Bases: Enum

AccountStatus enum of a group.

class ibm_watsonx_data_integration.cpd_models.user_model_on_prem.Authenticator(value)#

Bases: Enum

Authenticator enum of a group.

pydantic model ibm_watsonx_data_integration.cpd_models.user_model_on_prem.Group#

Bases: BaseModel

Represents a group the user is in.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field added_separately: bool | None = None#
field created_at: str | None = None#
field created_by: str | None = None#
field description: str | None = None#
field group_id: int [Required]#
field members_count: int | None = None#
field misc: GroupMisc | None = None#
field name: str [Required]#
field updated_at: str | None = None#
pydantic model ibm_watsonx_data_integration.cpd_models.user_model_on_prem.GroupMisc#

Bases: BaseModel

Miscellaneous metadata on a group.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field dark_mode: bool | None = None#
field ext_attributes: dict | None = None#
field realm_name: str | None = None#
pydantic model ibm_watsonx_data_integration.cpd_models.user_model_on_prem.SessionInfo#

Bases: BaseModel

Represents one active/recorded session for the user.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field can_refresh_until: int | None = None#
field created_timestamp: int [Required]#
field misc: dict | None = None#
field session_id: str [Required]#
pydantic model ibm_watsonx_data_integration.cpd_models.user_model_on_prem.SessionToken#

Bases: BaseModel

Decoded session token payload (subset modeled to match provided JSON).

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field api_request: bool | None = None#
field aud: str | None = None#
field authenticator: str | None = None#
field can_refresh_until: int | None = None#
field csrf_token: str | None = None#
field display_name: str | None = None#
field groups: list[int] | None = None#
field iam: dict | None = None#
field iss: str | None = None#
field permissions: list[str] | None = None#
field role: str | None = None#
field session_id: str | None = None#
field sub: str | None = None#
field uid: int | None = None#
field username: str | None = None#
pydantic model ibm_watsonx_data_integration.cpd_models.user_model_on_prem.UserMisc#

Bases: BaseModel

Per-user miscellaneous metadata.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

field dark_mode: bool | None = None#
field ext_attributes: dict | None = None#
field last_session_ended_timestamp: str | int | None = None#
field last_session_start_timestamp: str | int | None = None#
field realm_name: str | None = None#
field session_info: list[SessionInfo] | None = None (alias 'sessionInfo')#
pydantic model ibm_watsonx_data_integration.cpd_models.user_model_on_prem.UserProfileOnPrem#

Bases: BaseModel, ProjectCollaboratable

Model representing an identity user in the platform.

__init__ for User.

field authenticator: Authenticator | str [Required]#
field created_timestamp: int [Required]#
field current_account_status: AccountStatus | str [Required]#
field deletable: bool [Required]#
field display_name: str [Required] (alias 'displayName')#
field email: str [Required]#
field group_roles: list[str] [Optional]#
field groups: list[Group] [Optional]#
field internal_user: bool [Required]#
field last_modified_timestamp: int [Required]#
field misc: UserMisc | None = None#
field permissions: list[str] [Optional]#
field profile_picture: str | None = None#
field role: str | None = None#
field user_id: int [Required] (alias 'uid')#
field user_roles: list[str] [Optional]#
field username: str [Required]#
model_post_init(_BaseModel__context: Any)#

Don’t allow extra variables after initialization.

Return type:

None

property type: str#

This property returns the member type “user”.

class ibm_watsonx_data_integration.cpd_models.user_model_on_prem.UserProfilesOnPrem(platform: Platform)#

Bases: CollectionModel[UserProfileOnPrem]

Collection of User instances backed by the identity API.

__init__ for UserProfilesOnPrem.

Section Contents