Skip to content

Command Line Interface🔗

The command line tool is: ascli

The aspera-cli gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):

  • IBM Aspera High Speed Transfer Server (FASP and Node)
  • IBM Aspera on Cloud (including ATS)
  • IBM Aspera Faspex
  • IBM Aspera Shares
  • IBM Aspera Console
  • IBM Aspera Orchestrator
  • And more...

ascli provides the following features:

  • Commands to Aspera server products (on-premise and SaaS)
  • Any command line options (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files, ...
  • Commands, Options, and Option values shortcuts
  • FASP Transfer Agents can be: local ascp, or Connect Client, or any transfer node
  • Transfer parameters can be altered by modification of transfer-spec, this includes requiring multi-session
  • Allows transfers from products to products, essentially at node level (using the node transfer agent)
  • FASPStream creation (using Node API)
  • Watchfolder creation (using Node API)
  • Additional command plugins can be written by the user
  • Download of Faspex and Aspera on Cloud "external" links
  • Legacy SSH based FASP transfers and remote commands (ascmd)

Basic usage is displayed by executing:

ascli -h

See Usage.

Note

Not all features of ascli are documented here: explore commands on the command line, using -h.

Command Line Arguments🔗

Command line arguments are the units of command line typically separated by spaces (the argv of C).

ascli handles the following types of command line arguments:

  • Options: absolute position is not important, but relative order is, as a given option may be provided several times
  • Plugins for example, config, on first position
  • Resource Types for example, users
  • Verbs for example, create, to act on those resources or plugins.
  • Identifiers for example, 123, uniquely identifying a resource
  • Command Parameters arguments, for example, creation data, can be an Extended Value.

Command line arguments that are not options are referred to as Positional Arguments.

For example:

ascli plugin command verb --option-name=VAL1 VAL2
  • Positional Arguments: plugin command verb VAL2
  • One Option: option_name and its value: VAL1

Enumeration values (positional arguments and option names/values) support prefix matching. See Enumerations for details.

For example ascli config pre ov --for=c is the same as ascli config preset overview --format=csv.

The value of Options and Command Parameters is evaluated with the Extended Value syntax.

Positional Arguments🔗

When options (beginning with --) are removed from the command line, the remaining arguments are Positional Arguments with a significant, pre-defined order.

Plugins, Resource Types and Verbs are Enumerations.

Identifiers and Command Parameters are open values, but a given type is expected, either a simple String or an Extended Value.

The general structure of positional arguments is:

ascli <PLUGIN> <RESOURCE> <COMMAND> [<ID>] [<PARAMETERS>]

If a sub-resource is used:

ascli <PLUGIN> <RESOURCE1> <SUB_RESOURCE> [<ID>1] <COMMAND2> [<ID2>] [<PARAMETERS>]

Example:

ascli config ascp info
  • ascli is the executable executed by the shell
  • config is the name of the plugin to be used
  • ascp is the name of the resource (singleton, so no identifier is provided)
  • info is the verb to be performed (action on selected resource)

Resource types🔗

They identify a type of resource accessible in a given context. A resource type can be a singleton, or have multiple instances (identified by their identifier). A plugin is a kind of resource type singleton. A resource type can also be a grouping of other resource types, for example admin or files, like a singleton.

Verbs🔗

Standard resource Verbs are: create, show, list, modify, delete. Some entities also support additional verbs. When such additional commands relate to a resource also accessible in another context, they are placed under the do command. Such subcommands appear after the resource identifier, for example, in ascli aoc admin node do <NODE_ID> browse /, browse is a subcommand of node.

Typically, the create verb takes resource creation data as a parameter. show, modify and delete take an identifier, unless manipulating a singleton. list typically uses the query, select options. list and show typically use the fields option.

Identifiers🔗

Identifiers uniquely identify a resource. They are typically located immediately after a verb like show, modify or delete, itself placed after the resource type, for example: user show foobar. Some resources can also be selected by a unique field other than the native identifier (typically: id), using the percent selector.

Percent selector🔗

The percent selector is a special syntax that lets you identify a resource by any unique field, instead of its default identifier (typically a numeric id).

Syntax: %<FIELD>:<VALUE>

  • The leading % marks the argument as a percent selector (not a plain identifier).
  • <FIELD> is the name of the field to match on (e.g. name, email).
  • <VALUE> is the expected value of that field.

The tool then looks up the resource whose <FIELD> equals <VALUE> and uses its native identifier internally.

For example, to show the user whose name field is john:

ascli aoc admin user show %name:john

Command Parameters🔗

Command Parameters are typically mandatory values for a command, such as resource creation data.

Note

A Command Parameter could also have been designed as an option. However, since it is mandatory and such data typically do not need to be stored in a configuration file, using a Command Parameter is the better design choice. The advantages over an option are: a shorter command line (no option name, only position), a clearly mandatory value, and a position that unambiguously indicates its role. Extended Value syntax is supported, so it is possible to retrieve a value from the configuration file (using @preset:) or an environment variable (using @env:).

If a Command Parameter begins with -, then either use the @val: syntax (see Extended Value), or use the -- separator (see below).

A few Command Parameters are optional: they are always located at the end of the command line.

Enumerations🔗

Enumerations are arguments that must match a value from a predefined list in a given context.

The following are enumerations:

  • Plugins, Resource Types, and Verbs (positional arguments)
  • Option names (for example, --log-level)
  • Some option values (for example, debug for --log-level)

Prefix matching: You can use a unique prefix instead of the full value, provided it uniquely identifies the value in that context.

Examples:

  • Positional: ascli config pre ov --for=c is the same as ascli config preset overview --format=csv
  • Option name: --log-l=debug is the same as --log-level=debug
  • Option value: --format=c is the same as --format=csv

Note

While prefix matching works for option names, using full names is recommended for clarity.

Error handling: If an invalid or missing enumeration value is provided, an error message displays the list of valid values for that context.

Dot-path Notation🔗

ascli uses a unified dot-path notation in several places on the command line and in output. Understanding this concept once makes it immediately usable everywhere it appears.

Dot-path notation is used in four distinct places in ascli:

Surface Syntax in/out read/write
Option names --opt.key.sub=value Input write
Positional arguments @: key.sub=value ... [END] Input write
Extended value @preset: @preset:key.sub Input read
Output field names --out.fields=key.sub Output read

A dot-path is a String where segments are separated by . (dot), each segment designating a key in a nested structure:

  • A string segment designates a key in a Hash (associative array).
  • An integer segment designates an index in an Array.

For example, the path a.b.0 means: key a, then key b, then the first element of an array.

When a value is assigned to the path (write with =), it is automatically converted to the simplest matching type: Boolean, Integer, Float, or String. Values true and yes are converted to Boolean true, and values false and no to false. For an option expecting a list of values that includes yes or no (e.g. --out.table.pivot=no), the Boolean is converted back to that value.

Note

A value of 1 will be automatically converted to an Integer. When a specific type is required for the value, the Extended Value syntax modifiers @json: or @ruby: can be used. For example: --opt.x=1 generates {"x": 1}. To get a String: --opt.x=@json:\"1\" or --opt.x=@ruby:%q{1}. Likewise, to get the String yes: --opt.x=@json:\"yes\".

Example: dot-path to JSON output

ascli config echo @: a.b=1 a.c=2 a.d.0=hello a.d.1=world --format=json
{"a":{"b":1,"c":2,"d":["hello","world"]}}

Example: JSON to dot-path output

ascli config echo @json:'{"a":{"b":1,"c":2,"d":["hello","world"]}}'
╭───────┬───────╮
│ field │ value │
╞═══════╪═══════╡
│ a.b   │ 1     │
│ a.c   │ 2     │
│ a.d   │ hello │
│       │ world │
╰───────┴───────╯

Positional Arguments with Dot-path🔗

The @: syntax is a specialized argument type used for in-place definition of nested data structures. It allows you to construct complex parameters (Hashes and Arrays) directly from the command line, serving as a readable alternative to providing a single, serialized JSON string.

Usage and Syntax:

The general syntax for this argument is:

@:[<MARKER>] <DOT_PATH>=<VALUE> [<DOT_PATH>=<VALUE>] ... [<MARKER>]
  • @:[<MARKER>]: A standalone argument that starts collecting dot-path assignments into a single data structure. The optional text after : sets the end marker. If it is empty, the marker is END.

  • <DOT_PATH>=<VALUE>: An assignment using the standard dot-path notation. Multiple assignments can be provided in sequence to build a complex object. The value is converted like any dot-path value, and can be an Extended Value, for example public_key=@file:~/.ssh/id_rsa.pub. Options placed among the assignments are not collected: they are processed as usual, for example @: a=1 --format=json b=2.

  • <MARKER> (END by default): An optional argument that ends the collection.

  • Without the marker: All remaining positional arguments are collected, and each one must contain =.

  • With the marker: Only arguments between @: and the marker are used for the structure. Arguments after the marker are passed to the command as normal positional arguments.

  • Example with a custom marker: @:STOP a=1 STOP.

Info

Use END whenever any positional argument must follow the object built with @: (for example, a file list, or any other subsequent positional parameter). Without END, those arguments are collected as dot-path assignments instead of being passed to the command: an argument without = causes an error, and an argument containing = is silently added to the structure.

Example: Sending a package with a file list using @: for package information.

ascli aoc packages send @: name="<TITLE>" recipients.0=user@example.com END file1.dat file2.dat

Danger

In the above example, removing END causes an error, because file1.dat is read as a dot-path assignment and does not contain =.

Note

@: can also be used as an option value (for example, --query=@: a=b). In that case, only positional arguments after the option's position in the original command line are consumed, so sub-commands before the option are not affected. Use END as usual to stop collection when further positional arguments must follow: ascli aoc tier_restrictions --query=@: a=b END other_arg.

Options🔗

Command-line options, such as --log-level=debug, follow these conventions:

  • Prefix: All options begin with --.
  • Naming: Option names on command line use lowercase letters and hyphens (-) as word separators. Option names in the configuration file use underscores (_) as word separators. Example: --log-level=debug is log_level in config file.
  • Values: An option's value is assigned using = (for example, --log-level=debug). Alternatively, the value can be provided as a separate, space-separated token (for example, --log-level debug). Both forms are equivalent.
  • Prefix Usage: Options support prefix matching (see Enumerations), though full names are recommended.
  • Optionality: Most options are optional; they either have a default value or do not require one.
  • Order: Options can appear in any position on the command line, but their order may affect processing.

Exceptions and Special Cases:

  • Short Forms

Some options have short forms. For example, -Ptoto and -P toto are both equivalent to --preset=toto and --preset toto. See the manual or -h for details. - Flags

Certain options are flags and do not require a value (for example, -N). - Option Terminator

The special option -- ends option parsing. All subsequent arguments, including those starting with -, are treated as positional arguments. - dot-path notation - Cumulative Hashes

When an option of type Hash is set, the value is deep-merged with any existing value. Setting to @json:{} starts from an empty hash (the existing value is discarded for that merge step). Setting to @none: explicitly disables the option: it is set to nil and any preset or default value is prevented from restoring it. This is useful, for example, to disable a vault or any Hash option that is configured in a default preset.

Example:

ascli config echo -- --sample
--sample

Note

Here, --sample is taken as an argument, and not as an option, due to --.

Options may have a (hard coded) default value.

Options can be placed anywhere on command line and are evaluated in order. Usually the last value evaluated overrides previous values, but some options are cumulative, for example, ts.

Options are typically optional: to change the default behavior. But some are mandatory, so they can be placed in a configuration file, for example: connection information.

The value for any options can come from the following locations (in this order, last value evaluated overrides previous value):

Environment variables starting with prefix ASCLI_ are taken as option values, for example, ASCLI_OPTION_NAME is for --option-name.

Option show_config dry runs the configuration, and then returns currently set values for options.

ascli --show-config outputs global options only, and ascli [plugin] --show-config outputs global and plugin default options. In addition, option --show-config can be added at the end of any full command line, this displays the options that would be used for the command.

A command line argument is typically designed as option if:

  • It is optional, or
  • It is a mandatory parameter with a default value that would benefit from being set persistently (that is, in a configuration file or environment variable, for example, URL and credentials).

Interactive Input🔗

Some options and Command Parameters are mandatory and others are optional. By default, ascli raises an error for missing mandatory options or Command Parameters.

The behavior can be controlled with:

  • --interactive=<yes|no> (default=no)
  • yes: Missing mandatory parameters/arguments are asked to the user.
  • no: Missing mandatory parameters/arguments raise an error message.
  • --ask-options=<yes|no> (default=no)
  • yes: Optional parameters/arguments are asked to the user.

Output🔗

Command execution will result in output (terminal, stdout/stderr). The information displayed depends on the action.

To redirect results to a file, use option --out.file.

Types of output data🔗

Depending on action, the output will contain:

Result Type Description
single_object Displayed as a two-dimensional table: one line per field, first column is field name, and second is field value. Nested hashes are collapsed.
object_list Displayed as a two-dimensional table: one line per item, one column per field.
value_list A table with one column.
empty nothing
status A message.
other_struct A complex structure that cannot be displayed as an array.

With a structured format (json, jsonpp, yaml, ruby), the output is always valid in this format: a status is a string, an empty list is [], and no result is null.

Enhanced display of special values🔗

Special values are highlighted as follows in format=table:

Value Display
nil <null>
empty String <empty string>
empty Array <empty list>
empty Hash <empty dict>

Example:

ascli config echo @json:'{"ni":null,"es":"","ea":[],"eh":{}}'
╭───────┬────────────────╮
│ field │ value          │
╞═══════╪════════════════╡
│ ni    │ <null>         │
│ es    │ <empty string> │
│ ea    │ <empty list>   │
│ eh    │ <empty dict>   │
╰───────┴────────────────╯

Filtering results🔗

The result of a command can be filtered on the server side, by the API, and then on the client side, by ascli. Filtering on the server side reduces the data transferred, while filtering on the client side accepts any condition.

Options are evaluated in this order:

Order of evaluation of the options that filter results

  1. --query: parameters sent to the API.
  2. --out.secrets: secrets are hidden first, so that an expression cannot reveal them.
  3. --out.query: JMESPath expression or Ruby lambda, applied to the data as returned by the API.
  4. --out.select: selection of items (rows).
  5. --out.fields: selection of fields (columns).
  6. Display in the format set by --out.format.
To Use
Limit what the API returns --query, with parameters max and pmax
Filter on any condition, reshape, count --out.query
Select items by the value of fields --out.select
Choose the fields displayed --out.fields

Option: --query: Parameters of the API🔗

The query option can generally be used to add URL parameters to commands that list resources. It takes a Hash, corresponding to key/value pairs that appear in the query part of the request.

For example: --query=@json:'{"p1":"v1","p2":"v2"}' leads to query: ?p1=v1&p2=v2.

If the same parameter needs to be provided several times, then it is also possible to provide an Array.

For example: --query=@json:'{"p":["v1","v2"]}' leads to query: ?p=v1&p=v2.

If PHP-style arrays are expected in the API, add [] to the name of the parameter.

For example: --query=@json:'{"p[]":["v1","v2"]}' leads to query: ?p[]=v1&p[]=v2.

Commands that read a list by pages (for example, aoc admin user list) also accept parameters max and pmax. They are processed by ascli, not sent to the API: they limit the number of items (max) and of pages (pmax) requested.

For example: --query.max=10 reads the first 10 items.

Tip

Use --query=help on any list command to display the available query parameters and their types, as documented in the API. For example: ascli faspex5 admin accounts list --query=help

Option: --out.query: JMESPath expression🔗

Option --out.query applies a JMESPath expression to the result of a command, when it is an object or a list. JMESPath is a standard query language for JSON, also used by option --query of the AWS and Azure CLIs.

The expression is applied to the data as returned by the API (secrets hidden), before options --out.select and --out.fields. Its syntax is checked before the command is executed. Results of other types (status, no result) are not changed.

Examples:

# Name and email of ATS administrators
ascli aoc admin user list --out.query='[?ats_admin].{name: name, email: email}'
# Number of users
ascli aoc admin user list --out.query='length(@)'
# Email of users, one per line, for use in a shell loop
ascli aoc admin user list --out.query='[].email' --format=text

Tip

Enclose the expression in single quotes for the shell. In JMESPath, literal values are enclosed in backticks, for example: [?size > `1000`]. An expression with string literals in single quotes ('...') is enclosed in double quotes, for example: "[?name=='Ann']".

For processing that JMESPath does not provide (regular expressions, string manipulation, grouping), the value can be a Ruby lambda (@ruby:): it is called with the data, and returns the new data, converted to JSON data (for example, symbols and dates become strings). It is refused at Security Level medium and high.

# Number of users per email domain
ascli aoc admin user list --out.query=@ruby:'->(users){users.group_by{|u|u["email"].split("@").last}.transform_values(&:count)}'

The result of the query is displayed according to its type: a list of objects as a table, a list of values with one value per line, a single value as is. A list of lists (for example, [].[name, email]) is displayed only with a structured format (json, yaml, ...): to display a table, or CSV, use a multi-select hash instead, for example, [].{name: name, email: email}.

Note

Option --query gives filtering parameters to the API, while option --out.query filters and reshapes the result received. Option --out.query is the equivalent of option --query of the AWS and Azure CLIs.

Option: --out.select: Selection of items🔗

A list of objects (object_list) can be filtered with option --out.select (or --select). Its value is either a Hash or a Proc:

  • Hash: an item is selected if each key of the Hash is a field of the item with the same value, and the same type: true (Boolean) is not "true" (String).
  • Proc: a Ruby lambda called with each item (Hash), that returns true to select it, or false to remove it.

Example:

ascli aoc admin user list --out.fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --out.select=@json:'{"ats_admin":true}'
╭────────────────┬─────────────────────┬───────────╮
│ name           │ email               │ ats_admin │
╞════════════════╪═════════════════════╪═══════════╡
│ John Curtis    │ john@example.com    │ true      │
│ Laurent Martin │ laurent@example.com │ true      │
╰────────────────┴─────────────────────┴───────────╯

In the above example, the same result is obtained with option:

--out.select=@ruby:'->(i){i["ats_admin"]}'

A key can be a dot-path, for example, directory_user.type, with all formats.

Option: --out.fields: Selection of fields🔗

Depending on the command, results may include by default all fields, or only some selected fields. It is possible to define specific columns to be displayed, by setting option --out.fields (or --fields).

The --out.fields option is a list that can be either a comma-separated list or an Extended Value Array.

Individual elements of the list can be:

  • property: add property to the current list
  • -property: remove property from the current list
  • DEF: default list of fields (that is the default, when not set)
  • ALL: all fields
  • A Ruby RegEx: using @ruby:'/.../', or @re:... add those matching to the list

To display a property inside a complex structure (Hash, Array), use dot-path notation, only with --format=table or csv (see option --out.flat). With other formats, use a JMESPath multi-select hash with option --out.query, for example: --out.query='[].{name: name, type: directory_user.type}'.

Examples:

  • a,b,c: the list of attributes specified as a comma-separated list (overrides the default)
  • @list:,a,b,c: Array Extended Value: same as above
  • @json:'["a","b","c"]': Array Extended Value: same as above
  • b,DEF,-a: default property list, remove a and add b in first position
  • @ruby:'/^server/': Display all fields whose name begins with server

--out.select or --out.query🔗

Option --out.select is shorter for a test of equality, while option --out.query accepts any condition.

Condition --out.select --out.query
Equal to a Boolean @json:'{"ats_admin":true}' '[?ats_admin]'
Equal to a String @json:'{"name":"Ann"}' "[?name=='Ann']"
Field of a nested object @json:'{"directory_user.type":"active_directory"}' "[?directory_user.type=='active_directory']"
Several conditions (and) @json:'{"ats_admin":true,"name":"Ann"}' "[?ats_admin && name=='Ann']"
Negation, or Ruby lambda (@ruby:) '[?!ats_admin]', or: see Recipes
Number comparison Ruby lambda (@ruby:) '[?size > `1000000`]'
Substring Ruby lambda (@ruby:) '[?contains(email, `@example`)]'
Choice of fields With option --out.fields '[].{name: name, email: email}'
Count Not possible 'length(@)'

Warning

  • Values are compared with their type. With --out.select, "true" (String) does not match true (Boolean). In JMESPath, [?ats_admin == `true`] matches, but [?ats_admin == 'true'] does not: '...' is a string literal.
  • After --out.query, the default fields of the command no longer apply: all the fields of the result are displayed. Choose fields in the expression (multi-select hash), or with option --out.fields.
  • Secrets are hidden before --out.query is applied, so that the expression cannot reveal them.

Recipes🔗

# Users of Active Directory, with user name and email
ascli shares admin user all list --out.query="[?directory_user.type=='active_directory'].{username: username, email: email}"
# Users without directory (field missing or null)
ascli shares admin user all list --out.query='[?directory_user == null].username'
# Number of ATS administrators
ascli aoc admin user list --out.query='length([?ats_admin])'
# Users who are ATS administrators, or have no email
ascli aoc admin user list --out.query='[?ats_admin || !email].name'
# Emails of a domain, one per line
ascli aoc admin user list --out.query='[?ends_with(email, `@example.com`)].email' --format=text
# Largest file of a folder
ascli node browse / --out.query='max_by([?type==`file`], &size).path'
# Files larger than 1 MB
ascli node browse / --out.query='[?size > `1000000`].{path: path, size: size}'
# Files modified since a date
ascli node browse / --out.query='[?mtime >= `"2025-06-01"`].path'

Note

The JMESPath implementation of ascli (gem jmespath) also compares strings with <, <=, > and >=, which the JMESPath specification defines only for numbers. Dates in ISO 8601 format, in the same time zone, are compared in chronological order.

The result in JSON (--format=json) can also be processed by other tools, such as jq, whose language differs from JMESPath: for example, jq 'map(select(.directory_user.type == "active_directory"))'.

Other filters🔗

Some commands provide their own filters:

  • Percent selector %<field>:<value>: selects a single resource by the value of one of its fields.
  • find of plugin node (gen4) and aoc files find: recursive search of files on the client side, with a glob or a Ruby lambda (see Operation find).
  • faspex5 packages list: a Ruby lambda as last argument (see Faspex 5: List packages).
  • console transfer: query parameter filter (see Transfer filter).
  • preview: options --filter and --skip-folders (see Candidate detection).

Option: --out.format🔗

The style of output can be set using option --out.format (or --format):

format Output formatting
table Text table (default)
text Value as String
ruby Ruby code
json JSON code
jsonpp JSON pretty printed
yaml YAML
csv Comma Separated Values
image URL or data for a picture/video
nagios Suitable for Nagios

By default, results of type single_object and object_list are displayed using format table.

Option: --out.table🔗

The way format: table and csv are generated can be customized with option --out.table which expects a Hash.

For format=table, options are the ones described in gem terminal-table.

For example, to display a table with thick Unicode borders:

ascli config preset overview --out.table=@ruby:'{border: :unicode_thick_edge}'

Note

Other border styles exist, not limited to: :unicode, :unicode_round.

By default, if the terminal supports Unicode (see --out.utf8), then border=unicode_round is used.

A special parameter is defined: str_lst_sep (String), default is \n. It defines how lists of strings are displayed. Alternatively, set to ,.

When the output is a terminal, the table is fitted to the terminal width: columns with text (e.g. description) are wrapped, the widest first. If not enough, other columns (names, identifiers) are wrapped too, preferably after a punctuation (-, _, ., /). If the table cannot fit, columns are narrowed to their minimum width (20 characters for text, 10 for others, or heading). Parameter width (Integer) sets the maximum table width (e.g. --out.table.width=100), also when the output is not a terminal.

For format=csv, options are described in gem csv.

For example, to display a CSV with headers and quotes:

ascli config echo @json:'[{"name":"foo","id":1},{"name":"bar","id":8}]' --format=csv --out.table=@json:'{"headers":true,"force_quotes":true}'

Option: --out.flat: Single level Hash🔗

This option controls how object fields are displayed for complex objects.

Effective only when format is table or csv to display single_object or object_list.

If value is no, then object's field names are only the first level keys of the Hash result and values that are Hash are displayed as such in Ruby syntax.

If value is yes (default), then objects are "flattened" using dot-path notation with a variation:

  • Final arrays are displayed as a list of values, one per line (see str_lst_sep in --out.table above)
  • Array of Hash with only name keys are displayed as comma separated list of values
  • Array of Hash with only name and value keys are displayed like a Hash with value of name as key.

Example: The result of the command is a single object:

ascli config echo @json:'{"A":"a","B":[{"name":"B1","value":"b1"},{"name":"B2","value":"b2"}],"C":[{"C1":"c1"},{"C2":"c2"}],"D":{"D1":"d1","D2":"d2"}}'
╭────────┬───────╮
│ field  │ value │
╞════════╪═══════╡
│ A      │ a     │
│ B.B1   │ b1    │
│ B.B2   │ b2    │
│ C.0.C1 │ c1    │
│ C.1.C2 │ c2    │
│ D.D1   │ d1    │
│ D.D2   │ d2    │
╰────────┴───────╯

For the same command, adding option --out.flat=no:

╭───────┬────────────────────────────────────────────────────────────────────────╮
│ field │ value                                                                  │
╞═══════╪════════════════════════════════════════════════════════════════════════╡
│ A     │ a                                                                      │
│ B     │ [{"name" => "B1", "value" => "b1"}, {"name" => "B2", "value" => "b2"}] │
│ C     │ [{"C1" => "c1"}, {"C2" => "c2"}]                                       │
│ D     │ {"D1" => "d1", "D2" => "d2"}                                           │
╰───────┴────────────────────────────────────────────────────────────────────────╯

Option: --out.table.pivot🔗

This option controls how result fields are displayed as columns or lines, when option format is set to table. Values are false (default), true or single. On command line, no and yes can be used as well (converted to Boolean, see dot-path), but a structured value (e.g. --out=@json:'{"table":{"pivot":true}}') expects a Boolean. There are two types of results that are affected by this option:

Result Description
single_object A single item with multiple fields.
object_list A list of items, each with multiple fields.

An item (object) is displayed in one of those 2 ways:

Display rows columns
Simple Fields field and value
Transposed Items Fields

The display of result is as follows:

Result false true single
single_object Simple Simple Simple
object_list Transposed Simple
(Multiple objects)
Simple if 1 object.
transposed if 2+ objects.

This parameter can be set as a global default with:

ascli config preset set GLOBAL out.table.pivot single

Examples:

Simulate a result by executing this command:

ascli config echo @json:'<JSON_VALUE>' --out.table.pivot=<no|yes|single>

Example 1: A list of one object

[{"user":{"id":1,"name":"toto"},"project":"blash"}]

Display with no (Transposed):

╭─────────┬───────────┬─────────╮
│ user.id │ user.name │ project │
╞═════════╪═══════════╪═════════╡
│ 1       │ toto      │ blash   │
╰─────────┴───────────┴─────────╯

Display with yes and single (Simple):

╭───────────┬───────╮
│ field     │ value │
╞═══════════╪═══════╡
│ user.id   │ 1     │
│ user.name │ toto  │
│ project   │ blash │
╰───────────┴───────╯

Example 2: A list of two objects:

[{"id":1,"speed":111},{"id":"2","speed":222}]

Display with no and single (Transposed):

╭────┬───────╮
│ id │ speed │
╞════╪═══════╡
│ 1  │ 111   │
│ 2  │ 222   │
╰────┴───────╯

Display with yes (multiple Simple):

╭───────┬───────╮
│ field │ value │
╞═══════╪═══════╡
│ id    │ 1     │
│ speed │ 111   │
╰───────┴───────╯
╭───────┬───────╮
│ field │ value │
╞═══════╪═══════╡
│ id    │ 2     │
│ speed │ 222   │
╰───────┴───────╯

Option: --out.level: Verbosity of output🔗

Output messages are categorized into three types:

  • info output contains additional information, such as the number of elements in a table
  • data output contains the actual output of the command (object, or list of objects)
  • error output contains error messages

The option --out.level controls the level of output:

  • info displays all messages: info, data, and error
  • data displays data and error messages
  • error displays only error messages

Option: --out.secrets: Hide or show secrets in results🔗

  • If value is no (default), then secrets are redacted from command results: replaced with 🔑, or *** if the terminal does not support Unicode (see --out.utf8).
  • If value is yes, then secrets are shown in clear in results.

Terminal rendering: Colors and UTF-8🔗

By default, ascli detects the capabilities of the terminal:

Option Effect when yes Auto-detection (default)
--out.colors ANSI colors and styles in results, messages and logs no if NO_COLOR is set or results are written to a file (--out.file), else yes if CLICOLOR_FORCE is set and not 0, else yes if both stdout and stderr are terminals, TERM is not dumb and CLICOLOR is not 0
--out.utf8 Unicode characters: table borders, check marks, hidden secrets, symbols in messages, half blocks in images (else ASCII equivalents) yes if stdout is a terminal and the locale is UTF-8, or on Windows, the terminal is Windows Terminal, VS Code, Git Bash, ConEmu, or sets TERM to xterm*

Set either option to yes or no to override detection, for example to keep colors when piping to less -R, or to get plain ASCII output in a terminal:

ascli config preset overview --out.colors=no --out.utf8=no

Note

Logs issued before the option is processed (e.g. at startup) use the detected value.

Exit status🔗

The exit code of ascli indicates the category of error, so that scripts can react to it:

Code Description
0 Success
1 Other error
2 Command line error: unknown command or option, invalid or missing argument
3 Authentication or authorization failure: SSH, HTTP 401 or 403, OAuth grant
4 Resource not found, or several resources match the identifier, HTTP 404
5 Transfer failure
6 Network or TLS failure: name resolution, connection, timeout, certificate
7 Other error returned by a remote API
130 Interrupted by user (Ctrl-C)

With format nagios, the exit code is the status of the check: 0 (OK), 1 (WARNING), 2 (CRITICAL), and 3 (UNKNOWN) if the command fails, following the Nagios plugin guidelines.

Example:

ascli aoc files download /report.csv
case $? in
  0) echo 'Downloaded' ;;
  6) echo 'Network failure, retry later' ;;
  *) echo 'Failed' ;;
esac

Extended Value Syntax🔗

An Extended Value is a structured value - typically a Hash (key/value dictionary) or Array, potentially nested - that is passed on the command line wherever a simple string is not sufficient.

Most options and arguments are specified by a simple string (for example, username or url). However, some situations require a complex structure:

  • An option expects a structured value (Hash, Array) rather than a plain string.
  • The value is too long or complex to type inline (for example, a PEM private key, a JSON body).
  • The value must be read from a file, an environment variable, or a URL at runtime rather than embedded in the command.
  • The value needs type coercion: for example, an integer 1 instead of the string "1", or a boolean true.

The Extended Value syntax addresses all of these needs with a uniform, composable mechanism. It lets you specify what the value is and how to obtain or decode it, entirely on the command line.

Syntax and Decoders🔗

The syntax is:

<0 or more decoders><some text value or nothing>

Decoders act like a function with its parameter on right-hand side and are recognized by the prefix: @ and suffix :

@<NAME>:

Available Decoders🔗

The following decoders are supported:

Decoder Parameter Returns Description
base64 String String Decode a base64 encoded string.
csvt String Array Decode a titled CSV value.
env String String Read from the named environment variable. For example, --password=@env:MYPASSVAR
file String String Read value from specified file (prefix ~/ is replaced with the user's home folder). For example, --key=@file:~/.ssh/mykey
json String Any Decode JSON values. Convenient to provide complex structures.
lines String Array Split a string in multiple lines and return an Array.
list String Array Split a string in multiple items taking first character as separator and return an Array.
none None Nil A null value.
path String String Performs path expansion on specified path (prefix ~/ is replaced with the user's home folder). For example, --config-file=@path:~/sample_config.yml
preset String Any Get value from configuration file using dot-path notation: a whole preset (Hash) or a single value.
extend String String Evaluates embedded Extended Value syntax in string.
re String Regexp Ruby Regular Expression (short for @ruby:/.../)
ruby String Any Execute specified Ruby code.
s Any String Converts argument to String.
secret String String Ask password interactively: on terminal (hides input), or on a web page if option ui is graphical. Argument is the prompt.
stdin String String Read from stdin in text mode. Argument: <empty>, bin or chomp.
uri String String Read value from specified URL. Supported schemes: http:, https:, data:, file:. For example, --fpac=@uri:http://serv/f.pac or --key=@uri:file:/path/to/key.pem
val String String Prevent decoding by the decoders on the right. For example, --key=@val:@file:foo sets the option key to value @file:foo.
vault String String Read a secret from the Secret Vault: <label>.<field>, or a reference //<store>/<path>[?<options>][#<field>]. For example, --password=@vault:my_label.password
yaml String Any Decode YAML.
zlib String String Decompress data using zlib.
<empty> String Any The dot-path modifier: argument @: collects the following positional arguments as key.subkey=value assignments and builds a Hash or Array using dot-path notation. Shell-friendly alternative to @json: for structured values.
Argument is the end marker: <empty> for END, or a custom marker (e.g. @:STOP). Use the end marker to stop collection when further positional arguments must follow.
See Positional Arguments with Dot-path for full syntax.

Note

A few commands support a value of type Proc (lambda expression). For example, the Extended Value @ruby:'->(i){i["attr"]}' is a lambda expression that returns the value for key attr of the Hash parameter named i.

Chaining rule: When multiple decoders are combined, they are applied right-to-left: the rightmost decoder runs first on the literal string, and each decoder to its left receives the output of the one to its right.

To display the result of an Extended Value, use the config echo command.

The extend decoder is useful to evaluate embedded Extended Value syntax in a string. It expects a @ to close the embedded Extended Value syntax.

Option parser allows definition of a default parser when the positional parameter or option expects a Hash or Array. For example, with --parser=json, the parameter {} will be parsed as an empty JSON Hash, even without prefix @json:.

Syntax Equivalences🔗

There are four main ways to express a structured value (for example, a Hash with two keys). They are all equivalent and produce the same result:

Syntax Example Best for
@json: @json:'{"name":"alice","age":30}' Scripts, API payloads, copy-paste from docs
@yaml: @yaml:@stdin: + here-doc Multi-line or deeply nested structures
@ruby: @ruby:'{"name"=>"alice","age"=>30}' Advanced use: lambdas, file reads, logic
dot-path (key.subkey=value) via @: @: name=alice age=30 Interactive CLI, simple structures, no quoting

All four examples above produce the same Hash: {"name" => "alice", "age" => 30}.

ascli config echo @json:'{"name":"alice","age":30}' --format=json
ascli config echo @ruby:'{"name"=>"alice","age"=>30}' --format=json
ascli config echo @: name=alice age=30 --format=json
{"name":"alice","age":30}

Note

The dot-path form (key.subkey=value assignments via @:) does not need quotes for simple string values and is the most shell-friendly. The @json: form requires enclosing in single quotes to protect " from the shell. The @ruby: form accepts both single and double quotes and supports arbitrary Ruby expressions. The @yaml: form is most convenient when combined with a shell here-document (<<EOF).

Equivalences: Positional Parameter vs Option🔗

The same Hash value {"a":1,"b":"two"} can be expressed differently depending on whether it is supplied as a positional parameter or as an option.

As a positional parameter (argument to a command):

# JSON form
some_command @json:'{"a":1,"b":"two"}'
# Dot-path form (END is optional when no positional argument follows)
some_command @: a=1 b=two
some_command @: a=1 b=two END

As an option named opt:

# JSON form
--opt=@json:'{"a":1,"b":"two"}'
# Dot-path form via option name (each key as a separate option flag)
--opt.a=1 --opt.b=two
# Dot-path form via option value (consumes following positional arguments)
--opt=@: a=1 b=two END

Note

--opt.a=1 --opt.b=two and --opt=@json:'{"a":1,"b":"two"}' and --opt=@: a=1 b=two all produce the same Hash assigned to option opt. The --opt.key=value form is the most concise when only a few keys need to be set. Use END with @: when further positional arguments follow on the command line; it is optional otherwise.

Which Syntax to Choose?🔗

When in doubt: start with dot-path (--opt.key=value for options, @: key=value for positional arguments). It requires no quoting, auto-converts types, and reads naturally on the command line. Switch to @json: when you already have a JSON payload from an API doc or external tool.

Context Recommended syntax Reason
Interactive CLI, simple values dot-path key.subkey=value via @: or --opt.key=value No quoting needed, auto-typed, readable
Shell scripts, API payloads @json: Standard, portable, easy to copy from API docs
Multi-line or nested structures in scripts @yaml: + @stdin: heredoc Readable, no escaping
Dynamic values, lambdas, file reads @ruby: Full Ruby expressiveness

Common Usage Examples🔗

Example: Create a Hash value - for a small structure, the dot-path key.subkey=value form (via @:) is simpler; for larger or API-sourced structures, use @json::

ascli config echo @: key1=value1 key2=value2
ascli config echo @json:'{"key1":"value1","key2":"value2"}'

Example: read the content of the specified file, then, base64 decode, then unzip:

ascli config echo @zlib:@base64:@file:myfile.dat

Example: Create a Hash value with one key and the value is read from a file:

ascli config echo @ruby:'{"token_verification_key"=>File.read("mykey.txt")}'

Example: read a CSV file and create an Array of Hash for bulk provisioning:

cat test.csv
name,email
lolo,laurent@example.com
toto,titi@tutu.tata
ascli config echo @csvt:@file:test.csv
╭──────┬─────────────────────╮
│ name │ email               │
╞══════╪═════════════════════╡
│ lolo │ laurent@example.com │
│ toto │ titi@tutu.tata      │
╰──────┴─────────────────────╯

Example: create a Hash with values coming from a preset named config

ascli config echo @json:@extend:'{"hello":true,"version":"@preset:config.version@"}'
╭─────────┬────────╮
│ field   │ value  │
╞═════════╪════════╡
│ hello   │ true   │
│ version │ 4.21.1 │
╰─────────┴────────╯

Example: Create a Hash from YAML provided as shell Here document:

ascli config echo @yaml:@stdin: --format=json<<EOF
key1: value1
key2:
- item1
- item2
key3:
  key4: value4
  key5: value5
EOF
{"key1":"value1","key2":["item1","item2"],"key3":{"key4":"value4","key5":"value5"}}

Schema Discovery with help🔗

Any option or Command Parameter that expects a Hash value accepts the special value help. Instead of performing the action, ascli displays the schema of the expected Hash as a table (field name, type, description), then exits with an error.

When a mandatory Hash argument is missing, ascli automatically hints to use help:

ascli aoc packages send
ERRR Missing: Missing argument: package (Hash)
HINT:Give `help` as argument to retrieve the schema of the missing argument.

Following the hint and passing help as the argument displays the schema:

ascli aoc packages send help
INFO Schema: argument: package (Hash)
╭───────────────────────┬────────┬──────────┬────────────────────────────────────────────────────────────────────────────────────────────╮
│ name                  │ type   │ required │ description                                                                                │
╞═══════════════════════╪════════╪══════════╪════════════════════════════════════════════════════════════════════════════════════════════╡
│ bcc_recipients        │ array  │ false    │ <empty string>                                                                             │
│ bcc_recipients[].id   │ string │ true     │ The ID of the recipient.                                                                   │
│ bcc_recipients[].type │ enum   │ false    │ The entity type of the recipient.                                                          │
│                       │        │          │ Allowed: user, group                                                                       │
│ name                  │ string │ true     │ Package name. Required for POST. Optional for PUT.                                         │
│ note                  │ string │ false    │ The sender's message to recipients to include with the package. Maximum characters: 65535. │
│ recipients            │ array  │ false    │ <empty string>                                                                             │
│ recipients[].id       │ string │ true     │ The ID of the recipient.                                                                   │
│ recipients[].type     │ enum   │ false    │ The entity type of the recipient.                                                          │
│                       │        │          │ Allowed: user, group                                                                       │
...
╰───────────────────────┴────────┴──────────┴────────────────────────────────────────────────────────────────────────────────────────────╯

The same applies to options: display the schema of the transfer-spec option ts:

ascli --ts=help
INFO Schema: option: ts
╭─────────────────────┬─────────┬──────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ name                │ type    │ required │ description                                                                                                            │
╞═════════════════════╪═════════╪══════════╪════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
│ apply_local_docroot │ boolean │ false    │ Apply local docroot to source paths.                                                                                   │
│ cipher              │ enum    │ false    │ In transit encryption algorithms.                                                                                      │
│                     │         │          │ Allowed: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm │
│ authentication      │ string  │ false    │ Set to `token` for SSH bypass keys, else password asked if not provided.                                               │
...
╰─────────────────────┴─────────┴──────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

This works for any Hash option or positional parameter that has a defined schema.

Schema Validation🔗

The value of an option or Command Parameter with a schema defined by ascli (e.g. options ts, transfer, out, http_options, argument of orchestrator workflows start) is validated against it before the action. An invalid value is rejected with the path of the invalid element and the reason:

ascli config echo 1 --ts=@json:'{"direction":"sideways"}'
ERRR Argument: Option ts: value at `/direction` is not one of: ["send", "receive"] (use --ts=help for schema)
Use option -h to get help.

Option values are merged from several sources (presets, command line), so mandatory fields are not checked for options. Request bodies of product APIs (e.g. create and modify commands) are not validated by ascli: the API validates them.

Testing Extended Value🔗

Two complementary commands help verify that a value is parsed as expected:

What to check Tool How
Value of a positional parameter config echo Pass the expression as the sole argument
Value of an option --show-config Append --show-config to any command line

Checking a positional parameter with config echo:

config echo takes exactly one argument which can use the Extended Value syntax, and displays its parsed value. Unprocessed command line arguments are shown in the error message.

ascli config echo @json:'{"a":1,"b":"two"}' --format=json
ascli config echo @: a=1 b=two --format=json
{"a":1,"b":"two"}

Note

The value is shown after both shell command-line parsing and ascli Extended Value parsing.

Example: the shell parses three arguments (1, 2, 3), but config echo only processes the first:

ascli config echo 1 2 3
1
ERRR Argument: unprocessed values: ["2", "3"]

Checking an option value with --show-config:

Adding --show-config to any command line performs a dry run and displays the resolved value of all options that would be used, without executing the command.

To display a specific option, add --fields=<option_name>. Add --out.flat=no when the option holds a structured value (Hash, Array) to display it as-is rather than flattened into dot-path keys:

ascli --opt=@json:'{"a":1,"b":"two"}' some_plugin --show-config --fields=opt --out.flat=no
ascli --opt.a=1 --opt.b=two some_plugin --show-config --fields=opt --out.flat=no

Both lines above display the same resolved value for option opt.

In the following examples (using a POSIX shell, such as bash), several equivalent commands are provided. For all examples, most special character handling is not specific to ascli: It depends on the underlying syntax: shell, JSON, and so on. Depending on the case, a different format option is used to display the actual value.

For example, in the simple string Hello World, the space character is special for the shell, so it must be escaped so that a single value is represented.

Double quotes are processed by the shell to create a single string argument. For POSIX shells, single quotes can also be used in this case, or protect the special character (space) with a backslash.

ascli config echo "Hello World" --format=text
ascli config echo 'Hello World' --format=text
ascli config echo Hello\ World --format=text
Hello World

Security Level🔗

On the command line, ascli can do anything that the user running it is allowed to do. Option security restricts the features that access local resources or execute code, for example, when commands come from another program, such as an MCP client (see MCP Security). It is a Hash with the following keys:

Key Type Default Description
level string low low, medium, or high
allowed_paths Array<string> - With level medium: files and folders that @file: can read
writable_paths Array<string> - With levels medium and high: folders where files can be written
Level @file: @ruby:, @uri:, and @env: Secret processed by another extended value Write files Features that execute code
low Allowed Allowed Allowed Allowed Allowed
medium Only files in allowed_paths Refused Refused Only in writable_paths Refused
high Refused Refused Refused Only in writable_paths Refused

With level medium, the real path of the file is checked: a symbolic link leading outside of allowed_paths is refused.

With levels medium and high, files are written only in writable_paths (none by default): downloaded files (config download, and transfers that receive files), because a file written where the client chooses could replace a file that is used later, for example a plugin or a shell startup file. The real path of the folder is checked: neither .. nor a symbolic link leading outside of writable_paths is accepted.

Note

Reading files is not restricted for transfers: a command that sends files (for example server upload) reads the files that it is given, whatever the level. The security level restricts extended values, not the paths of transfers.

The same applies to @file:<path> in the parameters of stores of option vault, and in option vault_password (for example, token), although the file is read when used, not when the option is evaluated.

A secret (read by @secret:, @vault:, or @preset:) is hidden in results by its value: processed by another extended value, it could be revealed (for example, split by @list:@vault:..., or with a short value in a text by @extend:). With levels medium and high, a secret must be the whole value of an option or argument, for example @preset:<PRESET_NAME>.password for option password, and it cannot be a parameter name of config preset set.

Features that execute code (own, or downloaded), load code, or write a file chosen by the client, are available only at level low:

Feature Why
Option notify_template ERB template: Ruby code
Option fpac Proxy auto configuration: JavaScript code
Option ui, key command Command executed to open a URL
Options sdk_url and locations_url The SDK, including ascp, is downloaded from this location, then executed
Option out, key file (and output), option pid_file Write (or truncate, or delete) a file chosen by the client
Option home Configuration folder: its presets, and its folder plugins (Ruby code) are used
Option plugin_folder Plugins are Ruby code
Plugins of the folder plugins of the configuration folder, and of option plugin_folder Ruby code: not loaded at other levels, also if the file was there before
Command config plugins create Creates Ruby code, loaded as a plugin
Command faspex5 postprocessing Executes scripts, or Ruby code

At other levels, an option is refused whatever the way it is set (command line, or preset), and a command is not listed by config commands. Values from the environment of the process (for example ASCLI_HOME) are not restricted: they are set by the one who starts the process, not by a client. A sub-option of an option is restricted in the same way: it is marked x-security: low in the schema of the option (for example, file of option out), and refused when given, by dot notation (--out.file=...) or in a Hash. A preset is used later, maybe at level low: at level medium and high, config preset set, update and initialize refuse such an option, and a value that contains an extended value that the level refuses (for example @val:@ruby:...).

Options are evaluated in the order of the command line: place --security before options that use extended values.

ascli config echo @file:/etc/hosts --security.level=high
ERRR Argument: Evaluation of @file: for argument: value: @file: extended value is disabled by security level high: use level medium with the file in security.allowed_paths (option security), or give the value itself

Main, configuration and Persistency Folder🔗

ascli looks for configuration and persistency files (token cache, file lists, persistency files) in the folder specified using option home. The default value is [User's home folder]/.aspera/ascli.

Note

The [User's home folder] is determined using Ruby's Dir.home method. Primary source: The HOME environment variable. On Windows: Ruby also checks %HOMEDRIVE%%HOMEPATH% and %USERPROFILE% (via rb_w32_home_dir). Additionally, ascli sets the %HOME% environment variable to the value of %USERPROFILE% if it exists and is valid. Therefore, on Windows, %USERPROFILE% is preferred because it is generally more reliable than %HOMEDRIVE%%HOMEPATH%.

The configuration folder can be displayed using:

ascli config folder
/Users/kenji/.aspera/ascli

Note

This is equivalent to displaying the value of the home option.

ascli --show-config --fields=home

Example (Windows):

set ASCLI_HOME=C:\Users\Kenji\.aspera\ascli

ascli config folder

C:\Users\Kenji\.aspera\ascli

When OAuth is used (AoC, Faspex 5), ascli keeps a cache of generated bearer tokens in folder persist_store located in the configuration folder by default. Option cache_tokens (yes/no) allows controlling if OAuth tokens are cached on file system, or generated for each request. The command config tokens flush clears that cache. Tokens are kept on disk for a maximum of 30 minutes and garbage collected after that. When a token has expired, then a new token is generated, either using a refresh_token if it is available, or by the default method.

Configuration file🔗

On the first execution of ascli, an empty configuration file is created in the configuration folder (ascli config folder). There is no mandatory information required in this file. Its use is optional, as any option can be provided on the command line.

Although the file is a standard YAML file, ascli provides commands to read and modify it using the config command.

All options for ascli can be set on command line, or by env vars, or using Option Preset in the configuration file.

A configuration file provides a way to define default values, especially for authentication options, thus avoiding having to always specify those options on the command line.

The default configuration file is: $HOME/.aspera/ascli/config.yaml (this can be overridden with option config_file).

The configuration file is a catalog of named lists of options, called: Option Preset. Then, instead of specifying some common options on the command line (for example, address, credentials), it is possible to invoke the ones of an Option Preset (for example, <PRESET_NAME>) using the option preset: --preset=<PRESET_NAME> or its shortcut: -P<PRESET_NAME>.

Format of configuration file🔗

The configuration file is a Hash in a YAML file.

Example:

config:
  version: 0.3.7
default:
  config: cli_default
  server: demo_server
cli_default:
  interactive: no
demo_server:
  url: ssh://demo.asperasoft.com:33001
  username: asperaweb
  password: <PASSWORD>

We can see here:

  • The configuration was created with ascli version 0.3.7
  • The default Option Preset to load for server plugin is: demo_server
  • The Option Preset demo_server defines some options: the URL and credentials
  • The default Option Preset to load in any case is: cli_default

Two Option Presets are reserved:

  • config contains a single value: version showing the version used to create the configuration file. It is used to check compatibility.
  • default is reserved to define the default Option Preset name used for known plugins.

The user may create as many Option Presets as needed. For instance, a particular Option Preset can be created for a particular application instance and contain URL and credentials.

Values in the configuration also follow the Extended Value syntax.

Note

If the user wants to keep the Extended Value syntax inside the configuration file, the user shall use the @val: prefix.

Example:

ascli config preset set <AOC_ORG> private_key @val:@file:"$HOME/.aspera/ascli/<PKEY_NAME>"

This creates the Option Preset:

<AOC_ORG>:
  private_key: "@file:/Users/laurent/.aspera/ascli/<PKEY_NAME>"

So, the key file is read only at execution time and is not embedded in the configuration file.

Note

The main use of the configuration file is to store collections of options. Nevertheless, it can be used to store any type of information, not only option values. For example, the special preset: default.

Option Preset🔗

An Option Preset is a collection of options and their associated values in a named section in the configuration file.

A named Option Preset can be modified directly using ascli, which will update the configuration file:

ascli config preset <show|delete|get|unset|set|initialize|update|ask> <PRESET_NAME> ...

The syntax of all config preset commands, including list and overview, is displayed with ascli config commands config preset.

The command initialize allows setting several options at once, but it deletes an existing configuration instead of updating it, and expects a Hash Extended Value.

ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"<PASSWORD>","ts":{"precalculate_job_size":true}}'

One can also use positional arguments with dot-path:

ascli config preset initialize demo_server @: url=ssh://demo.asperasoft.com:33001 username=asperaweb password=<PASSWORD> ts.precalculate_job_size=true

The command update creates an Option Preset from options provided on the command line, for example:

ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=<PASSWORD> --ts=@json:'{"precalculate_job_size":true}'

This creates an Option Preset demo_server with all provided options.

Note

update takes ALL options provided on the command line (starting with -- with a value).

The command set allows setting individual options in an Option Preset:

ascli config preset set <PRESET_NAME> <PARAMETER_NAME> <PARAMETER_VALUE>

Example:

ascli config preset set demo_server password <PASSWORD>

The parameter name supports dot-notation to set nested hash options. If an intermediate hash already exists, the new value is deep-merged into it (existing sibling keys are preserved).

ascli config preset set GLOBAL out.table.pivot single
ascli config preset set GLOBAL out.level data

The parameter value is automatically coerced to its natural type: integers, floats and booleans (true/yes/false/no) are stored as native YAML types rather than strings.

To delete a key from a preset, pass @none: as the value (evaluates to nil):

ascli config preset set GLOBAL out.table.pivot @none:

A full terminal based overview of the configuration can be displayed using:

ascli config preset overview

The list of Option Presets can be displayed using:

ascli config preset list

Any key starting with _ (underscore) in a preset is treated as a comment and is ignored when the preset is applied. This is useful to annotate presets directly in the configuration file:

demo_server:
  _comment: "Demo server - public credentials"
  url: ssh://demo.asperasoft.com:33001
  username: asperaweb
  password: <PASSWORD>

Note

Comment keys are preserved and visible in config preset overview for inspection, but are never injected as CLI options.

By convention, _comment holds a short description of the preset. The shell completion of preset names (e.g. -P <tab>) shows it next to each name. The wizard sets it for the presets it creates: _comment: "wizard: <application>".

A good practice is to not manually edit the configuration file and use modification commands instead. If necessary, the configuration file can be opened in a text editor with:

ascli config open

Note

This starts the editor specified by env var EDITOR if defined.

It is possible to load an Option Preset from within another Option Preset using the preset option. For example if pcommon is a preset with common options, and pspecific is a preset with specific options, then pspecific can load pcommon using:

ascli config preset set pspecific preset pcommon

When pspecific is loaded, then cumulative option preset will be set, and it will also load pcommon.

Special Option Preset: config🔗

This preset name is reserved and contains a single key: version. This is the version of ascli which created the file.

Special Option Preset: default🔗

This preset name is reserved and contains a Hash, where the key is the name of a plugin, and the value is the name of another preset. Usually, Option presets are used to contain pre-defined options and values, but this preset contains names of presets to be used by default for plugins.

When a plugin is invoked, the preset associated with the name of the plugin is loaded, unless the option --no-default (or -N) is used.

Note

The special plugin name config in the default preset points to a preset containing global default options (such as --log-level, --vault, --interactive), loaded before any plugin is invoked. To set global default options, use the GLOBAL keyword (see Plugin: config).

Operations on this preset are done using regular config operations:

ascli config preset set default _plugin_name_ _default_preset_for_plugin_

Note

default is not a command, it is the name of the special preset.

ascli config preset get default _plugin_name_
"_default_preset_for_plugin_"

Handling invalid filename characters🔗

Some commands of ascli may create files or folders based on input that may contain invalid characters for the local file system. The option invalid_characters allows specifying a replacement character for a list of characters that are invalid in filenames on the local file system and replaces them with the specified character.

The first character specifies the replacement character, and the following characters are the invalid ones. This is used when a folder or file is created from a value that potentially contains invalid characters. For example, using the option package_folder, a package name may contain characters not allowed, such as /. The default value is _<>:"/\|?*, corresponding to replacement character _ and characters not allowed on Windows.

Note

This option is different from the replace_illegal_chars parameter in aspera.conf, which applies to transfers only.

Managing temporary files🔗

Some temporary files may be needed during runtime, for example the file list of ascp. The composite option temp (use --temp=help to display this schema) controls them:

Field Type Description
clean Bool Delete temporary files created by the command when it ends. false keeps them, e.g. for debugging. Files left by previous commands are still deleted according to expiry.
Default: true.
expiry String, Number, Bool Age after which file lists of ascp left by previous commands (e.g. interrupted) are deleted, when a command starts. They are in folder filelists of the main folder (option home). A duration, e.g. 5d, 12h, 1w, 1h 30m, 1d 02:00:00, a plain number is seconds. false or null (@none:): never delete them. 0: delete all of them, including file lists of transfers in progress in other executions of ascli. Such a transfer fails if ascp reads its file list again: next session of a multi-session transfer, or resume after an error. Use 0 only when no other transfer is in progress, not in a permanent configuration.
Default: 5d.
folder String Folder for temporary files, e.g. certificate for WSS transfers, logs of transferd.
- @sys: temporary folder of the system for the current user (Ruby Etc.systmpdir).
- @env: from environment variables, e.g. TMPDIR (Ruby Dir.tmpdir).
Default: @sys.

A plain value is the folder, for example: --temp=/data/tmp.

Temporary files of a command are deleted when it ends, unless clean is false. If a command is interrupted (for example, killed), its ascp file lists remain: a later command deletes them when they are older than expiry.

Warning

--temp.expiry=0 also deletes the file lists of transfers in progress in other executions of ascli, which may then fail: use it only when no other transfer is in progress.

Tip

Options temp_folder and clean_temp are deprecated aliases for temp.folder and temp.clean.

Plugin: config: Configuration🔗

Plugin config provides general commands for ascli:

  • Option Preset operations (configuration file)
  • wizard
  • vault
  • ascp
  • transferd (deprecated, use plugin transferd)

The default preset for config is read for any plugin invocation: this allows setting global options, such as --log-level or --interactive. When ascli starts, it looks for the default Option Preset and checks the value for config. If set, it loads the options independently of the plugin used.

Note

If default.config is not set, ascli automatically uses global_common_defaults as the global preset name and saves that mapping in the configuration file the first time the GLOBAL keyword is used.

Tip

If you do not know the name of the global preset, you can use GLOBAL to refer to it.

Show current default (global) Option Preset (config plugin):

ascli config preset get default config
global_common_defaults

Set a global parameter:

ascli config preset set GLOBAL version_check_days 0

If you want to use a custom name instead of global_common_defaults:

ascli config preset set default config <GLOBAL_OPTIONS_NAME>
ascli config preset set GLOBAL version_check_days 0

Tested commands for config🔗

Note

Add ascli config in front of the following commands:

agents jobs cleanup
agents jobs list
agents jobs status <CONF_ASYNC_NODE_UPLOAD>
agents list
agents parameters connect
agents parameters direct
agents parameters node
agents show connect
agents show direct
ascp errors
ascp info --sdk-folder=sdk_test_dir
ascp products list
ascp schema --format=jsonpp
ascp show
ascp spec
check_update
coffee
coffee --log-level=trace2 --log-format=caller
coffee --ui=text
coffee --ui=text --out.img.text=true
coffee --ui=text --out.img=@json:'{"text":true,"double":false}'
commands
commands aoc --expand-mounts=yes
commands aoc files
commands server
detect @: url=app.example.com
detect @: url=https://f5.example.com/path
detect @: url=https://f5.example.com/path plugin=faspex5
detect @: url=https://node.example.com/path
detect @: url=https://server1.example.com/path
detect @: url=https://shares.example.com/path plugin=shares
detect @: url=https://tst.example.com/path plugin=faspio
detect @: url=https://tst.example.com/path plugin=httpgw
detect @: url=my_org plugin=aoc
doc
doc github transfer-parameters
doc local quick-start --ui=text
doc toc
echo '<svg viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg"><circle cx="50" cy="50" r="50" fill="#069"/></svg>' --format=image --out.img.text=true
echo -- --special-string
echo @base64:SGVsbG8gV29ybGQK
echo @csvt:@stdin:
echo @env:USER
echo @json:'[{"email":"a@x.com"},{"email":"b@x.com"},{"email":"c@y.org"}]' --out.query=@ruby:'->(users){users.group_by{|u|u["email"].split("@").last}.transform_values(&:count)}' --format=json
echo @json:'[{"name":"alice","size":10},{"name":"bob","size":2000}]' --out.query='[?size > `100`].name' --format=text
echo @json:'[{"user":{"id":1,"name":"foo"},"project":"bar"}]' --out.table.pivot=single
echo @json:'[{"user":{"id":1,"name":"foo"},"project":"bar"}]' --out.table.pivot=yes
echo @json:'{"empty":"","bool":true}' --out.colors=yes --out.utf8=yes
echo @lines:@stdin:
echo @list:,1,2,3
echo @secret: --ui=text --out.secrets=yes
echo @stdin:bin
echo @uri:/etc/hosts
echo @uri:file:/etc/hosts
echo @uri:http://ifconfig.me
echo @uri:https://ifconfig.me
echo @uri:https://ifconfig.me/ip --cert-stores=@ruby:[OpenSSL::X509::DEFAULT_CERT_DIR,OpenSSL::X509::DEFAULT_CERT_FILE] --http-options.ssl_options=@list:,CIPHER_SERVER_PREFERENCE,-NETSCAPE_CA_DN_BUG --insecure=yes
echo @vault:my_preset.password
echo @zlib:@stdin:
echo hello
email_test --notify-to=my_email_external
file
file --log-level=debug --log-format=@ruby:'->(s, d, p, m){"#{d.strftime("%Y-%m-%d %H:%M:%S")} - #{p} - #{s} - #{m}\n"}'
file --log-level=debug --log-format=default
file --log-level=debug --log-format=standard
file --log-level=debug --logger=syslog
folder
gem name
gem path
gem version
genkey my_key
genkey my_key 4096
image https://eudemo.asperademo.com/wallpaper.jpg
initdemo
open
options aoc
options server
options server --select=@json:'{"option":"--display"}' --fields=replacement
plugins create my_command .
plugins list
preset delete conf_name
preset initialize conf_name @json:'{"p1":"v1","p2":"v2"}'
preset list
preset overview
preset set GLOBAL sdk_folder 'product:IBM Aspera Connect'
preset set GLOBAL vault @: default=local stores.local.type=file stores.local.file=/secure/vault_file
preset set GLOBAL vault_password <PASSWORD>
preset set GLOBAL version_check_days 0
preset set conf_name param value
preset set default shares conf_name
preset show conf_name
preset unset conf_name param
preset update conf_name --p1=v1 --p2=v2
proxy_check --fpac=@file:proxy.pac https://eudemo.asperademo.com --proxy-credentials=@list:,user,pass
pubkey @file:my_key
remote_certificate chain https://node.example.com/path
remote_certificate name https://node.example.com/path
remote_certificate only https://node.example.com/path
smtp_settings
sync spec
tokens flush
tokens list
vault check @vault://local/my_label
vault create @: label=my_label password=<PASSWORD> 'description=my secret'
vault delete foo --vault.type=system
vault delete my_label
vault field my_label
vault ids
vault info
vault list
vault overview
vault show my_label
wizard @: url=https://console.example.com/path plugin=console override=yes --username=my_username --password=<PASSWORD>
wizard @: url=https://f5.example.com/path plugin=faspex5 override=yes --username=my_username --private-key=@file:<PRIVATE_KEY_FILE> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET>
wizard @: url=https://node.example.com/path plugin=node --username=test --password=<PASSWORD>
wizard @: url=https://orch.example.com/path plugin=orchestrator --username=test --password=<PASSWORD>
wizard @: url=https://server1.example.com/path plugin=server --username=my_username --password=<PASSWORD>
wizard @: url=https://shares.example.com/path plugin=shares --username=test --password=<PASSWORD>
wizard @: url=https://tst.example.com/path plugin=faspio override=yes --username=my_username --password=<PASSWORD>
wizard @: url=https://tst.example.com/path plugin=httpgw
wizard @: url=my_org plugin=aoc --username=my_user_email --private-key=@file:<PRIVATE_KEY_FILE> --use-generic-client=yes
wizard @: url=my_org plugin=aoc preset=my_preset --username=my_user_email --private-key=@file:<PRIVATE_KEY_FILE>

Evaluation order of options🔗

Some options are global, others are available only for some plugins (the plugin is the first-level command).

Options are loaded using this algorithm:

  • If option --no-default (or -N) is specified, then no default value is loaded for the plugin
  • Else it looks for the name of the plugin as key in section default, the value is the name of the default Option Preset for it, and loads it.
  • If option --preset=<NAME> is specified (or -P<NAME>), this reads the Option Preset specified from the configuration file by name.
  • If option --preset=<EXTENDED_VALUE_HASH> is specified, it is used as option values (Hash of option/value pairs).
  • Environment variables are evaluated.
  • Command line options are evaluated.

Options are evaluated in the order of command line.

To avoid loading the default Option Preset for a plugin, use: -N

On command line, words in option names are separated by a dash (-). In the configuration file, the separator is an underscore. For example, --xxx-yyy on command line gives xxx_yyy in configuration file.

The main plugin name is config, so it is possible to define a default Option Preset for the main plugin with:

ascli config preset set cli_default interactive no
ascli config preset set default config cli_default

An Option Preset value can be removed with unset:

ascli config preset unset cli_default interactive

Example: Define options using command line:

ascli -N --url=<URL> --password=<PASSWORD> --username=<USERNAME> node --show-config

Example: Define options using a Hash:

ascli -N --preset=@json:'{"url":"<URL>","password":"<PASSWORD>","username":"<USERNAME>"}' node --show-config

Shell Completion🔗

ascli supports shell tab-completion for Bash, Zsh, Fish, and PowerShell (Windows PowerShell 5.1 and PowerShell 7, on any OS). The command ascli config completion <shell> displays the completion script for the given shell.

The default shell is Zsh on macOS, Bash on most Linux distributions, and PowerShell on Windows (its prompt starts with PS). In Bash and Zsh, echo $0 displays the name of the running shell.

To activate completion, add the line for your shell to its startup file:

# Bash, in ~/.bashrc
eval "$(ascli config completion bash)"
# Zsh, in ~/.zshrc, after compinit
eval "$(ascli config completion zsh)"
# Fish, in ~/.config/fish/config.fish
ascli config completion fish | source
# PowerShell, in $PROFILE (edit with notepad $PROFILE)
ascli config completion powershell | Out-String | Invoke-Expression

The startup file then executes ascli each time a shell starts. Alternatively, save the script once in the completion folder of the shell (and save it again after an upgrade of ascli):

# Bash, with package bash-completion
ascli config completion bash > ~/.local/share/bash-completion/completions/ascli
# Zsh (the folder must be in $fpath before compinit)
ascli config completion zsh > ~/.zsh/completions/_ascli
# Fish
ascli config completion fish > ~/.config/fish/completions/ascli.fish

PowerShell has no completion folder: save the script, and dot-source it in $PROFILE:

ascli config completion powershell | Set-Content "$HOME\ascli_completion.ps1"
Add-Content -Path $PROFILE -Value ". '$HOME\ascli_completion.ps1'"

In PowerShell, the words proposed are kept for the session, so only the first Tab at a given position executes ascli.

Once active, press Tab to complete:

ascli <Tab>                            # plugins and global options: aoc, server, node, ..., --format, ...
ascli server <Tab>                     # sub-commands and options of plugin: upload, download, ls, ..., --url, ...
ascli aoc admin <Tab>                  # sub-commands at any depth: user, node, ...
ascli server ls --u<Tab>               # options, after any command: --url, --username, ...
ascli --format=<Tab>                   # values of option: --format=json, --format=table, ...
ascli --ts.<Tab>                       # keys of option, from its schema: --ts.direction=, --ts.target_rate_kbps=, ...
ascli --out.t<Tab>                     # a sub-structure is followed by a dot: --out.table.
ascli --ts.direction=<Tab>             # values of key: --ts.direction=send, --ts.direction=receive
ascli --transfer=<Tab>                 # plain value of option, key agent: --transfer=direct, --transfer=node, ...
ascli faspex5 admin accounts list --query.<Tab> # keys of --query, from the command: --query.limit=, ...
ascli server upload <Tab>              # local files
ascli server download --to-folder <Tab> # local folders
ascli config genkey <Tab>              # local files, where a local path is expected
ascli --temp.folder=<Tab>              # local folders, for a key of option
ascli --ts=@<Tab>                      # extended value modifiers: @json:, @file:, @preset:, @:, ...
ascli --private-key=@file:<Tab>        # local files, after @file: or @path:
ascli -P <Tab>                         # presets: also for config preset show, @preset:
ascli config preset get myconf <Tab>   # parameters of preset: url, username, ...
ascli config agents jobs status <Tab>     # asynchronous transfer jobs
ascli node access_keys do <Tab>        # type of expected value, not inserted: <identifier>
ascli aoc packages send <Tab>          # type, and keys of structure (required ones with *): <Hash> name*, workspace_id*, ...
ascli aoc packages send @: <Tab>       # keys of structure built with @:, from its schema: name=, workspace_id=, recipients.0., ...
ascli aoc packages send @: archived=<Tab> # values of key: archived=true, archived=false
ascli aoc packages send @: name=x <Tab> # end marker of structure built with @: END, and the other keys

No space is inserted after a word that continues: key=, key., @file:, or a folder (Bash 4 and later). Zsh, Fish and PowerShell also display the description of commands, options, keys and extended value modifiers. In PowerShell, a word starting with @ is inserted with quotes (e.g. '@file:README.md'), as required by PowerShell. Zsh lists options once - is typed (zsh style prefix-needed). The type of an expected value is displayed by Bash and Zsh, and by PowerShell in the list of MenuComplete (Ctrl+Space).

The scripts call ascli config completion candidates -- [<word>...] <current word>, which lists the candidates for the word at the cursor, with their description, followed by a directive for the shell: :files or :folders (local paths, followed by the beginning of the word that is not part of the path, if any, e.g. :files --private-key=@file:), :hint <text> or :. The words are not evaluated (e.g. @env: is not replaced). The command ascli config completion words [<word>...] lists the words that can follow the given ones. It can be used directly to inspect available commands:

ascli config completion words aoc admin

Wizard🔗

The wizard is a command that asks the user for information and creates an Option Preset with the provided information for a given application.

It takes one optional argument: a Hash, usually given with @: as key=value items, in any order:

Key Description If not provided, ascli
url URL or hostname of the application asks for it.
plugin The plugin name to limit search to tries to detect known plugins from the URL
preset The preset name to save in config file generates a unique preset name
default Set the preset as default configuration for the plugin (yes/no) yes
override Override the existing default preset of the plugin, if it exists (yes/no) no

Options of the plugin (for example --username, --password, --client-id, --private-key), see the plugin, are given as usual on the command line: they are added to the Option Preset created by the wizard. Options that the wizard needs, but that are not provided, are asked. An unknown key in the Hash is refused.

For example:

ascli config wizard @: url=<URL> plugin=<PLUGIN> preset=<PRESET>

The command config detect takes the same argument, with keys url and plugin, and shows the detected applications, without creating a preset. Use ascli config wizard help to show the schema of the argument.

The simplest invocation (without argument: the URL is asked) is:

ascli config wizard

If the application requires a private key, the user can provide it with option --private-key (the key, e.g. @file:<path>, not only its path), or let the wizard generate one. The user is told where to place the associated public key PEM in the application.

Example of configuration for a plugin🔗

For Faspex 5, Shares, Node (including ATS, Aspera Transfer Service), Console, only username/password and URL are required (either on command line, or from configuration file). Those can be provided on the command line:

ascli shares files browse / --url=https://10.25.0.6 --username=john --password=<PASSWORD>

This can also be provisioned in a configuration file:

ascli config preset set shares06 url https://10.25.0.6
ascli config preset set shares06 username john
ascli config preset set shares06 password <PASSWORD>

This can also be done with one single command:

ascli config preset initialize shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"<PASSWORD>"}'

Or:

ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=<PASSWORD>
ascli config preset set default shares shares06
  • Display the content of the configuration file in table format
ascli config preset overview
  • Execute a command on the Shares application using default options
ascli shares files browse /

Secret Vault🔗

Secrets, for example, passwords, keys, are needed when connecting to applications. Those secrets are usually provided as command options: on the command line, in env vars, in files, and so on.

For security reasons, those secrets shall not be exposed in clear, either:

  • On terminal during input
  • In logs
  • In command output

Instead, they shall be hidden (logs, output) or encrypted (configuration).

Terminal output (command result) secret removal is controlled by option --out.secrets (default: no). Log secret removal is controlled by option log_secrets (default: no). Secrets are identified by the name of their field (for example, password, token), and by their value: values of secret options and of presets, of Extended Values @secret: and @vault:, tokens obtained or generated, and secrets of transfer specs are hidden wherever they appear, also in field names. In presets, a secret made of digits only (for example, a password not quoted in the YAML file) is converted to a string, so that it is identified as a secret. A value shorter than 8 characters is hidden only when it is the whole value, not inside a text, except in responses of mcp server, where it is also hidden as a word (not next to a letter or digit). Mandatory command line options can be requested interactively (for example, password) using option interactive. It is possible to use Extended Value @secret:[name] to ask for a secret interactively. If option ui is graphical (see Graphical Interactions), the secret is entered on a web page opened in the browser, served by a local web server on 127.0.0.1, else on terminal. In MCP server mode, the secret is entered on a web page, see Secrets entered in a web page. It is also possible to enter an option as an environment variable, for example, ASCLI_PASSWORD for option password and read the env var like this:

read -s ASCLI_PASSWORD
export ASCLI_PASSWORD

Another possibility is to retrieve values from a secret vault.

Option vault declares named secret stores (stores), and the name of the default store (default):

  • @vault:<label>.<field> reads field <field> of secret <label> of the default store, managed with command config vault (see Vault: Operations).
  • @vault://<store>/<path>[?<options>][#<field>] reads a secret of any store (see Vault: Secret references).
Field Type Description
cache_ttl Number Time in seconds a value read by reference is kept in memory, 0 for no cache.
Default: 60.
default String Name of the default store, in stores: read by @vault:<label>.<field>, managed by config vault, unlocked with option vault_password. No default store if not set.
stores Hash Name of store (letters, digits, _, ., -) → parameters of store, with type. Secrets are read by reference with @vault://<store>/<path>[?<options>][#<field>] from any store.
Required.

For example, to use as default store an encrypted file vault.bin in the configuration folder (see Vault: Encrypted file):

ascli config preset set GLOBAL vault @json:'{"default":"local","stores":{"local":{"type":"file","file":"vault.bin"}}}'

Parameters of a store, with type (see sections below):

Field Type Description
type String vault: IBM HashiCorp Vault secret store. Labels of config vault and @vault:<label>.<field> are in mount secret (KV v2). Reference @vault://<store>/<mount>/<path>#<field> (field required) in any KV v2 mount, options version=<n>, mount=<mount> (when the mount contains /).
allowed_prefixes Array[String] Allowed prefixes of <mount>/<path> of references: a prefix ending with / is a folder, else a path or a folder. All paths are allowed if not set.
approle Hash AppRole authentication: the token is obtained by login, renewed, and obtained again when refused.
approle.mount String Mount path of the AppRole auth method.
Default: approle.
approle.role_id String Role ID.
Required.
approle.secret_id String Secret ID (if the role requires one): the value, or @file:<path>.
ca_file String File with the CA certificates that verify the certificate of the server (HTTPS).
namespace String Namespace (Vault Enterprise).
token String The authentication token. Its value, or @file:<path> (e.g. written and renewed by Vault Agent), read again when modified. One of token or approle is required. For the default store, defaults to the value of vault_password, unless approle is set.
Example: <TOKEN>.
type Enum[String] Vault type selector.
Required.
Allowed values: vault.
url String URL of the HashiCorp Vault server.
Required.
Example: http://127.0.0.1:8200.
type String system: macOS system keychain secret store.
name String Name of the keychain to use (file <name>.keychain-db in the keychain search list).
Default: ascli.
type Enum[String] Vault type selector.
Required.
Allowed values: system.
type String secret_service: Linux Secret Service secret store (GNOME Keyring, KWallet, KeePassXC...), using secret-tool.
name String Name of the group of secrets (attribute service of the items).
Default: ascli.
type Enum[String] Vault type selector.
Required.
Allowed values: secret_service.
type String file: Encrypted file secret store. Reference @vault://<store>/<label>[#<field>], field password by default.
file String Path to the encrypted vault file, created when the first secret is added. Relative paths are resolved from the configuration folder. Read when used, and again when modified.
Default: vault.bin.
name String Same as file (compatibility).
password String Password of the vault file: the value, or @file:<path>, read again when modified. For the default store, defaults to the value of vault_password.
type Enum[String] Vault type selector.
Required.
Allowed values: file.
type String 1password: 1Password secret store (Connect REST API or CLI).
account String Account shorthand for op (used when source is cli). Uses the default account if omitted.
source Enum[String] Backend source. Use api for the Connect REST API, cli for the op CLI.
Required.
Allowed values: api, cli.
Default: api.
token String Bearer token for the Connect API (required when source is api). For the default store, defaults to the value of vault_password.
type Enum[String] Vault type selector.
Required.
Allowed values: 1password.
url String Base URL of the 1Password Connect server (required when source is api).
Example: http://localhost:8080.
vault String Name or ID of the 1Password vault to use with the CLI (used when source is cli). Uses the default vault if omitted.
vault_id String ID of the 1Password vault to use (required when source is api).
type String keeper: Keeper Secrets Manager secret store (Keeper Vault), using gem keeper_secrets_manager. A secret is a record shared with the application: its label is its title or its UID. Reference @vault://<store>/<title or uid>[#<field>], field password by default: standard field by type (login, password, url...), custom field by label, or notes.
config String Configuration of the client device (base64 or JSON, e.g. from ksm profile export): the value, or @file:<path>. For the default store, defaults to the value of vault_password.
folder_uid String UID of the shared folder where config vault create adds records (not needed to read).
type Enum[String] Vault type selector.
Required.
Allowed values: keeper.

Note

The parameters of one store, without stores (for example, {"type":"file","name":"vault.bin"}), are deprecated: they define a store named after its type (for example, file), which is the default store. Set the store in stores, and its name in default instead: {"default":"local","stores":{"local":{"type":"file","file":"vault.bin"}}}.

Secrets used to access a store (token, approle.secret_id, password) are the value, or @file:<path> (for example, a credential of systemd, or a token written by Vault Agent): the file is read when used, and again when it is modified. As for extended value @file:, reading the file is restricted by option security (see Security Level). In a YAML file, quote the value: @ cannot start a plain scalar.

The secret of the default store (password of type file, token of type vault without approle, of type 1password with source api, and config of type keeper), if not in its parameters, is the value of option vault_password. To avoid exposing the vault password in the shell history, provide it via an environment variable:

export ASCLI_VAULT_PASSWORD=<YOUR_PASSWORD>

Tip

Set it interactively using read so that it is not recorded in the shell history:

read -rs ASCLI_VAULT_PASSWORD && export ASCLI_VAULT_PASSWORD

Vault: IBM HashiCorp Vault🔗

https://developer.hashicorp.com/vault

Quick start macOS:

brew tap hashicorp/tap
brew install hashicorp/tap/vault
vault server -dev -dev-root-token-id=<TOKEN>
--vault=@json:'{"default":"corp","stores":{"corp":{"type":"vault","url":"http://127.0.0.1:8200"}}}' --vault-password=<TOKEN>

With a token in a file (for example, written and renewed by Vault Agent), read again when the file is modified:

--vault=@json:'{"default":"corp","stores":{"corp":{"type":"vault","url":"https://vault.example.com:8200","token":"@file:/run/vault/token"}}}'

With AppRole authentication, the token is obtained by login, and renewed:

--vault=@json:'{"default":"corp","stores":{"corp":{"type":"vault","url":"https://vault.example.com:8200","approle":{"role_id":"<ROLE_ID>","secret_id":"@file:/etc/vault/secret_id"}}}}'

Vault: System keychain🔗

Warning

macOS only

It is possible to manage secrets in macOS keychain. Commands list, ids and delete are not supported: use the Keychain Access app.

--vault=@json:'{"default":"keychain","stores":{"keychain":{"type":"system","name":"ascli"}}}'

Vault: Linux Secret Service🔗

Warning

Linux only

It is possible to manage secrets in the Linux Secret Service (GNOME Keyring, KWallet, KeePassXC...), using the secret-tool command of libsecret (for example, package libsecret-tools on Debian and Ubuntu). A session with an unlocked keyring is required. Secrets are grouped with attribute service, which is parameter name (default: ascli).

--vault=@json:'{"default":"keyring","stores":{"keyring":{"type":"secret_service","name":"ascli"}}}'

Vault: Encrypted file🔗

It is possible to store and use secrets encrypted in a file, with a store of type file:

{"default":"local","stores":{"local":{"type":"file","file":"vault.bin"}}}

To configure the encrypted file vault as the default store, set the vault option in the global preset:

ascli config preset set GLOBAL vault @json:'{"default":"local","stores":{"local":{"type":"file","file":"vault.bin"}}}'

The vault file is created automatically when the first secret is added - no explicit initialization is needed. It is read when used, and again when it is modified.

Warning

The vault_password option should not be stored in the config file: doing so would protect secrets with a password that is itself stored in plain text, defeating the purpose of the vault. Use an environment variable as described above, or parameter password with @file:<path> (for example, a credential of systemd).

Vault: 1Password🔗

Use type: 1password with a source parameter to select the backend.

Source: API (Connect REST API)🔗

https://developer.1password.com/docs/connect/

1Password Connect exposes a local REST API backed by a 1Password vault. Deploy the Connect server with Docker, then point ascli at it:

docker run -d --name op-connect \
  -p 8080:8080 \
  -v /path/to/1password-credentials.json:/home/opuser/.op/1password-credentials.json \
  1password/connect-api:latest
--vault=@json:'{"default":"op","stores":{"op":{"type":"1password","source":"api","url":"http://localhost:8080","vault_id":"<VAULT_ID>"}}}' \
--vault-password=<CONNECT_TOKEN>

Tip

The vault_id is the alphanumeric ID shown in the 1Password web app under Vault Settings. The vault_password holds the Connect bearer token generated when creating the Connect server credentials.

Note

The 1Password Service Accounts API is also supported: use url=https://api.1password.com with a service account token as vault_password. No local server is required in that case.

Source: Cli (op CLI)🔗

https://developer.1password.com/docs/cli/

Requires the op CLI to be installed and signed in. No server to deploy: authentication is handled by the 1Password desktop app (biometric unlock) or by op signin.

--vault=@json:'{"default":"op","stores":{"op":{"type":"1password","source":"cli"}}}'
# or, to target a specific vault and account:
--vault=@json:'{"default":"op","stores":{"op":{"type":"1password","source":"cli","vault":"<VAULT_NAME>","account":"<ACCOUNT_SHORTHAND>"}}}'

Tip

Install op on macOS with brew install 1password-cli, then connect it to the desktop app: op signin

Referencing items🔗

An item can be referenced by its title (label) or by its 1Password ID, in @vault:<item>.<field>:

--password=@vault:<ITEM_ID>.password
--password=@vault:<ITEM_TITLE>.password

Note

If several items share the same title, use the item ID (or the vault parameter to restrict the search).

Vault: Keeper🔗

https://docs.keeper.io/keeperpam/secrets-manager/overview

Keeper Secrets Manager (KSM) gives an application access to the records of shared folders of a Keeper vault, without the credentials of a user. It requires the optional gem keeper_secrets_manager:

gem install keeper_secrets_manager

Create an application and a client device in Keeper (see the Quick Start Guide), share the folders of secrets with the application, and get the configuration of the device (base64 or JSON, for example with ksm profile export). Parameter config is the value, or @file:<path>:

--vault=@json:'{"default":"keeper","stores":{"keeper":{"type":"keeper","config":"@file:<KSM_CONFIG_FILE>"}}}'

If config is not set in the default store, it is the value of option vault_password.

A secret is a record of the vault: its label is its title, or its UID. The field of a reference is password by default: standard field of the record (login, password, url...), custom field by its label, or notes. Field username is an alias of login.

--password=@vault:<RECORD_TITLE>.password
--password=@vault://keeper/<RECORD_TITLE>#<FIELD>

To create records with config vault create (type login), set parameter folder_uid: UID of the shared folder where records are added. Reading does not need it.

Note

If several records share the same title, use the UID. Deleting a record with config vault delete requires that the application has edit permission.

Vault: Operations🔗

Secrets of the default store can be manipulated using the config vault command:

  • info : Show vault information
  • ids : List secret labels in the vault
  • list : List all secrets with full details
  • overview : List labels of all secrets with their field names, without values
  • field : List field names of a secret by label, without values
  • show : Show a secret by label (or id)
  • create : Add a new secret to the vault
  • delete : Delete a secret by label (or id)
  • password : Set a new password for the vault
  • import : Import secrets from a JSON array (supports --bulk)
  • check : Check that a Secret Reference can be read, without its value

To add a new password entry in the vault for label <NAME>:

ascli config vault create @: label=<NAME> password=@secret:password description='for this account'

Vault: Migration between vaults🔗

To migrate all secrets from one vault backend to another (for example, from the encrypted file vault to 1Password), use vault list piped into vault import.

Note

Use list (not ids) as the source: ids returns only labels, while list returns the full secret details needed for import.

ascli config vault list --format=json --out.level=data --out.secrets=yes \
  --vault=@json:'{"default":"local","stores":{"local":{"type":"file","file":"<SOURCE_VAULT_FILE>"}}}' \
  --vault-password=<SOURCE_PASSWORD> | \
ascli config vault import @json:@stdin: --bulk=yes \
  --vault=@json:'{"default":"op","stores":{"op":{"type":"1password","url":"<CONNECT_URL>","vault_id":"<VAULT_ID>"}}}' \
  --vault-password=<CONNECT_TOKEN>

Tip

Use --out.level=data on the list command so that only the raw JSON array is written to stdout, with no table headers or status lines, and --out.secrets=yes so that secrets are not replaced by a placeholder.

The import command accepts a JSON array where each element is a vault secret object (same schema as create). --bulk makes each entry reported individually in the result table; omit it to get a single-line summary. On error (for example a duplicate label), the default behavior is to stop (--bfail=yes). Pass --bfail=no to skip failures and continue with the remaining entries.

Vault: Secret references🔗

Secrets can also be read by reference, from any store of option vault:

@vault://<store>/<path>[?<options>][#<field>]
  • <store>: name of a store in stores of option vault.
  • <path>: path of the secret in the store, and <field>: field of the secret. Both are percent-decoded (for example, %20 for a space). For most types, the path is the label, and the field is password by default. For type vault, the path is <mount>/<path> in a KV v2 secrets engine, and the field is required.
  • <options>: options of the store, as a URL query (for example, version=2).

References only read secrets: they are managed with the tools of the store (for example, vault kv put for HashiCorp Vault), or with config vault for the default store. Compared to @vault:<label>.<field>, several stores can be used in the same command, and a HashiCorp Vault store reads any KV v2 mount, a given version, and limits the paths that can be read by reference (allowed_prefixes). Values read by reference are kept in memory for cache_ttl seconds. The configuration of stores is the same as in applications that use the library with references secret://<store>/<path> (for example, IBM Aspera Orchestrator).

For example, a HashiCorp Vault server with AppRole authentication, limited to secrets under kv/aspera/, and the encrypted file of config vault as default store, unlocked with option vault_password:

--vault=@json:'{"default":"local","stores":{"corp":{"type":"vault","url":"https://vault.example.com:8200","approle":{"role_id":"<ROLE_ID>","secret_id":"@file:/etc/vault/secret_id"},"allowed_prefixes":["kv/aspera/"]},"local":{"type":"file","file":"vault.bin"}}}'

References in a preset: to field password of secret aspera/faspex in mount kv, and to field password (default field of type file) of label <VAULT_LABEL> in the encrypted file:

ascli config preset update <PRESET_NAME> --url=... --username=... --password=@val:@vault://corp/kv/aspera/faspex#password
ascli config preset update <PRESET_NAME> --url=... --username=... --password=@val:@vault://local/<VAULT_LABEL>

To check that a reference can be read, without its value (store, version, time of update):

ascli config vault check @vault://corp/kv/aspera/faspex#password

Configuration Finder🔗

When a secret is needed by a sub command, the command can search for existing configurations in the configuration file by setting option secret to the special value PRESET.

The lookup is done by comparing the service URL and username (or access key).

Securing passwords and secrets🔗

A password can be saved in clear in an Option Preset together with other account information (URL, username, and so on). Example:

ascli config preset update <PRESET_NAME> --url=... --username=... --password=...

For more secure storage, first create a vault entry (see Vault: Operations), then refer to it in the preset:

ascli config preset update <PRESET_NAME> --url=... --username=... --password=@val:@vault:<VAULT_LABEL>.password

Note

Use @val: in front of @vault: so that the Extended Value is not evaluated.

To migrate existing clear-text passwords from presets to the vault in one step, use the secure command. The vault must already be configured (option vault, see Secret Vault).

ascli config preset secure

This scans all presets (or a single one if a name is provided) and, for every option whose name ends with password or secret, moves the value into the vault and replaces it with a @vault: reference.

ascli config preset secure <PRESET_NAME>

When a vault is configured, the update, set, and ask preset commands automatically move secret values (options ending with password or secret) into the vault as they are written, so no manual migration is needed. To disable this behavior for a single command, pass --vault=@none:.

Private Key🔗

Some Aspera applications allow the user to be authenticated using Public Key Cryptography:

  • For SSH: Server
  • For OAuth JWT: AoC, Faspex 5, faspio Gateway

It consists of using a pair of associated keys: a private key and a public key. The same pair can be used for multiple applications. The file containing the private key (key pair) can optionally be protected by a passphrase. If the key is protected by a passphrase, then the passphrase is prompted when the key is used. Some plugins support option passphrase.

Keys of type ed25519 and OpenSSH encoded keys (default on modern Linux) are supported. See section: Private key type ed25519. For OpenSSH encoded keys protected by a passphrase, install gem bcrypt_pbkdf (not available for JRuby):

gem install bcrypt_pbkdf

The following sample commands use the shell variable KEY_PAIR_PATH. Set it to the desired safe location of the private key. Typically, located in folder $HOME/.ssh or $HOME/.aspera/ascli. For example:

KEY_PAIR_PATH=~/.aspera/ascli/<PKEY_NAME>

Several methods can be used to generate a key pair.

The format expected for keys is PEM.

If another format is used, such as DER, it can be converted to PEM, for example, using openssl.

ascli for key generation🔗

The generated key is of type RSA, by default: 4096 bit. For convenience, the public key is also extracted with extension .pub. Files are PEM encoded. The key is not passphrase protected.

ascli config genkey ${KEY_PAIR_PATH} 4096

To display the public key of a private key:

ascli config pubkey @file:${KEY_PAIR_PATH}

ssh-keygen🔗

Both private and public keys are generated, option -N is for passphrase.

ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${KEY_PAIR_PATH}

openssl🔗

To generate a key pair with a passphrase, the following can be used on any system:

openssl genrsa -passout pass:<PASSPHRASE> -out ${KEY_PAIR_PATH} 4096
openssl rsa -pubout -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.pub

openssl is sometimes compiled to support option -nodes (no DES, that is, no passphrase, for example, on macOS). In that case, add option -nodes instead of -passout pass:<PASSPHRASE> to generate a key without passphrase.

If option -nodes is not available, the passphrase can be removed using this method:

openssl rsa -passin pass:<PASSPHRASE> -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.no_des
mv ${KEY_PAIR_PATH}.no_des ${KEY_PAIR_PATH}

To change (or add) the passphrase for a key:

openssl rsa -des3 -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.with_des
mv ${KEY_PAIR_PATH}.with_des ${KEY_PAIR_PATH}

Using an application to generate a key pair🔗

Many applications are available, including on the internet, to generate key pairs. For example: https://cryptotools.net/rsagen

Warning

Private keys are sensitive information, and shall be kept secret (like a password), so using online tools is risky.

Web service🔗

ascli is a command-line tool: it runs in a shell, with the credentials and presets of the user who starts it, and it assumes that only this user uses it. It is not intended to be a public-facing web server. When commands come from another program, for example an MCP client, see Security Level.

Some commands start a web server, for a specific client:

Command Client Protection required when url is not loopback
mcp server, http transport MCP client auth_token, see MCP Security
faspex5 postprocessing Faspex 5 web hooks secret_token or allowed_ips, see Faspex 4-style post-processing
faspex5 gateway, aoc gateway Faspex 4 client applications username and password, see Faspex 4 Gateway
node emulator Node API clients, for tests username and password, see Node emulator

By default, the server listens on a loopback address: localhost, an address of 127.0.0.0/8, or ::1. It can be reached only from the same host.

Warning

Loopback restricts the host, not the user: on a host shared with other users, they can also reach the server. On such a host, also set the protection of the command.

When the host of url is not a loopback address, for example 0.0.0.0 (all interfaces), the server can be reached from the network: the command refuses to start without its protection. On a network, use HTTPS (https:// in url) or a reverse proxy with TLS: with HTTP, passwords and tokens are sent in clear.

Other features listen on a loopback address only, to receive input from the browser of the user:

The server serves HTTP, or HTTPS with a certificate. The following parameters are supported:

Parameter Type Default Description
url String http://localhost:8080 Base URL on which requests are received; a path can be included.
cert String - (HTTPS) Path to certificate file (with ext. .pfx or .p12 for PKCS12).
key String - (HTTPS) Path to private key file (PEM), or passphrase for PKCS12.
chain String - (HTTPS) Path to certificate chain (PEM only).

Parameter url (base URL) defines:

  • If http or https is used
  • The local port number (default 443 for HTTPS, 80 for HTTP)
  • The base path, that is, the path under which requests are received (useful for routing when a reverse proxy is used).

A web page open in a browser can send requests to a local web server, also on localhost. To refuse them, the server checks each request:

  • When url is a loopback address (localhost, 127.0.0.0/8, ::1), header Host shall be a loopback name or address, else HTTP 403: a host name that resolves to 127.0.0.1 is refused (DNS rebinding).
  • Header Origin, sent by browsers, shall be the origin of the server, else HTTP 403.
  • A JSON body shall have Content-Type: application/json, else HTTP 415: a web page cannot send it to another site without the agreement of that site.

Clients that are not browsers send no Origin, and the Host of the URL they call. They shall send the type of a JSON body: for example, with curl, add -H 'Content-Type: application/json'.

Image and video thumbnails🔗

ascli can display thumbnails for images and videos in the terminal, using iTerm2 inline image protocol or through colorized text.

This feature can be used:

  • In the thumbnail command of node when using gen4/access key API.
  • When using the show command of preview plugin.
  • coffee and image commands of config plugin.
  • Any displayed value which is a URL to image can be displayed with option format set to image

The following options can be specified in option --out.img:

Field Type Description
double Bool Use double vertical resolution via half-block characters (Unicode). Default is true if the terminal supports Unicode (see utf8).
font_ratio Number Terminal font aspect ratio (height divided by width).
Default: 2.3.
reserve Integer Number of terminal lines reserved for non-image output.
Default: 3.
text Bool Display the image as colorized text instead of using the iTerm2 inline protocol.
Default: false.

Examples:

  • Display image as colorized text (requires rmagick)
ascli config image https://eudemo.asperademo.com/wallpaper.jpg --ui=text --out.img.text=true
  • Display image from byte stream as image in terminal (requires iTerm2-compatible terminal)
curl -so - https://eudemo.asperademo.com/wallpaper.jpg | ascli config image @stdin:bin
  • Display image from file (requires iTerm2-compatible terminal)
ascli config image @stdin:bin < A-team.jpg

Graphical Interactions: Browser and Text Editor🔗

Some actions may require the use of a graphical tool:

  • A browser for Aspera on Cloud authentication (web auth method)
  • A browser to enter a secret requested with @secret:
  • A text editor for editing the configuration file

By default, ascli assumes that a graphical environment is available on Windows and macOS; on other systems, it relies on the presence of the DISPLAY environment variable. The composite option ui (use --ui=help to display this schema) selects the mode, the way a web login is received, and the browser command:

Field Type Description
command Array[String] Command that opens a URL, instead of the system default. First item is the executable (full path, or name of an executable found in PATH), then its arguments. The URL is added as last argument. When set, it is used to open URLs, whatever the mode.
mode Enum[String] - auto: browser on Windows and macOS, or if env var DISPLAY is set, else terminal.
- terminal: URLs to open are displayed on terminal, and secrets are entered on terminal.
- browser: a browser opens URLs, and secrets are entered on a web page.
text is an alias for terminal, and graphical is an alias for browser.
Files to edit are always opened with a text editor, whatever the mode.
Allowed values: auto, terminal, browser, text, graphical.
Default: auto.
web_auth Enum[String] How the result of a web login (--auth=web, SSO) is received.
- port: ascli listens on the loopback port of redirect_uri, the browser is redirected to it.
- paste: no port is opened. After login, the browser shows a connection error on redirect_uri: copy the full URL from the address bar of the browser and paste it in the terminal. Use it when the browser cannot reach the host that runs ascli and a SSH tunnel is not possible. For shares, Shares displays a code (a long text of letters, digits, - and _) instead: paste it in the terminal.
Allowed values: port, paste.
Default: port.

A plain value is the mode:

  • --ui=browser (or graphical) forces a graphical environment: a browser is opened for URLs or secrets.
  • --ui=terminal (or text) forces a text environment: the URL to open is displayed on terminal, and secrets are entered on terminal.
  • --ui=auto selects the mode from the environment (default).

The browser is opened, by priority, with: --ui.command, or the system default. The system default is opened with the following command:

OS Command
macOS open <url>
Windows start explorer "<url>"
Linux xdg-open <url>

To edit a file (for example config open), whatever the mode, the editor is, by priority: the command in the environment variable EDITOR, else notepad.exe on Windows, else the system default above (or --ui.command if defined).

When a command is defined, it is used to open URLs whatever the mode. The command is an array: the URL is appended as last argument. The executable is either a full path, or the name of an executable found in the PATH. For example, to use Firefox:

ascli config echo @secret: --ui=browser --ui.command=@json:'["firefox","--private-window"]'

Web login flow🔗

Authentication methods that need a user login in a browser (--auth=web, including for Shares) all work the same way: ascli listens on a local port, the browser is redirected to it after login.

Web login flow: ascli listens on a loopback port and receives the result of the login from the browser

  1. ascli generates a random state and starts a local HTTP server on the loopback address of redirect_uri (default: http://localhost:12345).
  2. ascli opens the login URL in the browser (see Graphical Interactions): the authorization endpoint with redirect_uri and state for web, or, for Shares, the SAML login page with RelayState. If no graphical environment is available, the URL is displayed, to be opened on any browser of the same host.
  3. The user logs in on the server, possibly redirected to an identity provider.
  4. The server redirects the browser to redirect_uri, with the OAuth code (web), or, for Shares, the tokens, and the state.
  5. The browser sends this request to ascli, which checks that state is the one it generated, and stops listening.
  6. ascli requests the token to the server: exchange of the code (web) or, for Shares, of the refresh token, and saves it in the cache. Next calls use the cached token, renewed if needed without a browser.

The code or tokens never go to another host than the local one: the host of redirect_uri shall be loopback.

Option ui.web_auth🔗

Option ui.web_auth (part of the composite option ui, see Graphical Interactions) selects how the result of a web login is received:

Value Local port Use when
port (default) Yes: ascli listens on the loopback address of redirect_uri The browser runs on the same system as ascli, or an SSH tunnel is possible.
paste No: redirect_uri is only used in the login URL The browser cannot reach the system that runs ascli, and no tunnel is possible.

With paste, after login the user gives the result to ascli through the terminal (nothing is displayed, as it contains a code or tokens):

  • the code displayed by Shares (a long text of letters, digits, - and _), or the code shown by ascli config web_login on the workstation,
  • or the full URL (or only the query) from the address bar of the browser, which shows a connection error on redirect_uri: this is expected.

In all cases the state is checked. Details are in the following sections.

Example:

ascli <PLUGIN> <COMMAND> --ui=terminal --ui.web_auth=paste

Web login when the browser is on another system🔗

The browser sends the result of the login to redirect_uri, on the loopback address of the system where the browser runs. If ascli runs on another system than the browser (for example, ascli is installed on a server, and the user is connected with SSH from a workstation), the redirection does not reach ascli.

To solve this, forward the port of redirect_uri from the workstation to the server with an SSH tunnel, and open the login URL in the browser of the workstation:

  1. On the workstation, open the SSH session with a tunnel (same port on both sides, default 12345):
ssh -L 127.0.0.1:12345:127.0.0.1:12345 <SERVER_USER>@<SERVER_HOST>
  1. In this session, run the command with --ui=terminal, so that the login URL is displayed instead of opening a browser on the server (this is the default if the server has no graphical environment):
ascli <PLUGIN> <COMMAND> --ui=terminal
  1. Copy the displayed URL in the browser of the workstation, and log in.
  2. The browser is redirected to http://localhost:12345: the tunnel sends the request to ascli on the server, and the flow continues as described above.

The tunnel only exists as long as the SSH session is open.

If no tunnel is possible (for example, a jump server between the workstation and the server), use --ui.web_auth=paste: ascli then opens no local port, and the result of the login is given through the terminal. There are two ways, depending on whether ascli is also installed on the workstation.

With --ui.web_auth=port (default), in an SSH session with --ui=terminal, a warning reminds this option.

With ascli on the workstation🔗

ascli on the workstation opens the login page, receives the redirection of the browser, and shows a code to paste in the terminal of the server. Nothing else is needed on the workstation: no configuration, no preset, no access to the server of the application. The exchange of the code for a token is done by ascli on the server: the secret of the client does not leave it.

  1. In the SSH session, run the command with --ui=terminal --ui.web_auth=paste:
ascli <PLUGIN> <COMMAND> --ui=terminal --ui.web_auth=paste
  1. ascli displays a message that proposes the two ways to log in, with the login URL for the first one, and a command to run on the workstation for the second one: it contains a code made of letters, digits, - and _. Then ascli waits:
USER ACTION: To continue, choose one of these two ways:
  1. With any browser (ascli is not needed on the workstation):
     - Open this URL in a browser and log in:
         https://...
     - The browser shows a connection error on http://localhost:12345: this is expected.
     - Copy the full URL from the address bar of the browser, and paste it in the prompt below.
  2. If ascli is installed on the workstation that has the browser:
     - Run there:
         ascli config web_login <CODE>
     - Log in in the page that opens.
     - Copy the code shown by the browser, and paste it in the prompt below.

The command is displayed only if ascli does not start the browser itself (--ui=terminal without --ui.command). If the browser is started by ascli (--ui=graphical, or --ui.command), it is on the same host as ascli, and the redirection may be immediate (user already logged in): only the steps to log in, copy the URL from the browser and paste it are displayed.

  1. On the workstation, run this command. The first time for a login host, ascli displays the host of the login page, and asks to trust it (yes): it opens a URL that comes from another system. Trusted hosts are saved; without a terminal, an unknown host is refused. ascli then opens the browser on the login page and listens on redirect_uri.
  2. Log in. The browser shows a page "Login Complete" with the code to paste, hidden by default:
  3. The button Copy puts the code in the clipboard.
  4. The eye button shows or hides the code, to select it manually.
  5. Paste the code in the terminal of the server (nothing is displayed). ascli on the server checks the state, and the flow continues as described above, from step 6.

The command on the workstation:

  • accepts only a login page in https, and a redirect_uri on a loopback address; the host shown for confirmation is the one of the URL, not a label given in the code;
  • is available only at security level low;
  • ends once the browser has received the page with the code; the port of redirect_uri shall be free on the workstation (the URL is registered for the login and cannot be changed);
  • gives to the server only the parameters it needs (the code and the state).

The codes are compressed and encrypted with a key that is not secret (derived from a constant and the current time, in slots of 15 minutes): they are hard to read and are valid for 15 to 30 minutes (the clocks of both computers must be correct), but they are not confidential: do not keep them, nor send them. The code shown by Shares also uses the random state of the login in its key. The codes are authenticated: a code that is incomplete, mistyped or modified is detected. Spaces and line breaks are ignored (letter case matters): a code displayed on several lines can be pasted.

Without ascli on the workstation🔗

If ascli is not installed on the workstation, the URL of the error page is used:

  1. In the SSH session, run the command with --ui=terminal --ui.web_auth=paste.
  2. Copy the displayed login URL in the browser of the workstation, and log in.
  3. The browser is redirected to redirect_uri (for example http://localhost:12345/?code=...&state=...) and shows a connection error: this is expected.
  4. Copy the full URL from the address bar of the browser, and paste it in the terminal (nothing is displayed, as it contains a code or tokens). The query alone (code=...&state=...) is also accepted. The state is mandatory: it is checked like in the standard flow, then the flow continues from step 6.

Both ways accept, at the prompt, the code, the URL or the query. If standard input is not a terminal, the line is read from it. The size of a pasted line may be limited by the terminal: the code of the reply contains only what is needed.

Notes:

  • For Shares with --auth=web, paste is simpler: Shares itself displays the code, so neither the tunnel, nor ascli config web_login, nor the URL of the error page is used (see Authentication of SAML users).
  • The port must be free on the workstation. To use another port, change it in the tunnel and in --redirect-uri, for example --redirect-uri=http://localhost:23456 and -L 127.0.0.1:23456:127.0.0.1:23456. For Aspera on Cloud, this redirect_uri shall also be declared in the API client (see API Client Registration).
  • The tunnel listens on 127.0.0.1: if the browser resolves localhost to ::1, use --redirect-uri=http://127.0.0.1:12345.
  • On Windows, ssh is the OpenSSH client provided with the system; with PuTTY, add a local port forwarding (Connection → SSH → Tunnels) from 127.0.0.1:12345 to 127.0.0.1:12345.
  • Another solution is X11 forwarding: run an X server on the workstation (for example VcXsrv or X410 on Windows) and connect with ssh -X. The browser then runs on the server and is displayed on the workstation, so no tunnel is needed. It is heavier (X server to install, slower display), and the browser is the one of the server.
  • Without interaction (automation), prefer a method that needs no browser, such as --auth=jwt for Aspera on Cloud.

Logging, Debugging🔗

The gem is equipped with traces, mainly for debugging and learning APIs. The composite option log groups all logging parameters (use --log=help to display this schema):

Field Type Description
format Enum[String] Log line formatter. Use @ruby: for a custom lambda.
Allowed values: default, caller, standard.
Default: default.
level Enum[String] Minimum log level displayed.
Allowed values: trace2, trace1, debug, info, warn, error, fatal, unknown.
Default: warn.
secrets Bool Show passwords and secrets in logs.
Default: false.
type Enum[String] Log output destination.
Allowed values: stderr, stdout, syslog.
Default: stderr.

Tip

Individual options log_level, log_format, logger, and log_secrets are also available as aliases for log.level, log.format, log.type, and log.secrets respectively.

log_level and log_secrets🔗

To increase debug level, use option log_level (for example, using command line --log-level=xx, env var ASCLI_LOG_LEVEL, or an Option Preset).

Note

When using the direct agent (ascp), additional transfer logs from ascp can be activated using ascp options and ascp_args, see direct.

By default, passwords and secrets are redacted from logs. Set option log_secrets to yes to include secrets in logs.

log_format🔗

Option log_format (alias for log.format) supports a few pre-defined formatters or a custom one using @ruby:. A custom formatter is a lambda that takes 4 arguments; see Ruby Formatter: severity, time, progname, msg. The default formatter is:

->(s, _d, _p, m){"#{s[0..2]}#{s[-1]} #{m}\n"}

Logging examples🔗

  • Display debugging log on stdout:
ascli config preset overview --log-level=debug --logger=stdout

Or equivalently using dot-path notation:

ascli config preset overview --log.level=debug --log.type=stdout
  • Log errors to syslog:
ascli config preset overview --log-level=error --logger=syslog

Or using the composite option in a preset:

log:
  level: error
  type: syslog

Note

When ascli is used interactively in a shell, the shell itself will usually log executed commands in the history file (history | grep ascli).

Learning Aspera Product APIs (REST)🔗

ascli uses mainly REST APIs to interact with Aspera applications.

To get traces of execution, with dump of API calls, use argument: --log-level=debug.

To display HTTP/S traffic set option log_level to trace2: --log-level=trace2. It will display the exact content of HTTP requests and responses.

HTTP socket parameters🔗

To ignore SSL certificate for any address/port, use option: insecure, that is, --insecure=yes. To ignore SSL certificate for a list of specific address/port, use option ignore_certificate, set to a URL or an Array of URLs for which the certificate is ignored (only the address and port are matched), for example, --ignore-certificate=https://127.0.0.1:9092 or --ignore-certificate=@list:,https://127.0.0.1:9092,https://127.0.0.1:9093

Note

Ignoring certificate also applies to ascp WSS.

Ignoring a certificate is not recommended. It is preferable to add the certificate to the trusted store. When certificate validation is skipped, a warning is displayed. To disable this warning, set the option warn_insecure to no.

HTTP connection parameters (not ascp WSS) can be adjusted using option http_options:

Field Type Description
download_partial_suffix String File suffix appended to the target filename while a download is in progress. (Rest)
Default: .http_partial.
keep_alive_timeout Number Maximum time in seconds to keep an idle persistent connection open. (Net::HTTP)
Default: 2.
open_timeout Number Maximum time in seconds to wait for a connection to be established. (Net::HTTP)
Default: 60.
read_timeout Number Maximum time in seconds to wait for data to be read from the server. (Net::HTTP)
Default: 60.
retry_max Integer Maximum number of retry attempts. (Rest)
Default: 1.
retry_on_error Bool Retry the request on any HTTP error response. (Rest)
Default: false.
retry_on_timeout Bool Retry the request on connection timeout. (Rest)
Default: true.
retry_on_unavailable Bool Retry the request on HTTP 503 Service Unavailable. (Rest)
Default: true.
retry_sleep Integer Delay in seconds between retry attempts. (Rest)
Default: 4.
ssl_options Array[String] List of OpenSSL option flags to set on the HTTP session (e.g. IGNORE_UNEXPECTED_EOF). (SSL)
token_cache_max_age Integer Maximum age in seconds of a cached OAuth token before it is discarded. (OAuth)
Default: 1800.
token_refresh_threshold Integer Remaining token lifetime in seconds below which the token is proactively refreshed. (OAuth)
Default: 120.
user_agent String Value sent in the User-Agent HTTP request header. (Rest)
write_timeout Number Maximum time in seconds to wait for data to be written to the server. (Net::HTTP)
Default: 60.

Time values are set in seconds and can be of type either Integer or Float. Any additional key is forwarded as a setter to the Net::HTTP session. For a full list, refer to the Ruby library: Net::HTTP.

Like any other option, those can be set either on command line, or in configuration file, either in a global preset or server-specific one.

Example:

ascli aoc admin package list --http-options=@json:'{"read_timeout":10.0}'

The option ssl_options corresponds to a list of options as listed in man SSL_CTX_set_options. The default initial value is the default of Ruby as specified in openssl/ssl.rb. Each option can be specified as a String with the same name as in the OpenSSL library by removing the prefix: SSL_OP_, or an Integer (for example, 0 resets to no option). If the name appears in the list, the option is set. If the name appears in the list prefixed with a hyphen (-), the option is cleared. For example to enable option SSL_OP_CIPHER_SERVER_PREFERENCE, add it to the list as CIPHER_SERVER_PREFERENCE. To disable option SSL_OP_SAFARI_ECDHE_ECDSA_BUG, add it as -SAFARI_ECDHE_ECDSA_BUG.

Example:

{"ssl_options":["CIPHER_SERVER_PREFERENCE","-SAFARI_ECDHE_ECDSA_BUG"]}

Proxy🔗

There are several types of network connections, each of them uses a different mechanism to define a (forward) proxy:

  • REST calls (APIs) and HTTP Gateway
  • ascp WSS and Legacy Aspera HTTP/S Fallback
  • ascp SSH and UDP (Aspera FASP)

See the following sections.

Proxy for REST and HTTP Gateway🔗

REST API calls and transfers based on HTTP Gateway both use Ruby's Net::HTTP class. See Ruby find proxy.

When Ruby HTTP is used, there are two possibilities to define an HTTP proxy to be used.

The http_proxy environment variable (lower case) can be set to the URL of the proxy (with optional credentials). Syntax is: (http|https)://[user:password@]host:port. For example, http://myproxy.org.net:3128.

Note

Ruby expects a URL and myproxy.org.net:3128 alone is not valid.

Credentials for proxy are optional but can also be specified:

export http_proxy=http://<USERNAME>:<PASSWORD>@proxy.example.com:3128

Option http_proxy does the same (set env var) but on command line:

ascli --http-proxy=http://<USERNAME>:<PASSWORD>@host:port ...

Alternatively, the fpac option (function for proxy auto config) can be set to a Proxy Auto Configuration (PAC) JavaScript value.

Note

Proxy credentials are not supported in PAC files.

To read the script from a URL (http:, https: and file:), use prefix: @uri:. A minimal script can be specified to define the use of a local proxy:

ascli --fpac='function FindProxyForURL(url, host){return "PROXY localhost:3128"}' ...

The result of a PAC file can be tested with command: config proxy_check. Example, using command line option:

ascli config proxy_check --fpac='function FindProxyForURL(url, host) {return "PROXY proxy.example.com:3128;DIRECT";}' http://example.com
proxy://proxy.example.com:3128
ascli config proxy_check --fpac=@file:./proxy.pac http://www.example.com
proxy://proxy.example.com:8080
ascli config proxy_check --fpac=@uri:http://server/proxy.pac http://www.example.com
proxy://proxy.example.com:8080

If the proxy found with the PAC requires credentials, then use option proxy_credentials with username and password provided as an Array:

ascli --proxy-credentials=@json:'["<USERNAME>","<PASSWORD>"]' ...
ascli --proxy-credentials=@list::<USERNAME>:<PASSWORD> ...

Proxy for Legacy Aspera HTTP/S Fallback🔗

Only supported with the direct agent: To specify a proxy for legacy HTTP fallback, use ascp native option -x and ascp_args: --transfer=@json:'{"ascp_args":["-x","<PROXY_URL>"]}'.

FASP proxy (forward) for transfers🔗

To specify a FASP proxy (forward), set the transfer-spec parameter: proxy (only supported with the direct agent).

For example, for an Aspera forward proxy not encrypted (HTTP) without authentication running on port 9091, the option would be:

--ts=@json:'{"proxy":"dnat://proxy.example.org:9091"}'

Or, alternatively, (prefer transfer spec like above, generally):

--transfer=@json:'{"ascp_args":["--proxy","dnat://proxy.example.org:9091"]}'

FASP configuration🔗

ascli uses one of the transfer agents to execute transfers.

By default, it uses the direct agent, which is a local ascp. Nevertheless, ascli does not come with ascp installed. This is the reason why it is advised to install the Aspera Transfer Daemon during installation (ascli transferd install).

By default, ascli uses the ascp binary found in well known locations, that is, typical Aspera product installation paths.

The config plugin allows finding and specifying the location of ascp. It provides the following commands for ascp sub-command:

  • show : Show the path of ascp used
  • info : Show information on ascp and its environment
  • install : Install the Transfer SDK (deprecated, same as transferd install)
  • spec : List transfer spec parameters supported by ascp
  • schema : Show the JSON schema of the transfer spec
  • errors : List known ascp errors and whether they are retry-able
  • products list : List Aspera transfer products available locally

Selection of ascp location for direct agent🔗

Option: sdk_folder is used to specify the location of ascp. By default, ascli uses ascp from the Transfer SDK installed in its configuration folder (see config ascp install).

To override and use an alternate ascp path, use option sdk_folder (--sdk-folder=).

For a permanent change, set a global default. For example, <INSTALL_DIR> could be ~/my_install_dir on Linux, or C:\Users\admin\.aspera\ascli\sdk on Windows.

Using a POSIX shell:

ascli config preset set GLOBAL sdk_folder <INSTALL_DIR>
INFO Updated: global_common_defaults: sdk_folder <- <INSTALL_DIR>
INFO Saving config file: /home/john/.aspera/ascli/config.yaml

If the path has spaces, read section: Shell and Command line parsing.

A special value product:<PRODUCT_NAME> can be used for option sdk_folder. It specifies to use ascp from the given product name. A special value for product name is FIRST, which means: use the first product found in the internal list. In that case, other files (SSH keys, aspera.conf, transferd) are still taken from the default SDK folder.

Locally installed Aspera products can be listed with:

ascli config ascp products list
╭───────────────────────────────────────┬──────────────────────────────────────╮
│ name                                  │ app_root                             │
╞═══════════════════════════════════════╪══════════════════════════════════════╡
│ IBM Aspera Transfer SDK               │ /Users/laurent/.aspera/sdk           │
│ IBM Aspera for Desktop                │ /Applications/IBM Aspera.app         │
│ IBM Aspera Connect                    │ /Applications/IBM Aspera Connect.app │
│ IBM Aspera High-Speed Transfer Server │ /Library/Aspera                      │
╰───────────────────────────────────────┴──────────────────────────────────────╯

To permanently use the ascp of a product:

ascli config preset set GLOBAL sdk_folder 'product:IBM Aspera Connect'
INFO Updated: default: config <- global_common_defaults
INFO Updated: global_common_defaults: sdk_folder <- product:IBM Aspera Connect
INFO Saving config file: /home/john/.aspera/ascli/config.yaml

To show the path of currently used ascp:

ascli config ascp show
/Users/laurent/.aspera/sdk/ascp
ascli config ascp info
╭─────────┬──────────────────────────────────────────────────╮
│ field   │ value                                            │
╞═════════╪══════════════════════════════════════════════════╡
│ ascp    │ /Users/john/.aspera/sdk/ascp                     │
...
╰─────────┴──────────────────────────────────────────────────╯

Transfer Clients: Agents🔗

Some actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (ascp). Transfers will be executed by a transfer client, here called Transfer Agent.

The following agents are supported and selected with option transfer:

transfer location Description of agent
direct local direct execution of ascp
transferd local Aspera Transfer Daemon
connect local Aspera Connect Client (to be deprecated in 2027, use desktop)
desktop local Aspera for Desktop
node remote Aspera Transfer Node
httpgw remote Aspera HTTP Gateway

The list of agents and their short identifier can be displayed with:

ascli config agents list

To display the configurable parameters of a specific agent (with required fields, default values, enumerated values):

ascli config agents parameters <agent name>

For example:

ascli config agents parameters direct
ascli config agents parameters node

To display the description of an agent:

ascli config agents show <agent name>

Note

All transfer operations are seen from the point of view of the agent. For example, an agent executing an upload, or package send operation will effectively push files to the related server from the system where the agent runs.

All above agents (including direct) receive transfer parameters as a transfer-spec. Parameters in transfer-spec can be modified with option ts.

Specific options for agents are provided with option transfer (as a Hash, optionally with an agent key).

Note

Parameters in transfer are specific for each agent type and are described in the agents respective sections.

Asynchronous Transfer Mode🔗

Adding asynchronous: true to the transfer option makes ascli return immediately after submitting the transfer, without waiting for completion.

The returned job_id can be used at any time to check progress:

# Start a download and return immediately
ascli server download /remote/big-file.bin --to-folder=/tmp \
  '--transfer=@json:{"agent":"desktop","asynchronous":true}'

# Poll status later
ascli config agents jobs status <job_id>

# List all pending/completed async transfers
ascli config agents jobs list

# Remove completed/failed entries
ascli config agents jobs cleanup

Note

config transfer (list, status, cleanup) is deprecated after 4.28.0: use config agents jobs. It still works, with a warning.

All transfer agents support asynchronous mode:

Agent Status source Persisted across restart
desktop JSON-RPC get_transfer (auto-discovered URL) Yes
node REST ops/transfers/{id} Yes
connect REST transfers/info/{id} (auto-discovered URL) Yes
transferd gRPC monitor_transfers Yes
direct In-process thread state (re-queryable while process lives, e.g. MCP mode) No: returns unknown after restart
httpgw In-process thread state (re-queryable while process lives, e.g. MCP mode) No: returns unknown after restart

For direct and httpgw, the transfer runs as a Ruby thread inside the ascli process. The job_id is persisted on disk but the live thread state is only available as long as the same process is running. If the process is restarted, config agents jobs status returns unknown for those jobs.

Parameters of the agent are persisted, except secrets (e.g. password of agent node): config agents jobs status takes them from the current transfer option, if it is for the same agent (e.g. same option on command line, or in a preset).

Note

asynchronous and MCP - When ascli is used as an MCP server, an AI assistant calling a transfer command may time out or cancel the request and retry, causing duplicate transfers. Using asynchronous: true eliminates this risk: the transfer is submitted immediately, the job_id is returned, and the AI can poll config agents jobs status <job_id> to track progress without any risk of duplication. This works with all agents, including direct and httpgw.

Agent: Direct🔗

The direct agent directly executes a local ascp in ascli. This is the default agent for ascli (option --transfer=direct). ascli will locally search installed Aspera products, including SDK, and use ascp from that component. See FASP.

Agent: Direct: transfer🔗

The transfer option accepts the following optional parameters to control multi-session, Web Socket Session, Resume policy and add any argument to ascp:

Field Type Description
ascp_args Array[String] List of native ascp command-line arguments.
Default: [].
client_ssh_key Enum[String] SSH key type to use for token-based transfers.
Allowed values: rsa, dsa_rsa, per_client.
Default: rsa.
file_list Bool If true, source paths are written to a temp file passed to ascp via --file-list or --file-pair-list.
If false, source paths are placed directly on the ascp command line.
Default: true.
monitor Bool Enable use of the ascp management port for transfer monitoring.
Default: true.
multi_incr_udp Bool Multi session - Increment UDP port for each session.
If true, each session uses a different UDP port starting at fasp_port (default: 33001).
If false, all sessions use the same fasp_port (or ascp default).
Default: true on Windows, false on other platforms.
quiet Bool Suppress the ascp progress bar display.
If false, the progress bar of option progress_bar is not displayed, unless that option is set.
Default: true.
resume Hash Configuration for automatic transfer resume on interruption.
resume.iter_max Integer Maximum number of retry attempts on error.
Default: 7.
resume.sleep_factor Integer Multiplier applied to sleep duration between consecutive retry attempts.
Default: 2.
resume.sleep_initial Integer Initial sleep duration (in seconds) before first retry.
Default: 2.
resume.sleep_max Integer Maximum sleep duration (in seconds) between retry attempts.
Default: 60.
spawn_delay_sec Number Multi session - Delay (in seconds) between starting each ascp session.
Default: 2.
spawn_timeout_sec Number Multi session - Maximum time (in seconds) to verify that ascp is running.
Default: 2.
trusted_certs Array[String] List of trusted certificate repositories.
wss Bool Enable Web Socket Session when available.
Default: true.

In case of transfer interruption, the agent will resume a transfer up to iter_max times. Sleep between iterations is given by the following formula where iter_index is the current iteration index, starting at 0:

min( sleep_max, sleep_initial * sleep_factor ^ iter_index )

By default, the progress bar of ascli is displayed (see Transfer progress bar): it aggregates all sessions of a multi-session transfer.

To display the native progress bar of ascp instead, set parameter quiet to false:

--transfer.quiet=false

In that case, the progress bar of ascli is not displayed, unless option progress_bar is set. For multi-session transfers, each ascp process displays its own progress bar.

To skip usage of management port (which disables the progress bar of ascli), set option monitor to false.

By default, Ruby's root CA store is used to validate any HTTPS endpoint used by ascp (for example, WSS). To use a custom certificate store, use the trusted_certs option of direct agent's option transfer. To use ascp's default, use option:

--transfer.trusted_certs=@none:

Some transfer errors are considered retry-able (for example, timeout) and others are not (for example, wrong password). The list of known protocol errors and retry level can be listed:

ascli config ascp errors

Error codes of sessions (errcode in ascp logs) are a different numbering, listed with their category and the corresponding error above:

ascli config ascp errors session

Examples:

ascli ... --transfer.wss=true --transfer.resume.iter_max=20
ascli ... --transfer.spawn_delay_sec=2.5 --transfer.multi_incr_udp=false

Parameter ascp_args can also be used to activate logging using option -L of ascp. For example, to activate debug level 2 for ascp (DD), and display those logs on the terminal (-):

--transfer=@json:'{"ascp_args":["-DDL-"]}'

This is useful to debug if a transfer fails.

To store ascp logs in file aspera-scp-transfer.log in a folder, use --transfer=@json:'{"ascp_args":["-L","/path/to/folder"]}'.

To analyze ascp log files (ascp 4.0 and later), give log files, or folders with log files, to config ascp logs:

ascli config ascp logs summary /path/to/folder
ascli config ascp logs sessions /path/to/folder/aspera-scp-transfer.log
Command Result
summary Statistics of sessions, errors by category, maximum throughput, findings.
health Health check for monitoring in Nagios format (failed sessions and low efficiency over last N minutes).
sessions One row per session, with diagnostics.
diagnostics One row per diagnostic of a session: id, severity (error, warning, info), message and recommendation (action to take, when the diagnostic has one).
multi_sessions Multi-session transfers: sessions grouped by transfer, with aggregate rate.
snapshots Aggregate rates per period.
snapshot_sessions Rates of each session per period.
peers Sessions and errors by peer address.
transfers Transferred files.
licenses Licenses of local and peer systems.
overdrives Sessions with loss due to overdriving of network or storage.
records Log lines parsed into records: samples, sessions or transfers (see below).
fields Description of the fields of records: samples, sessions or transfers.

Log files compressed with gzip (e.g. rotations aspera.log-20261101.gz, aspera-scp-transfer.1.log.gz) are read directly, in a folder or given as argument; gzip is detected from the first bytes of the file, not from its name. Argument - reads the logs from the standard input, plain or compressed with gzip:

ascli config ascp logs summary /var/log/aspera.log-20261101.gz
zcat /var/log/aspera.log-2026*.gz | ascli config ascp logs summary -

To analyze the logs of an HSTS read over SSH, instead of files of the local machine, use command logs of the plugin server.

A file must contain ascp log lines (FASP messages, or lines of process ascp in syslog): in a folder, other files are ignored; a file given as argument that is not an ascp log is refused.

When logs of both sides of a session are analyzed (client and server), each session includes the status, error and diagnostics of the other side (peer_side, peer_diagnostics).

For example, to list only errors:

ascli config ascp logs diagnostics /path/to/folder --select=@json:'{"severity":"error"}'

Besides the default columns, sessions rows have context columns to filter sessions (show them with --fields=ALL, or select with --fields=session,product_name,transport,authentication, or filter with --select/--query); a column is absent when the log does not give it:

Column Value
product_name Product that wrote the log, from the line <product> version <version> (e.g. Aspera High-Speed Transfer Server, Aspera Transfer SDK, Aspera Connect), without IBM.
transport ssh if Session Start has tcp_port, wss if it has wss_port.
authentication token if ascp logs that ASPERA_AUTH_TOKEN is set, else node if it logs the storage of a node access key. Never the token, the access key or the storage root.
proxy Address of the proxy (proxy_ip of Session Params), absent if there is none.
fips true or false (fips_enabled of Session Params).

Diagnostics of a session include: category of error (e.g. udp_setup, storage, authentication) with a hint, retry of a failed session, parameters set by the server (cipher, resume policy, virtual link), efficiency and loss of control messages, decline of the rate during the transfer (rate_decline: second half of the samples below 70% of the first half, not evaluated if the peer changes the rate, if another transfer session starts meanwhile, or if the session failed), and for a sender, which limit of the receiver (network, disk, virtual link) or reading from storage limits the rate. Other diagnostics: no packet received for some seconds, by the receiver (reception_interruption) or by the sender, which gets no feedback of the receiver (feedback_interruption); overflow of the retransmission timer (rex_timer_overflow: the retransmission RTT is very large); maximum rate of the license of ascp below the target rate (license_rate_limit); license that expires within license_expiry_days (default 30) of the start of the session, or has expired (license_expiry); and many small files transferred (small_files): at least small_files_min_count (default 100) files with a mean size below small_files_max_mean (default 1 MB); path MTU measured by ascp below small_mtu_max (default 1400, ascp measures 1492, 1006, 508 or 296), which suggests a VPN or a tunnel (small_mtu). Two configuration messages, which have no effect on the transfer, are not diagnostics but only appear in findings of summary: OpenSSL modules folder not found (ossl-modules not found), and a schedule of virtual link capacity that could not be registered (unable to submit change event).

The recommendation of a diagnostic is only in the list diagnostics (not in sessions, where diagnostics is a list of messages, and not in summary): it tells what to check or change, with the ascp options and aspera.conf parameters concerned (e.g. UDP port and firewalls, target and minimum rate, storage write, SSH). Diagnostics without action (e.g. information, or cancelled by the user) have no recommendation; the diagnostic of the category of a failed session gives its hint in message.

Option ascp_logs (dot-notation) sets period (snapshot period in minutes, default 1), zero_bytes (include sessions without transferred bytes), mtu (when not found in logs), resolve (host names of peers), timezone, health_minutes (window in minutes for health check command, default 60), remote_max_size (maximum total size in bytes of the log files read from a server with server logs, default 268435456), and thresholds (diagnostic thresholds, including rate_decline_ratio, rate_decline_min_samples, rate_decline_delay, rate_decline_end_fraction, license_expiry_days, small_files_min_count, small_files_max_mean and small_mtu_max), e.g. --ascp-logs.period=5 or --ascp-logs.thresholds.efficiency=95.

Times of log lines are read in the time zone of the analyst, and displayed in it. To read the log of another system without shifting it, set the time zone of the log with timezone: UTC or an offset +HH:MM, e.g. --ascp-logs.timezone=+02:00. Syslog lines (Oct 25 00:02:20) have no year: it is deduced from the modification date of the log file (the current date for the standard input), the log being before this date. So a log from December to January, analyzed in March, has its December lines in the previous year. Keep the modification date of the file when copying it (e.g. cp -p, scp -p).

Command records gives the log lines parsed into records, with the field names of the log lines, to filter them with --out.query (JMESPath):

Record Log lines
samples Receiver bl or Sender bl line, with the DS Qs line that follows it in field DS.
sessions Session Start, Session Params, Session Stop and Session Statistics lines of a session, and lines of the process about it (see below).
transfers Transfer Start and Transfer Stop lines of a file.

A group of a log line becomes an object: for example, rate t/m/c/n/vl/vr/r=... becomes rate.t, rate.m, rate.c, etc. Values are numbers when the log value is a number: rates in bps, durations in ms, s or µs, and percentages without %. Session Stop and Transfer Stop have the fields of Session Start and Transfer Start, with their final values. When several log files are read, field log_file gives the log file of each record. Field cookie of Session Params is never output.

A session record also has the fields of these lines of the process (ascp), when the log has them:

Field Log line
fasp_version, license_max_rate, license_expires Initializing FASP version <version>, license expires on <date>, max rate=<N> Mbps, ...: license_max_rate (bps) and license_expires are absent if the license is unlimited or has no expiry date. account no. and license no. are never output.
pMTU, start_brtt Measured pMTU: <N> Bytes, start_brtt: <N> ms.
datagram_size, block_size, path_mtu datagram size <N>B, block size <N>B, path MTU <N>B (no path_mtu in old versions).
peer_rate_params List of Peer requested rate params adopted lines, in the order of the log: time, target_rate, min_rate and transfer_policy of each change.

For example, samples where the calculated rate is lower than the target rate, and sessions with loss of control messages from the receiver:

ascli config ascp logs records samples /path/to/aspera-scp-transfer.log --out.query='[?rate.c < rate.t].{time: time, session: uuid, calculated: rate.c, network: rate.n, lost: bl2.l}'
ascli config ascp logs records sessions /path/to/folder --out.query='[?rcvr_ctl.lost_pct > `1`]'

Command fields gives the type and description (with unit) of each field of a kind of record (samples, sessions or transfers), with its path as used in --out.query (e.g. rate.c):

ascli config ascp logs fields samples
ascli config ascp logs fields sessions
ascli config ascp logs fields transfers

Note

When transfer agent direct is used, the list of files to transfer is provided to ascp using either --file-list or --file-pair-list and a temp file list, unless --file-list or --file-pair-list is already provided via transfer parameter ascp_args. To place source paths directly on the ascp command line instead of using a temp file, set file_list to false in transfer: --transfer.file_list=false.

In addition to standard methods described in section File List, it is also possible to specify the list of files using transfer parameter ascp_args:

--sources=@ts --transfer=@json:'{"ascp_args":["--file-list","myfilelist"]}'

Note

File lists are shown here; similar options exist for file pair lists.

Note

This method avoids the creation of a copy of the file list: if the standard options --sources=@lines:@file:... --src-type=... are used, then the file list is read and parsed, and a new file list is created in a temporary folder.

Note

This method has limitations: it applies only to the direct transfer agent (that is, local ascp) and not to Aspera on Cloud.

Agent: Direct: Management messages🔗

By default, ascli gets notifications from ascp on its management port. This can be disabled with parameter: monitor=false of transfer.

It is also possible to send messages to ascp using this management port. A typical use is to change the target rate of a running transfer.

The communication is done through a JSON file that shall be created in ascli's config folder as displayed with:

ascli config folder

The name of the file shall be: send_<PID>, where <PID> is the process ID of the running ascli.

If there is only one ascli running, one can get the PID like this:

ps -axo pid,command|grep ascli|grep -v grep|cut -f1 -d' '

Example to change the target rate:

echo '{"type":"RATE","rate":300000}' > ~/.aspera/ascli/send_67470

When ascli detects this file, it uses it during a transfer and then deletes it.

Note

The JSON's keys use snake case, that is, lower case with _ as word separator. The list of message type values can be found in aspera/ascp/management.rb: OPERATIONS. The list of parameters is PARAMETERS (native names are capitalized, keys in the JSON file are in snake case).

This agent supports a local configuration file, aspera.conf, where Virtual links can be configured.

On a server (HSTS), the following commands can be used to set a global virtual link:

asconfigurator -x 'set_trunk_data;id,1;trunk_name,in;trunk_capacity,100000;trunk_on,true'
asconfigurator -x 'set_trunk_data;id,2;trunk_name,out;trunk_capacity,100000;trunk_on,true'
asconfigurator -x 'set_node_data;transfer_in_bandwidth_aggregate_trunk_id,1'
asconfigurator -x 'set_node_data;transfer_out_bandwidth_aggregate_trunk_id,2'

This command is not available on clients: instead, edit the file aspera.conf (its location is shown by ascli config ascp info --fields=aspera_conf) and modify the sections default and trunks like this for a global 100 Mbps virtual link (capacity is in kbps):

<?xml version='1.0' encoding='UTF-8'?>
<CONF version="2">
    <default>
        <transfer>
            <in>
                <bandwidth>
                    <aggregate>
                        <trunk_id>1</trunk_id>
                    </aggregate>
                </bandwidth>
            </in>
            <out>
                <bandwidth>
                    <aggregate>
                        <trunk_id>2</trunk_id>
                    </aggregate>
                </bandwidth>
            </out>
        </transfer>
    </default>
    <trunks>
        <trunk>
            <id>1</id>
            <name>in</name>
            <on>true</on>
            <capacity>
                <schedule format="ranges">100000</schedule>
            </capacity>
        </trunk>
        <trunk>
            <id>2</id>
            <name>out</name>
            <capacity>
                <schedule format="ranges">100000</schedule>
            </capacity>
            <on>true</on>
        </trunk>
    </trunks>
</CONF>

It is also possible to set a schedule with different time and days, for example for the value of schedule:

start=08 end=19 days=mon,tue,wed,thu capacity=900000;1000000

Agent: Direct: aspera.conf: File name with special characters🔗

By default, ascp replaces file system disallowed characters in transferred file names with _. On Windows, it concerns: * : | < > " ' ?.

Replacing illegal characters in transferred file names is a built-in feature of Aspera transfers. This behavior can be customized with the aspera.conf configuration file. For example, to replace illegal character | with an underscore _:

  1. First, locate the configuration file with:
ascli config ascp info --fields=aspera_conf

Typically, it is located at $HOME/.aspera/sdk/aspera.conf

  1. Edit this file, and add the following line inside the XML section CONF.default.file_system:
        <replace_illegal_chars>_|</replace_illegal_chars>

The result should look like this:

<?xml version='1.0' encoding='UTF-8'?>
<CONF version="2">
<default>
    <file_system>
        <resume_suffix>.aspera-ckpt</resume_suffix>
        <partial_file_suffix>.partial</partial_file_suffix>
        <replace_illegal_chars>_|</replace_illegal_chars>
    </file_system>
</default>
</CONF>

According to the documentation, the parameter works as follows:

  • The first character in the value is the replacement character.
  • All other characters listed after it are the illegal ones to be replaced.

So in this example: | will be replaced with _.

If the mounted target file system is Windows but ascli runs on Linux, there are several additional illegal characters to handle. In that case, set the parameter as shown below. XML special characters such as <, >, " and ' must be escaped properly:

<replace_illegal_chars>_*:|&lt;&gt;&quot;&apos;?</replace_illegal_chars>

In this example:

  • _ is the replacement character.
  • The list of characters that will be replaced is: *:|<>"'?

So, for example:

  • report|final?.txt becomes report_final_.txt
  • data*backup"2025".csv becomes data_backup_2025_.csv

Agent: Connect Client🔗

Note: IBM Aspera Connect will be deprecated in 2027. Use IBM Aspera for Desktop instead: --transfer=desktop. ascli displays a warning when the connect agent is used.

By specifying option: --transfer=connect, ascli will start transfers using the locally installed IBM Aspera Connect Client. There are no specific parameters for option transfer.

Agent: Desktop Client🔗

By specifying option: --transfer=desktop, ascli will start transfers using the locally installed IBM Aspera Desktop Client. There are no specific parameters for option transfer.

Agent: Node API🔗

By specifying option: --transfer=node, ascli starts transfers in an Aspera Transfer Server using the Node API, either on a local or remote node. This is especially useful for direct node-to-node transfers.

Parameters provided in option transfer are:

Field Type Description
password String Password, secret, or bearer token.
Required.
root_id String Root file ID. Mandatory only when password is a bearer token.
url String URL of the Node API.
Required.
username String Node API user or access key.
Required.

Like any other option, transfer can get its value from a pre-configured Option Preset:

--transfer=@preset:<PRESET_NAME> --transfer.agent=node

It can also directly use the Extended Value syntax:

--transfer=@json:'{"agent":"node","url":"https://...","username":"<USERNAME>","password":"<PASSWORD>"}'

If transfer is not specified and a default node has been configured (name in node for section default) then this node is used by default.

If the password value begins with Bearer then the username is expected to be an access key and the parameter root_id is mandatory and specifies the file ID of the top folder to use on the node using this access key. It can be either the access key's root file ID, or any authorized file ID underneath it.

Agent: HTTP Gateway🔗

The Aspera HTTP Gateway is a service that allows sending and receiving files using HTTPS.

By specifying option: --transfer=httpgw, ascli will start transfers using the Aspera HTTP Gateway.

Parameters provided in option transfer are:

Field Type Description
api_version Enum[String] Force use of a specific API version.
Allowed values: v1, v2.
Default: v2.
synchronous Bool Wait for acknowledgment after each message.
Default: false.
upload_chunk_size Integer Size in bytes of each upload chunk.
Default: 64000.
url String URL of the HTTP Gateway.
Required.

Example:

ascli faspex5 packages receive 323 --transfer=httpgw --transfer.url=https://asperagw.example.com:9443/aspera/http-gwy

Note

The gateway only supports transfers authorized with a token.

If the application, for example, AoC or Faspex 5, is configured to use the HTTP Gateway, then ascli will automatically use the gateway URL if --transfer=httpgw is specified, so transfer URL becomes optional.

Agent: Transfer Daemon🔗

Another possibility is to use the Transfer Daemon (transferd). Set option transfer to transferd.

Options for transfer are:

Field Type Description
api_key String API key of an existing daemon (start is false) that has authentication enabled in its configuration. It is sent in the metadata api_key of each call.
ca_file String Path to the certificate authority file (PEM) that signed the certificate of an existing daemon (start is false) that has tls enabled in its configuration. A self-signed certificate is its own authority.
start Bool Start a new daemon instance.
Default: true.
stop Bool Stop the daemon when ascli exits.
Default: true.
url String Address and port listened by the daemon (e.g. :55002). If port is 0, the daemon listens on a random available port. If no address is given, 127.0.0.1 is used.
Default: :0.

Note

If port zero is specified in the URL, then the daemon will listen on a random available port. If no address is specified, then 127.0.0.1 is used.

For example, to use an external, already running transferd, use option:

--transfer=@json:'{"agent":"transferd","url":":55002","start":false,"stop":false}'

The gem grpc is not part of default dependencies, as it requires compilation of a native part. So, to use the Transfer Daemon you should install this gem:

gem install grpc

If the execution complains about incompatible libraries, then force recompilation of the native part:

gem uninstall grpc

gem install grpc --platform ruby

On Windows the compilation may fail for various reasons (3.1.1):

  • cannot find -lx64-ucrt-ruby310

→ copy the file [Ruby main dir]\lib\libx64-ucrt-ruby310.dll.a to (remove the dll extension) [Ruby main dir]\lib\libx64-ucrt-ruby310.a

  • conflicting types for 'gettimeofday'

→ edit the file [Ruby main dir]/include/ruby-[version]/ruby/win32.h and change the signature of gettimeofday to gettimeofday(struct timeval *, void *), that is, change struct timezone to void

Transfer Daemon as a system service🔗

The Transfer Daemon (transferd) can run permanently as a systemd service, for applications that use an existing daemon (see Agent: Transfer Daemon, start set to false). ascli installs the SDK, writes the configuration file of the daemon (/etc/transferd/transferd.json) and the systemd unit, then enables and starts the service.

Commands are in transferd service (plugin transferd):

Command Action
spec Show the schema of the configuration file of the daemon (transferd.json). Works on any system.
install Install the SDK if it is missing, then do configure.
configure Merge the given configuration into the existing file, validate it, write the file and the unit, reload systemd, enable and restart the service, and wait until the daemon listens.
status Show the version, the state of the service, the endpoint and whether it listens. The API key is never shown.
uninstall Stop and remove the service. With purge set to true in service, also remove the configuration file and the log folder.
use Switch to an installed version of the SDK and restart the service (see below).
versions List the installed versions of the SDK (see below).

Except spec and versions, commands take the optional last positional argument CONFIG: a Hash (Extended Value) with two optional sections:

  • service: how the service is installed.
  • config: content of transferd.json.
{
  "service": {
    "user": "asperadaemon"
  },
  "config": {
    "address": "127.0.0.1",
    "port": 55002,
    "log_directory": "/var/log/aspera-transferd",
    "workers": {"max_concurrent": 10},
    "authentication": {"enabled": true}
  }
}

Parameters of section service:

Field Type Description
after Array[String] Units that shall be started before the daemon (After= in the unit).
Default: ["network.target"].
environment Hash Environment variables set in the unit. None is set unless given here. TRANSFERD_ADDRESS, TRANSFERD_PORT, TRANSFERD_MGMT_PORT and TRANSFERD_MGMT_ADDRESS override the configuration file.
group String Group that runs the daemon. Defaults to the primary group of user.
name String Name of the service. The unit file is <name>.service.
Default: aspera-transferd.
purge Bool Used by uninstall. Also remove the configuration file and the log folder.
Default: false.
unit_dir String Folder where the unit file is created.
Default: /etc/systemd/system.
user String Account that runs the daemon. It must exist, it is not created.
Default: asperadaemon.
version String Version of the SDK to install, used by install when the SDK folder is named current (versions side by side). Defaults to the SDK version of this tool.

Section config follows the JSON schema of the daemon, displayed with:

ascli transferd service spec
Field Type Description
$schema String Optional URI or path of this schema, used by editors for completion and validation. Ignored by transferd.
address String IP address or host name on which the gRPC API listens. Use 0.0.0.0 to listen on all IPv4 interfaces (for example in a container); when not listening on a loopback address, consider enabling tls and authentication. Overridden by environment variable TRANSFERD_ADDRESS, and by option --address (-a), which takes precedence.
Default: 127.0.0.1.
Example: 127.0.0.1.
api_time_settings Hash Keepalive and connection lifetime settings of the gRPC API server, in seconds.
api_time_settings.max_conn_age Integer Maximum lifetime of a client connection, with a random variation of +/-10%; the client must then reconnect. 0 or omitted: no limit.
Default: 0.
Example: 3600.
api_time_settings.max_conn_age_grace Integer Additional time given to ongoing calls once max_conn_age is reached, before the connection is forcibly closed. 0 or omitted: no limit.
Default: 0.
Example: 10.
api_time_settings.max_conn_idle Integer Time after which a client connection with no ongoing call is closed. 0 or omitted: 86400 (24 hours).
Default: 86400.
Example: 120.
api_time_settings.time Integer Time without activity from a client after which the server pings it to check that the connection is still alive. 0 or omitted: 7200 (2 hours).
Default: 7200.
Example: 60.
api_time_settings.timeout Integer Time the server waits for activity after a keepalive ping before closing the connection. 0 or omitted: 20.
Default: 20.
Example: 3.
authentication Hash API key authentication of clients of the gRPC API.
authentication.api_key String Value that clients must send in gRPC request metadata entry api_key. It is stored in clear text: restrict read access to this file.
Example: change-me-to-a-long-random-value.
authentication.enabled Bool Requires every call to carry the API key; other calls are rejected with status UNAUTHENTICATED. api_key is then required.
Default: false.
fasp_management Hash Management port on which the ascp processes started by transferd report transfer progress.
fasp_management.address String IP address on which the management server listens. Change it only when ascp must reach transferd through an interface other than loopback. Overridden by environment variable TRANSFERD_MGMT_ADDRESS, and by option --mgmtaddress (-A), which takes precedence.
Default: 127.0.0.1.
Example: 127.0.0.1.
fasp_management.port Integer TCP port of the management server. 0 or omitted: a free port is selected at startup. Overridden by environment variable TRANSFERD_MGMT_PORT, and by option --mgmtport (-P), which takes precedence.
Default: 0.
Example: 23456.
fasp_management.promiscuous_mode_enabled Bool When true, transferd also reports transfers of ascp processes that it did not start. So that those processes find the management port, transferd writes it in file transferd..optport of directory ../var/run/aspera, relative to the FASP runtime bin directory. Those ascp processes must use the same FASP runtime as transferd (see fasp_runtime).
Default: false.
fasp_runtime Hash FASP runtime (ascp, async, license, aspera.conf) used for transfers. Omitted: the runtime embedded in the SDK is used. When this section is present, set use_embedded explicitly: if it is not true, user_defined is required.
fasp_runtime.extra_config Array[Hash] Extra command line options added to the executables started for transfers. Does not apply to async.
fasp_runtime.extra_config[].command String End of the path of the executable to which options are added, for example ascp (ascp.exe on Windows). Only the first matching entry is used.
Required.
Example: ascp.
fasp_runtime.extra_config[].options Array[String] Options added to the command line, one argument per item, before the transfer arguments.
Example: ["--save-before-overwrite"].
fasp_runtime.force_version String Accepted for backward compatibility. Has no effect.
fasp_runtime.log Hash Default logging of ascp, for transfers whose request configuration does not set localLog or logLevel. Does not apply to async.
fasp_runtime.log.dir String Directory where ascp writes its log (ascp option -L). Empty or omitted: default location of ascp.
Example: /var/log/aspera.
fasp_runtime.log.level Integer Level of detail of the ascp log: 0 for basic information (files transferred, transfer rate), 1 to 4 for increasingly detailed debug messages (ascp options -D to -DDDD).
Default: 0.
fasp_runtime.use_embedded Bool true: use the FASP runtime embedded in the SDK, and ignore user_defined. false: use the runtime located by user_defined. Omitted while the fasp_runtime section is present: false.
Example: true.
fasp_runtime.user_defined Hash Location of a separately installed FASP runtime. Required when use_embedded is not true, ignored otherwise. Both directories must exist at startup.
fasp_runtime.user_defined.bin String Directory containing the ascp and async executables. A relative path is resolved from the working directory of transferd.
Required.
Example: /opt/aspera/bin.
fasp_runtime.user_defined.etc String Directory containing the license file aspera-license and the configuration file aspera.conf. A relative path is resolved from the working directory of transferd.
Required.
Example: /opt/aspera/etc.
log_directory String Directory of the transferd log file transferd.log (rotated at 10 MiB, 10 files kept). A relative path is resolved from the directory of the transferd executable. The directory is created if missing. Empty or omitted: logs are written to standard output.
Example: /var/log/aspera.
log_format Enum[String] Format of the transferd log: text for human-readable lines, json for one JSON object per line. Any value other than text gives json.
Allowed values: text, json.
Default: text.
log_level Enum[String] Level of detail of the transferd log, from most (trace) to least (panic) verbose. Omitted or unrecognized value: info.
Allowed values: trace, debug, info, warning, error, fatal, panic.
Default: info.
port Integer TCP port on which the gRPC API listens. 0 or omitted: a free port is selected at startup; it is written in the log and in file transferd..optport of runtime_dir. The configuration file shipped with the SDK sets 55002. Overridden by environment variable TRANSFERD_PORT, and by option --port (-p), which takes precedence.
Default: 0.
Example: 55002.
reflection Bool Enables the gRPC server reflection service, so that tools such as grpcurl or grpcui can list and call the API without the transferd.proto file. Useful for debugging; leave disabled otherwise.
Default: false.
runtime_dir String Directory for runtime files. When port is 0, transferd writes the selected port in file transferd..optport there. Created with owner-only permissions if missing. Omitted: ../var/run/aspera relative to the directory of the transferd executable.
Example: /var/run/aspera.
tls Hash TLS for the gRPC API. When enabled, clients must connect using TLS.
tls.certificate String Path of the PEM server certificate file, which may include intermediate certificates. A relative path is resolved from the working directory of transferd.
Example: /etc/transferd/server.crt.
tls.enabled Bool Enables TLS. certificate and key are then required.
Default: false.
tls.key String Path of the PEM private key file matching certificate. A relative path is resolved from the working directory of transferd.
Example: /etc/transferd/server.key.
tls.min_tls_version Enum[String] Minimum TLS version accepted from clients. Omitted or other value: TLSv12.
Allowed values: TLSv12, TLSv13.
Default: TLSv12.
workers Hash Transfer execution and retention.
workers.max_concurrent Integer Maximum number of transfers running at the same time. Additional transfers wait in a queue until a running one ends. 0 or omitted: 100.
Default: 100.
Example: 10.
workers.transfer_cleaning_period Integer Interval, in seconds, at which transferd removes ended transfers older than transfer_max_age. 0 or omitted: 3600 (1 hour).
Default: 3600.
workers.transfer_max_age Integer Time, in seconds, during which an ended transfer remains known to transferd (status queries, events) before it can be removed. Removal happens at the next cleaning, so a transfer may remain up to transfer_max_age + transfer_cleaning_period. 0 or omitted: 7200 (2 hours).
Default: 7200.
Example: 86400.

The configuration is checked before anything is written:

  • Keys not given keep their value in the existing transferd.json (deep merge), and a key set to null is removed.
  • An unknown key, a value of a wrong type, or an inconsistent configuration (for example tls.enabled without certificate and key) is refused, with the name of the key and of the file if the faulty key is in the existing file. The daemon itself ignores unknown keys silently.
  • The section fasp_runtime is computed from the SDK folder, and can be overridden.
  • If authentication.enabled is true and no api_key is given, the key of the existing file is kept, else a random one is generated. The file is written in clear text with mode 0640, owner user:group. To get the key from the vault, write {"config":{"authentication":{"api_key":"@vault:transferd.api_key@"}}} in an Extended Value @json:@extend:'...'.

Requirements:

  • systemd on Linux only. On other systems, or without systemctl, commands fail with a message (only spec and --dry-run work everywhere).
  • Run as root, except with --dry-run=yes (option is a boolean), which shows the actions and the files without any change, and spec and versions.
  • The account (asperadaemon by default) is not created by ascli. If it does not exist, the command fails and shows the command to create it: useradd --system --no-create-home --shell /usr/sbin/nologin asperadaemon. This is the account used by other Aspera products on the same host, so that files have the same owner.
  • If the version of the SDK installed is different from the version of the schema, then a warning is displayed (it is not an error).

Example:

sudo ascli transferd service install --sdk-folder=/opt/aspera/transferd/current @json:'{"config":{"port":55002,"authentication":{"enabled":true}}}'
ascli transferd service configure --dry-run=yes @json:'{"config":{"log_level":"debug"}}'
sudo ascli transferd service status
sudo ascli transferd service uninstall @json:'{"service":{"purge":true}}'

Versions side by side and rollback🔗

If the SDK folder (option sdk_folder) is named current, e.g. --sdk-folder=/opt/aspera/transferd/current, then ascli uses the layout <prefix>/<version> with a link current that points to the version in use:

/opt/aspera/transferd/1.1.9
/opt/aspera/transferd/1.1.10
/opt/aspera/transferd/current -> 1.1.10
  • service install installs the version given in service.version (default: version tested with this release of ascli) in <prefix>/<version>, and sets the link current. If the installation fails, then the link is set back to the previous version.
  • The unit and transferd.json refer to current: switching the link and restarting the service is enough to change the version.
  • service versions lists the installed versions and the one in use.
  • service use <VERSION> switches the link and restarts the service. If the service does not start, then the link is set back to the previous version and the service is restarted.

Any other folder name is used as is, with no link.

Transfer Specification🔗

Some commands lead to file transfer (upload/download). All parameters necessary for this transfer are described in a transfer-spec (Transfer Specification), such as:

  • Server address
  • Transfer username
  • Credentials
  • File list
  • And so on

ascli builds the transfer-spec internally as a Hash. It is not necessary to provide additional parameters on the command line for a transfer.

It is possible to modify or add any of the supported transfer-spec parameters using the ts option. The ts option accepts a Hash Extended Value containing one or several transfer-spec parameters. Multiple ts options on command line are cumulative, and the Hash value is deeply merged. To remove a (deep) key from transfer spec, set the value to null.

Note

Default transfer spec values can be displayed with command:

ascli config ascp info --fields=ts --out.flat=no

It is possible to specify ascp options when the transfer option is set to direct using transfer option parameter: ascp_args. Example:

--transfer=@json:'{"ascp_args":["-l","100m"]}'

Or an equivalent (using dotted expression):

--transfer.ascp_args=@list:' -l 100m'

This is especially useful for ascp command line parameters not supported in the transfer spec.

The use of a transfer-spec instead of ascp command line arguments has the following advantages:

  • It is common to all Transfer Agents
  • It does not depend on command line limitations (special characters, and so on)

Transfer Parameters🔗

All standard transfer-spec parameters can be specified. A transfer-spec can also be saved/overridden in the configuration file.

References:

Parameters can be displayed with commands:

ascli config ascp spec

An optional parameter can be specified to display only the parameters supported by a specific transfer agent, or overview for one column per agent indicating which agents support each parameter:

ascli config ascp spec direct
ascli config ascp spec overview
ascli config ascp spec overview --fields=name,a,c,d,h,n,t

A JSON Schema can be generated with command:

ascli config ascp schema --format=jsonpp

An optional parameter can be specified to display the schema for a specific transfer agent:

ascli config ascp schema transferd --format=jsonpp

The description gives the supported agents, if not all, and the corresponding ascp argument or environment variable.

Transfer Specification Reference🔗

ID Name
A Direct
C Connect
D Desktop
H Httpgw
N Node
T Transferd
Field Type Description
apply_local_docroot Bool Apply local docroot to source paths.
(A, T)
(--apply-local-docroot)
authentication String Set to token for SSH bypass keys, else password asked if not provided.
(C)
cipher Enum[String] In transit encryption algorithms.
Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm.
Default: none.
(-c (conversion){enum})
cipher_allowed Enum[String] Returned by node API. Valid literals include aes-128 and none.
Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm.
(C)
content_protection Enum[String] Enable client-side content protection (CSEAR, encryption-at-rest).
For uploads, set to encrypt to transfer encrypted files and store them on the server with the extension .aspera-env. (aspera.conf parameter transfer_encryption_content_protection_extension). To download and decrypt encrypted files, set to decrypt
content_protection_password must be specified if this option is set.
Allowed values: encrypt, decrypt.
(--file-crypt={enum})
content_protection_password String Password for encryption/decryption of transferred assets.
(env:ASPERA_SCP_FILEPASS)
cookie String Metadata for transfer specified by application.
(env:ASPERA_SCP_COOKIE)
create_dir Bool Create target directory if it doesn't already exist.
If all the following conditions are met, then the destination_root specifies a filename instead of destination folder:
- create_dir is false
- A single source file is given on command line
- The target folder specified by destination_root does not exist
In all other cases, destination_root specifies a folder, and it is created if it does not already exist. I.e. if any of those conditions is met:
- create_dir is true
- Multiple source files are provided
- List of source files are provided in a file (list or pair), default for Node API and ascli.
- The target folder exists
(-d)
delete_before_transfer Bool Before transfer, delete files that exist at the destination but not at the source.
The source and destination arguments must be directories that have matching names.
Objects on the destination that have the same name but different type or size as objects on the source are not deleted.
(--delete-before-transfer)
delete_source Bool Remove transferred source files after transfer success. Equivalent to remove_after_transfer + remove_empty_directories + remove_empty_source_directory. Take precedence over those.
(A, N, T)
destination_root String Destination root directory.
Required.
destination_root_id String The file ID of the destination root directory.
Required when using Bearer token auth for the destination node.
(T)
dgram_size Integer UDP datagram size in bytes.
(-Z {integer})
direction Enum[String] Direction of transfer (on client side).
Required.
Allowed values: send, receive.
(--mode=(conversion){enum})
exclude_newer_than String Exclude files, but not directories, from the transfer if they are newer than the specified number of seconds added to the source computer's epoch.
e.g. -86400 for newer than a day back.
(--exclude-newer-than={string})
exclude_older_than String Exclude files, but not directories, from the transfer if they are older than the specified number of seconds added to the source computer's epoch.
e.g. -86400 for older than a day back.
(--exclude-older-than={string})
fail_bad_filepass Bool Fail on bad file decryption passphrase.
(A, T)
(--fail-bad-filepass)
fasp_port Integer Specifies fasp (UDP) port.
Default: 33001.
(-O {integer})
fasp_proxy Hash Proxy for communications between the remote server and the (local) client.
(T)
file_checksum Enum[String] Enable checksum reporting for transferred files by specifying the hash to use.
Allowed values: sha-512, sha-384, sha-256, sha1, md5, none.
Default: none.
(A, N)
(--file-checksum={enum})
http_fallback Bool, String When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed.
(-y (conversion){boolean\|string})
http_fallback_port Integer Specifies HTTP port when no cipher is used.
(-t {integer})
https_fallback_port Integer Specifies HTTPS port when cipher is used.
(-t {integer})
icos Hash Configuration parameters for IBM Cloud Object Storage (ICOS).
(T)
keepalive Bool The session is running in persistent session mode.
(A, T)
(--keepalive)
lock_min_rate Bool n/a
(C)
lock_min_rate_kbps Bool If true, lock the minimum transfer rate to the value set for min_rate_kbps.
If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.
(C, T)
lock_rate_policy Bool If true, lock the rate policy to the default value.
(C, T)
lock_target_rate Bool n/a
(C)
lock_target_rate_kbps Bool If true, lock the target transfer rate to the default value set for target_rate_kbps.
If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.
(C, T)
min_rate_cap_kbps Integer The highest minimum rate that an incoming transfer can request, in kilobits per second.
Client minimum rate requests that exceed the minimum rate cap are ignored.
The default value of unlimited applies no cap to the minimum rate. (Default: 0)
(C, T)
min_rate_kbps Integer Set the minimum transfer rate in kilobits per second.
Default: 0.
(-m {integer})
move_after_transfer String Move source files to the specified archive-dir directory after they are transferred correctly.
Available as of 3.8.0. Details in ascp manual.
Requires write permissions on the source.
If src_base is specified, files are moved to archive-dir/path-relative-to-srcbase.
archive-dir must be in the same file system (or cloud storage account) as the source files being transferred.
archive-dir is subject to the same docroot restrictions as source files.
move_after_transfer and remove_after_transfer are mutually exclusive options.
After files have been moved to the archive, the original source directory structure is left in place. Empty directories are not saved to archive-dir. To remove empty source directories after a successful move operation, also set remove_empty_directories to true. When using remove_empty_directories, empty directory removal examination starts at the srcbase and proceeds down any subdirectories. If no srcbase is used and a file path (as opposed to a directory path) is specified, then only the immediate parent directory is examined and removed if it is empty following the move of the source file.
(A, N, T)
(--move-after-transfer={string})
multi_session Integer Use multi-session transfer. max 128.
Each participant on one host needs an independent UDP (-O) port.
Large files are split between sessions only when transferring with resume_policy=none.
(special:-C {integer})
multi_session_threshold Integer Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.
(0=no file is split)
(A, N, T)
(--multi-session-threshold={integer})
obfuscate_file_names Bool HTTP Gateway obfuscates file names when set to true.
(H)
overwrite Enum[String] Overwrite files at the destination with source files of the same name based on the policy:
- always : Always overwrite the file.
- never : Never overwrite the file. If the destination contains partial files that are older or the same as the source files and resume is enabled, the partial files resume transfer. Partial files with checksums or sizes that differ from the source files are not overwritten.
- diff : (default) Overwrite the file if it is different from the source, depending on the compare method (default is size). If the destination is object storage, diff has the same effect as always. If resume is not enabled, partial files are overwritten if they are different from the source, otherwise they are skipped. If resume is enabled, only partial files with different sizes or checksums from the source are overwritten; otherwise, files resume.
- diff+older : Overwrite the file if it is older and different from the source, depending on the compare method (default is size). If resume is not enabled, partial files are overwritten if they are older and different from the source, otherwise they are skipped. If resume is enabled, only partial files that are different and older than the source are overwritten, otherwise they are resumed.
- older : Overwrite the file if its timestamp is older than the source timestamp.
If you set an overwrite policy of diff or diff+older, difference is determined by the value set for resume_policy:
none : The source and destination files are always considered different and the destination file is always overwritten
attributes : The source and destination files are compared based on file attributes
sparse_checksum : The source and destination files are compared based on sparse checksums, (currently file size)
full_checksum : The source and destination files are compared based on full checksums
Allowed values: never, always, diff, older, diff+older.
Default: diff.
(--overwrite={enum})
password String Password for local Windows user when transfer user associated with node API user is not the same as the one running asperanoded.
Allows impersonating the transfer user and have access to resources (e.g. network shares).
Windows only, node API only.
(N)
paths Array Array of path to the source (required) and a path to the destination (optional).
Required.
precalculate_job_size Bool Specifies whether to precalculate the job size.
(--precalculate-job-size / --precalculate-disable)
preserve_access_time Bool Preserve the source-file access timestamps at the destination.
Because source access times are updated by the transfer operation, the timestamp that is preserved is the one just before to the transfer.
(--preserve-access-time)
preserve_acls Enum[String] Preserve access control lists.
Allowed values: none, native, metafile.
Default: none.
(A, T)
(--preserve-acls={enum})
preserve_creation_time Bool Preserve source-file creation timestamps at the destination.
Only Windows systems retain information about creation time. If the destination is not a Windows computer, this option is ignored.
(--preserve-creation-time)
preserve_extended_attrs Enum[String] Preserve the extended attributes.
Allowed values: none, native, metafile.
Default: none.
(A, T)
(--preserve-xattrs={enum})
preserve_file_owner_gid Bool Preserve the group ID for a file owner.
(A, T)
(--preserve-file-owner-gid)
preserve_file_owner_uid Bool Preserve the user ID for a file owner.
(A, T)
(--preserve-file-owner-uid)
preserve_modification_time Bool Set the modification time, the last time a file or directory was modified (written), of a transferred file to the modification of the source file or directory.
Preserve source-file modification timestamps at the destination.
(--preserve-modification-time)
preserve_remote_acls Enum[String] Preserve remote access control lists.
Allowed values: none, native, metafile.
Default: none.
(A, T)
(--remote-preserve-acls={enum})
preserve_remote_extended_attrs Enum[String] Preserve remote extended attributes.
Allowed values: none, native, metafile.
Default: none.
(A, T)
(--remote-preserve-xattrs={enum})
preserve_source_access_time Bool Preserve the time logged for when the source file was accessed.
(A, T)
(--preserve-source-access-time)
preserve_times Bool Preserve file timestamps.
(A, N, T)
(-p)
proxy String Specify the address of the Aspera high-speed proxy server.
dnat(s)://[user[:password]@]server:port
Default ports for DNAT and DNATS protocols are 9091 and 9092.
Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.
(A)
(--proxy={string})
rate_policy Enum[String] The transfer rate policy to use when sharing bandwidth. Allowable values:
- high : When sharing bandwidth, transfer at twice the rate of a transfer using a fair policy.
- fair : (Default) Share bandwidth equally with other traffic.
- low : Use only unused bandwidth.
- fixed : Transfer at the target rate, regardless of the actual network capacity. Do not share bandwidth. Aspera recommends that you do not use this setting except under special circumstances, otherwise the destination storage can be damaged.
Allowed values: low, fair, high, fixed.
(--policy={enum})
rate_policy_allowed Enum[String] Specifies most aggressive rate policy that is allowed. Returned by node API.
Allowed values: low, fair, high, fixed.
(C)
remote_access_key String The access key ID of the access key that was used to construct the bearer token that is used to authenticate to the remote node.
(T)
remote_host String IP or fully qualified domain name (FQDN) of the remote server.
Required.
(--host={string})
remote_password String SSH session password.
(env:ASPERA_SCP_PASS)
remote_user String Remote user. Default value is xfer on node or connect.
Required.
(--user={string})
remove_after_transfer Bool Remove SRC files after transfer success.
(A, N, T)
(--remove-after-transfer)
remove_empty_directories Bool Specifies whether to remove empty directories.
(A, N, T)
(--remove-empty-directories)
remove_empty_source_dir Bool Remove empty source subdirectories and remove the source directory itself, if empty.
(T)
remove_empty_source_directory Bool Remove empty source subdirectories and remove the source directory itself, if empty.
(A)
(--remove-empty-source-directory)
remove_skipped Bool Must also have remove_after_transfer set to true. Defaults to false. If true, skipped files will be removed as well.
(A, C, N)
(--remove-skipped)
resume_policy Enum[String] If a transfer is interrupted or fails to finish, this policy directs the transfer to resume without retransferring the files. Allowable values:
- none : Always re-transfer the entire file.
- attrs : Compare file attributes and resume if they match, and re-transfer if they do not.
- sparse_csum : Compare file attributes and the sparse file checksums; resume if they match, and re-transfer if they do not.
- full_csum : Compare file attributes and the full file checksums; resume if they match, and re-transfer if they do not.
Note: transferd uses values: attributes, sparse_checksum, full_checksum.
Allowed values: none, attrs, sparse_csum, full_csum.
Default: faspmgr:none;other:sparse_csum.
(-k (conversion){enum})
retry_duration Integer, String Specifies how long to wait before retrying transfer (e.g. 5min).
(T)
save_before_overwrite Bool If a transfer would result in an existing file . being overwritten, move that file to .yyyy.mm.dd.hh.mm.ss.index. (where index is set to 1 at the beginning of each new second and incremented for each file saved in this manner during the same second) in the same directory before writing the new file.
File attributes are maintained in the renamed file.
(A, N, T)
(--save-before-overwrite)
skip_duplicate_check Bool Don't check for duplicate files at the destination.
(A, T)
(--skip-dir-traversal-dupes)
skip_special_files Bool All assets other than files, directories and symbolic links are considered special. A transfer will fail if the user attempts to transfer special assets. If true, ascp skips special assets and proceeds with the transfer of all other assets.
(A, T)
(--skip-special-files)
source_root String Path to be prepended to each source path.
This is either a conventional path or it can be a URI but only if there is no root defined.
(--source-prefix64=(conversion){string})
source_root_id String The file ID of the source root directory. Required when using Bearer token auth for the source node.
(N, T)
src_base String Specify the prefix to be stripped off from each source object.
The remaining portion of the source path is kept intact at the destination.
Special care must be taken when used with cloud storage.
(A, N, T)
(--src-base64=(conversion){string})
src_base64 String The folder name below which the directory structure is preserved (base64 encoded).
(A, T)
(--src-base64={string})
ssh_args Array Add arguments to the command-line arguments passed to the external ssh program (implies -SSH). The arguments are inserted before any key file(s) supplied to ascp and before the user/host arguments.
(A, T)
(special:--ssh-arg={array})
ssh_port Integer Specifies SSH (TCP) port.
Default: direct:22, other:33001.
(-P {integer})
ssh_private_key String Private key used for SSH authentication.
Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII...
Note the JSON encoding: \n for newlines.
(A, T)
(env:ASPERA_SCP_KEY)
ssh_private_key_passphrase String The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2.
(A, T)
(env:ASPERA_SCP_PASS)
ssh_private_key_path String Path to private key for SSH.
(A, T)
(-i {string})
sshfp String Check it against server SSH host key fingerprint.
(--check-sshfp={string})
symlink_policy Enum[String] Handle source side symbolic links. Refer to HSTS manual for more details.
- follow: Follow a symbolic link and transfer the contents of the linked file or directory when the link target is in the user's docroot.
- copy: Copy only the symbolic link. If a file with the same name exists at the destination, the symbolic link does not replace the file.
- skip: Skip-symbolic links. The link or the file to which it points are not transferred.
- copy+force: Copy only the symbolic link. If a file with the same name exists at the destination, the symbolic link replaces the file. If the file of the same name at the destination is a symbolic link to a directory, it is not replaced.
Allowed values: follow, copy, copy+force, skip.
Default: follow.
(--symbolic-links={enum})
tags Hash Metadata for transfer as JSON. Key aspera is reserved. Key aspera.xfer_retry specifies a "retry timeout" but is not actually used for node API initiated transfers.
(--tags64=(conversion){object})
tags64 String Base64 string for tags.
(A, T)
(--tags64={string})
target_rate String, Integer Pseudo-parameter: desired transfer rate in bps, with an optional unit suffix. Converted to target_rate_kbps (rounded down) before the transfer is started, overriding it if both are present. Also applies to async (sync). Accepted formats: integer (bps), or integer followed by k/K (x1000), m/M (x1000000), g/G (x1000000000). Examples: 100000000, 100000k, 100m, 1g.
target_rate_cap_kbps Integer Maximum target rate for incoming transfers, in kilobits per second. Returned by upload/download_setup node API.
(C, T)
target_rate_kbps Integer Specifies desired speed for the transfer.
(-l {integer})
title String Title of the transfer.
(C, N, T)
token String Authorization token. Type is Bearer, Basic or ATM.
(env:ASPERA_SCP_TOKEN)
use_ascp4 Bool Specify version of protocol. ascp4 will be deprecated.
Default: false.
(A, N, T)
use_system_ssh Bool Use an external ssh program instead of the built-in libssh2 implementation to establish the connection to the remote host. The desired ssh program must be in the environment's PATH.
To enable debugging of the ssh process, supply -DD and --ssh-arg=-vv arguments to ascp.
(A, T)
(-SSH)
wss_enabled Bool Server has Web Socket service enabled.
(special:--ws-connect)
wss_port Integer TCP port used for Web Socket service feed.
xfer_max_retries Integer Maximum number of retries, for node API initiated transfers. Shall not exceed aspera.conf parameter transfer_manager_max_retries (default 3).
(N)

Destination folder for transfers🔗

The destination folder is set by ascli by default to:

  • . for downloads
  • / for uploads

It is specified by the transfer-spec parameter destination_root. As such, it can be modified with option: --ts=@json:'{"destination_root":"<PATH>"}'. The option to_folder provides an equivalent and convenient way to change this parameter: --to-folder=<PATH>.

List of files for transfers🔗

When uploading, downloading or sending files, the user must specify the list of files to transfer.

By default, the list of files to transfer is provided on the command line.

The list of (source) files to transfer is specified by (Extended Value) option sources (default: @args). The list is either the list of source files, or a combined source/destination list (see below) depending on value of option src_type (default: list).

In ascli, all transfer parameters, including file list, are provided to the transfer agent in a transfer-spec so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer daemon...). So, eventually, the list of files to transfer is provided to the transfer agent using the transfer-spec field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional). The sources and src_type options provide convenient ways to populate the transfer spec with the source file list.

Possible values for option sources are:

  • @args : (default) the list of files (or file pair) is directly provided on the command line (after commands): unused arguments (not starting with -) are considered as source files.

Info

When using @: to build a command parameter and --sources=@args (default), the END marker is required between the @: block and the file list. See Positional Arguments with Dot-path.

Example:

ascli server upload ~/mysample.file secondfile

This is the same as (with default values):

ascli server upload --sources=@args --src-type=list ~/mysample.file secondfile

Tip

An Extended Value can be tested with the command config echo

Examples:

  • Using Extended Value

    Create the file list:

    echo ~/mysample.file > myfilelist.txt
    echo secondfile >> myfilelist.txt
    

    Use the file list: one path per line:

    --sources=@lines:@file:myfilelist.txt
    
  • Using JSON array

    --sources=@json:'["file1","file2"]'
    
  • Using STDIN, one path per line

    --sources=@lines:@stdin:
    
  • Using Ruby code (one path per line in file)

    --sources=@ruby:'File.read("myfilelist.txt").split("\n")'
    
  • @ts : the user provides the list of files directly in the paths field of transfer spec (option ts). Examples:

  • Using transfer spec

--sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'

The option src_type allows specifying if the list specified in option sources is a simple file list or if it is a file pair list.

Note

Option src_type is not used if option sources is set to @ts

Supported values for src_type are:

  • list : (default) the path of destination is the same as source and each entry is a source file path
  • pair : the first element is the first source, the second element is the first destination, and so on.

Example: Source file 200KB.1 is renamed sample1 on destination:

ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1

Source directory structure on destination🔗

This section is not specific to ascli: it describes ascp behavior.

The transfer destination is normally expected to designate a destination folder.

But there is one exception: The destination specifies the new item name when the following are met:

  • There is a single source item (file or folder)
  • Transfer spec create_dir is not set to true (ascp option -d not provided)
  • Destination is not an existing folder
  • The dirname of destination is an existing folder

For this reason, it is recommended to set create_dir to true for consistent behavior between single and multiple item transfers; this is the default in ascli.

If a simple source file list is provided (no destination in paths, that is, no file_pair_list provided), the destination folder is used as destination folder for each source file, and source file folder names are not preserved.

The inner structure of source items that are folders is preserved on destination.

A leading / on destination is ignored (relative to docroot) unless docroot is not set (relative to home).

In the following table source folder d3 contains 2 files: f1 and d4/f2.

Source files Destination Folders on Dest. create_dir Destination Files
f1 d/f - false Error: d does not exist.
f1 d/f d false d/f (renamed)
f1 d/f/. d false d/f (renamed)
f1 d/f d/f false d/f/f1
f1 f2 d d false d/f1 d/f2
d3 d - false d/f1 d/f2 (renamed)
f1 d - true d/f1
f1 f2 d - true d/f1 d/f2
d1/f1 d2/f2 d - true d/f1 d/f2
d3 d - true d/d3/f1 d/d3/d4/f2

If a file pair list is provided, then it is possible to rename or specify a different destination folder for each source (relative to the destination).

If transfer spec has a src_base, it has the side effect that the simple source file list is considered as a file pair list, and so the lower structure of source folders is preserved on destination.

Source files Destination src_base Destination Files
d1/d2/f2 d1/d3/f3 d d1 d/d2/f2 d/d3/f3

Advanced Example: Send files ./file1 and ./folder2/files2 to server (for example, /Upload) and keep the original file names and folders, that is, send file1 to /Upload/file1 and files2 to /Upload/folder2/files2.

  • If files are specified as ./file1 ./folder2/files2,

then destination will be: /Upload/file1 /Upload/files2

  • One possibility is to specify a file pair list: --src-type=pair file1 file1 folder2/files2 folder2/files2
  • Another possibility is to specify a source base (transfer spec parameter src_base): --ts.src_base=$PWD $PWD/file1 $PWD/folder2/files2

The . path cannot be used as a source base.

  • Similarly, create a temporary soft link (Linux): ln -s . tmp_base and use --ts.src_base=tmp_base tmp_base/file1 tmp_base/folder2/files2
  • One can also similarly use --sources=@ts and specify the list of files in the paths field of transfer spec with both source and destination for each file.

Multi-session transfer🔗

Multi session, that is, starting a transfer of a file set using multiple sessions (one ascp process per session) is supported on direct and node agents, not yet on connect.

  • --transfer=node
--ts=@json:'{"multi_session":10,"multi_session_threshold":1}'

Multi-session is directly supported by the node daemon.

  • --transfer=direct
--ts=@json:'{"multi_session":5,"multi_session_threshold":1,"resume_policy":"none"}'

Note

resume_policy set to attr may cause problems: none or sparse_csum shall be preferred.

ascli starts multiple ascp for Multi-session using direct agent.

When multi-session is used, one separate UDP port is used per session (refer to ascp manual page).

Content protection🔗

Content protection (Client-Side Encryption at Rest, CSEAR) ensures that files remain encrypted while stored on the server. With CSEAR, the client encrypts files during upload and decrypts files during download, using a passphrase known only to the users sharing the files.

  • Upload: Files are encrypted on the client side before being sent to the server.
  • Download: Files are decrypted on the client side as they are retrieved from the server.

At all times, files remain encrypted on the server; encryption and decryption occur exclusively on the client side.

Activating CSEAR consists of setting transfer spec parameters:

  • content_protection : activate encryption (encrypt for upload) or decryption (decrypt for download)
  • content_protection_password : the passphrase to be used.

Example: parameter to download a Faspex package and decrypt on the fly

--ts=@json:'{"content_protection":"decrypt","content_protection_password":"<PASSWORD>"}'

Note

Faspex 5 requires package parameter ear_enabled set to true for CSEAR. In that case the transfer spec parameter content_protection is automatically set. content_protection_password is then required in all cases.

Transfer Spec Examples🔗

  • Change target rate
--ts=@json:'{"target_rate_kbps":500000}'
  • Override the FASP SSH port to a specific TCP port:
--ts=@json:'{"ssh_port":33002}'
  • Force HTTP fallback mode:
--ts=@json:'{"http_fallback":"force"}'
  • Activate progress when not activated by default on server
--ts=@json:'{"precalculate_job_size":true}'

Transfer progress bar🔗

File transfer operations are monitored, and a progress bar is displayed on the standard error if option progress_bar (Bool) is set to yes (default if the standard error is a terminal). So, the progress bar is displayed even if the output of the command is redirected to a file.

The same progress bar is used for any type of transfer: using ascp, server to server, using HTTPS, and so on.

It shows the elapsed time, the percentage, the rate in megabits per second (Mbps, 1,000,000 bits per second, averaged over the last 5 seconds) and the estimated remaining time. As long as the total size is not known (for example, when the job size is not pre-calculated), the transferred size is shown instead of the percentage. For multi-session transfers, sessions are aggregated, and the number of running sessions is shown in brackets. Files already at destination (resumed transfer) count in the progress, but not in the rate. If the transfer fails, the progress bar stops at the reached progress, and shows failed. Log lines are displayed above the progress bar.

Agent direct can display the native progress bar of ascp instead (see direct).

Scheduler🔗

ascli does not include a built-in scheduler. Automated execution should therefore rely on operating system facilities.

Two common execution modes are supported:

  • Scheduled execution: run ascli commands periodically.

  • Daemon/service mode: run ascli continuously as a server.

Creating a wrapping script🔗

Before configuring scheduling or services, it is often useful to create a wrapper script that prepares the environment and optionally enforces execution limits.

This script may:

  • Configure environment variables (PATH, Ruby, and so on)

  • Enforce a maximum runtime

  • Centralize command options

Example (Linux)

#!/bin/bash
# Load the Ruby environment if required
# Forward arguments to ascli and ensure it does not exceed 30 min execution
exec timeout 30m ascli "${@}"

Save as:

/home/xfer/bin/ascli_tool

Then, make it executable.

Windows: Scheduler🔗

Windows provides the Task Scheduler.

Tasks can be configured using:

By default, Windows Task Scheduler prevents overlapping executions.

Linux: systemd Timer🔗

Most modern Linux distributions use systemd which provides scheduling via timer units. Define a name for the job, for example: ascli_job as <NAME> below.

  1. Create the service

    /etc/systemd/system/<NAME>.service

    [Unit]
    Description=Run ascli preview scan
    
    [Service]
    Type=oneshot
    ExecStart=/home/xfer/bin/ascli_tool preview scan --logger=syslog --out.level=error
    User=xfer
    Group=xfer
    
  2. Create the timer (same base name)

    /etc/systemd/system/<NAME>.timer

    [Unit]
    Description=Hourly execution of <NAME>
    
    [Timer]
    OnCalendar=hourly
    AccuracySec=1min
    Persistent=true
    
    [Install]
    WantedBy=timers.target
    
  3. Enable the timer

    sudo systemctl daemon-reload
    sudo systemctl enable --now <NAME>.timer
    

Check status:

systemctl status <NAME>.timer
systemctl list-timers | grep <NAME>

Run service once to test:

sudo systemctl start <NAME>.service

Linux: cron Scheduler🔗

Unix-like systems also provide cron.

Example of crontab for user xfer.

crontab<<EOF
0    * * * *  /home/xfer/bin/ascli_tool preview scan --logger=syslog --out.level=error
2-59 * * * *  /home/xfer/bin/ascli_tool preview trevents --logger=syslog --out.level=error
EOF

Linux also provides anacron for daily or hourly jobs that must run even if the system was previously offline.

Note

Logging options are specified directly in the cron entry so command output remains visible when running the command manually.

Running as system service (Daemon mode)🔗

Some commands run continuously (for example, listening on a network port). In this case, it is recommended to run ascli as a system service.

On Linux this is typically done using systemd.

For example, below <NAME> is ascli_svc.

A wrapping script, again, is convenient: /usr/local/bin/start_<NAME>.sh:

Service definition at /etc/systemd/system/<NAME>.service:

[Unit]
Description=My ascli daemon
After=network.target

[Service]
ExecStart=/usr/local/bin/start_<NAME>.sh
Restart=on-failure
RestartSec=15
User=xfer
# Optional, specify a working directory
# WorkingDirectory=/path/to/working/dir
# Optional, redirect output
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable --now <NAME>.service
systemctl status <NAME>.service
journalctl -u <NAME>.service

Preventing Concurrent Execution🔗

When running scheduled jobs, it may be necessary to ensure that only one instance of ascli runs at a time.

This avoids situations such as:

  • Multiple instances accumulating and overloading the system

  • Duplicate transfers

  • Duplicate preview outputs

Option: lock_port🔗

ascli provides a built-in locking mechanism via the lock_port option.

  • lock_port prevents concurrent executions of the same command.

  • If another instance is already running and holding the specified port, the new execution exits immediately.

Under the hood, this works by opening a local TCP server port. If the port is already in use, the process fails, effectively acting as a local lock. The lock is released automatically when the process exits.

Note

When using ascli in containers, lock_port only works within the same container. Each container has its own network namespace, so the lock does not apply across containers.

Example: Test ascli locking by running this command in two separate terminals within 30 seconds:

ascli config echo @ruby:'sleep 30' --lock-port=12345
  • The first instance will sleep for 30 seconds.

  • The second instance will exit immediately with:

WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).

OS-based concurrency protection🔗

Operating system schedulers also provide protection against overlapping runs.

  • Windows Task Scheduler prevents concurrent runs by default.
  • Linux cron can leverage flock to achieve the same effect:
/usr/bin/flock -w 0 /var/cron.lock ascli ...
  • systemd avoids overlaps when scheduling a Type=oneshot service.
[Unit]
Description=My ascli daemon
...

[Service]
ExecStart=/usr/local/bin/start_<NAME>.sh
Type=oneshot
RemainAfterExit=yes
...

Note

Systemd timers can schedule this service and ensure no overlapping runs occur.

"Provençal"🔗

ascp, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's open,read,write,close primitives), but also to do the same operations on other data storage such as S3, Hadoop and others. This mechanism is called PVCL (from Provençal, a restaurant located in Sophia Antipolis). Several PVCL adapters are available, one is embedded in ascp, the others are provided in shared libraries and must be activated.

The list of supported PVCL adapters can be retrieved with command:

ascli config ascp info --fields=@re:'^pvcl'
process v1
shares v1
noded v1
faux v1
file v1
stdio v1
stdio-tar v1

The output shows the adapters: process, shares, noded, faux, file, stdio, stdio-tar.

Those adapters can be used wherever a file path is used in ascp including configuration. They act as a pseudo drive.

The simplified format is:

<ADAPTER>:///<SUB_FILE_PATH>?<ARG1>=<VAL1>&...

One of the adapters, used in this manual, for testing, is faux. It is a pseudo file system allowing generation of file data without actual storage (on source or destination).

Other PVCL modules exist and are implemented in shared libraries. Those libraries are available only on HSTS. To activate a PVCL library, place the corresponding shared library in the same folder as ascp.

Example:

cp /opt/aspera/lib/pvcl/libpvcl_cloud.so $(ascli config ascp info --fields=root)

Then check available modules as shown previously (ascp info).

faux: for testing🔗

This adapter can be used to simulate a file or a directory. This is a feature of ascp, not ascli. The following is an extract of the man page of ascp.

To discard data at the destination, the destination argument is set to faux://.

To send uninitialized data in place of an actual source file, the source file is replaced with an argument of the form:

faux:///filename?filesize

Where:

  • filename is the name that will be assigned to the file on the destination
  • filesize is the number of bytes that will be sent (in decimal).

Note

Characters ? and & are shell special characters (wildcard and background), so faux file specification on command line should be protected (using quotes or \). If not, the shell may give error: no matches found or equivalent.

For all sizes, a suffix can be added (case-insensitive) to the size: k, m, g, t, p, e (values are powers of 2, for example, 1M is 220, that is, 1 mebibyte, not megabyte). The maximum allowed value is 8*260. Extremely large faux file sizes (petabyte range and above) will likely fail due to lack of destination storage unless destination is faux://.

To send uninitialized data in place of a source directory, the source argument is replaced with an argument of the form:

faux:///dirname?<ARG1>=<VAL1>&...

Where:

  • dirname is the folder name and can contain / to specify a subfolder.
  • Supported arguments are:
Name Type Description
count int Number of files
Mandatory
file string Basename for files
Default: file
size int Size of first file.
Default: 0
inc int Increment applied to determine next file size
Default: 0
seq enum Sequence in determining next file size
Values: random, sequential
Default: sequential
buf_init enum How source data is initialized
Option none is not allowed for downloads.
Values:none, zero, random
Default:zero

The sequence parameter is applied as follows:

  • If seq is random then each file size is:

  • size +/- (inc * rand())

  • Where rand is a random number between 0 and 1
  • File size must not be negative; inc is set to size if it is greater than size.
  • Similarly, overall file size must be less than 8*260. If size + inc is greater, inc will be reduced to limit size + inc to 7*260.

  • If seq is sequential then each file size is:

  • size + ((file_index - 1) * inc)

  • Where first file is index 1
  • So file1 is size bytes, file2 is size + inc bytes, file3 is size + inc * 2 bytes, and so on
  • As with random, inc will be adjusted if size + (count * inc) is not less than 8*260.

Filenames generated are of the form: <FILE>_<00000 ... count>_<FILESIZE>

Examples:

  • Upload 20 gibibytes of generated data to file myfile in directory /Upload
ascli server upload faux:///myfile\?20g --to-folder=/Upload
  • Upload a file /tmp/sample but do not save results to disk (no docroot on destination)
ascli server upload /tmp/sample --to-folder=faux://
  • Upload a faux directory mydir containing 1 million files, sequentially with sizes ranging from 0 to 2 Mebibyte - 2 bytes, with the base name of each file being testfile to /Upload
ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
  • Upload a faux directory mydir containing 1000 files, of size 1 byte, with the base name of each file being testfile to /Upload
ascli server upload "faux:///mydir?file=testfile&count=1000&size=1" --to-folder=/Upload

file: for growing files🔗

A growing file is a file that another program is still writing during the transfer, for example a live recording, a log file, or a stream capture. The built-in file PVCL adapter of ascp transfers such a file: ascp sends the data as the file grows, and completes the transfer when the file stops changing. This is a feature of ascp, not ascli.

For more information, see the HSTS ascp command reference.

Syntax of the file: URI🔗

To transfer a growing file, specify the source file as a file: URI that contains the grow parameter:

file:///<PATH>?grow=<WAIT_TIME>[&<PARAMETER>=<VALUE>...]

The path of the file follows the prefix file:///:

Path Example URI File
Relative file:///live.log?grow=60 live.log in the current folder
Absolute file:////var/log/app.log?grow=60 /var/log/app.log

An absolute path starts with /, so the URI of an absolute path contains four slashes.

The URI supports the following parameters:

Parameter Description
grow Wait time, in seconds: the transfer completes when the file does not change during this time.
Required.
If the value is not a number, for example grow=default, the wait time is 10 seconds.
wait_start Start of the wait time:
- mtime: the last modification time of the file.
- null_read: the time when ascp reaches the end of the file, that is, when a read of the file returns no data.
Default: mtime
confirm_stop If true, the program that writes the file confirms the end of the file. See Confirming the end of a growing file.
Ignored if wait_start is null_read.
Default: false
max_grow_size Maximum size of the file, in bytes.
Accepts the suffixes k, m, g, t, p, and e (powers of 1024).
Default: 263 - 1 bytes, or the maximum file size of the file system if lower.

Info

The growing file must be on a native file system, that is, a file system that the operating system accesses directly.

Tip

On the command line, enclose the URI in single quotes: the characters ? and & have a special meaning for the shell.

Completion of the transfer🔗

ascp sends the data as the program writes it. When ascp has sent all the data that the file contains, it starts to count the wait time. If the file grows during the wait time, ascp sends the new data and starts to count the wait time again. When the wait time ends without a change to the file, the transfer completes.

As a result:

  • The command completes after the wait time that follows the last write. For example, with grow=120, the command completes about 2 minutes after the program stops writing.
  • If the program pauses for longer than the wait time, the transfer completes before the end of the file, and the destination file is incomplete. Set a wait time that is longer than the longest pause of the program.

To complete the transfer without waiting, use confirm_stop.

Confirming the end of a growing file🔗

With confirm_stop=true, the program that writes the file signals the end of the file, so the transfer does not wait.

  1. Start the transfer with confirm_stop=true and a wait time that is longer than the longest pause of the program:
ascli server upload live.log --ts.source_root='file:///?grow=600&confirm_stop=true' --to-folder=/Upload
  1. When the program finishes writing the file, set the modification time of the file to a time earlier than the wait time. For example:
touch -t 200001010000 live.log

The transfer completes immediately.

Info

With confirm_stop=true, if the wait time ends before the program sets the modification time, ascp considers the file invalid and the transfer fails.

To report an error in the file, with or without confirm_stop, the program sets the modification time of the file to 0, that is, 1970-01-01 00:00:00 UTC. ascp then fails the transfer of the file. For example:

TZ=UTC touch -t 197001010000 live.log

Uploading a growing file🔗

ascp activates the growing file mode only if it finds the grow parameter in one of the following locations:

  • The first source on its command line.
  • The source prefix: ascp option --source-prefix, set by transfer spec parameter source_root.
  • The docroot of the transfer user, on the server.

ascp does not look for the grow parameter in a file list. By default, the direct agent provides the sources to ascp in a file list. As a result, if you specify a file: URI as a source with the default options, the transfer fails with the error Session initiation failed, fail to set pvcl.

Use one of the following methods. The examples use the direct agent.

  • Set the URI as source prefix (recommended)

Set transfer spec parameter source_root to a file: URI that contains the folder of the files and the parameters. Specify the sources as paths relative to this folder. ascp inserts each source path before the parameters: for example, file:////var/log?grow=120 and app.log give file:////var/log/app.log?grow=120. All the files of the transfer use the same parameters.

To upload live.log from the current folder:

ascli server upload live.log --ts.source_root='file:///?grow=120' --to-folder=/Upload

To upload /var/log/app.log:

ascli server upload app.log --ts.source_root='file:////var/log?grow=120' --to-folder=/Upload
  • Set the URI as source on the command line

Set transfer parameter file_list to false, so that ascli places the source on the ascp command line instead of in a file list. Use this method for a single file.

ascli server upload 'file:///live.log?grow=120' --to-folder=/Upload --transfer.file_list=false

Tip

To display the native progress bar of ascp instead of the progress bar of ascli, add --transfer.quiet=false.

Downloading a growing file🔗

For a download, the growing file is on the server. The transfer user on the server usually has a docroot, and the server does not resolve a file: URI from the client in this docroot: the transfer fails with the error No such file or directory. Instead, the server administrator sets the parameters in the docroot of the transfer user. For example, on the server:

asconfigurator -x 'set_user_data;user_name,xfer_live;absolute,file:////data/live?grow=120'

Then, the client downloads the file with a path relative to the docroot, as for any other file:

ascli server download live.log --to-folder=.

All the transfers of this user are then growing file transfers, so use a dedicated transfer user. This method also applies to transfers that use a token, for example with the Node API or Aspera on Cloud: for a download with a token, ascp does not accept a file: URI as source prefix.

If the transfer user has no docroot and the transfer does not use a token, you can also use the upload methods, with absolute paths on the server.

Usage🔗

ascli -h
WARN No config file found. New configuration file: /home/runner/.aspera/ascli/config.yaml
NAME
        ascli -- a command line tool for Aspera Applications (v4.29.0.pre)

SYNOPSIS
        ascli [GLOBAL_OPTIONS] <command> [OPTIONS] [ARGS]

DESCRIPTION
        Use Aspera application to perform operations on command line.
        Documentation and examples: https://rubygems.org/gems/aspera-cli
        execute: ascli conf doc
        or visit: https://ibm.biz/ascli-doc
        source repo: https://github.com/IBM/aspera-cli

ENVIRONMENT VARIABLES
        Any option can be set as an environment variable, refer to the manual

COMMANDS
        To list first level commands, execute: ascli
        Note that commands can be written shortened (provided it is unique).

OPTIONS
        Options begin with a '-' (minus), and value is provided on command line.
        Special values are supported beginning with special prefix @pfx:, where pfx is one of:
        val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, s, secret, stdin, yaml, zlib, extend, preset, vault
        Dates format is 'YYYY-MM-DD HH:MM:SS', or 'now' or '-<num>h'

ARGS
        Some commands require mandatory arguments, e.g. a path.

EXIT STATUS
        0    Success
        1    Other error
        2    Command line error: unknown command or option, invalid or missing argument
        3    Authentication or authorization failure: SSH, HTTP 401 or 403, OAuth grant
        4    Resource not found, or several resources match the identifier, HTTP 404
        5    Transfer failure
        6    Network or TLS failure: name resolution, connection, timeout, certificate
        7    Other error returned by a remote API
        130  Interrupted by user (Ctrl-C)

OPTIONS: global
    --interactive=yes|no           Use interactive input of missing params
    --ask-options=yes|no           Ask even optional options
    --out=HASH                     Output rendering options (dot-notation: format, level, file, fields, select, query, table[.pivot], flat, secrets, colors, utf8, img)
    --display=info|data|error      Output only some information (deprecated after 4.27.0: use --out.level)
    --format=ENUM                  Output format (also: --out.format)
    --output=VALUE                 Destination for results (deprecated after 4.27.0: use --out.file)
    --fields=LIST                  Comma separated list of: fields, or ALL, or DEF (also: --out.fields)
    --select=HASH                  Select only some items in lists: column, value (also: --out.select)
    --table-style=HASH             (Table) Display style (deprecated after 4.27.0: use --out.table)
    --flat-hash=yes|no             (Table) Display deep values as additional keys (deprecated after 4.27.0: use --out.flat)
    --multi-single=no|yes|single   (Table) Control how object list is displayed as single table, or multiple objects (deprecated after 4.27.0: use --out.table.pivot)
    --show-secrets=yes|no          Show secrets on command output (deprecated after 4.27.0: use --out.secrets)
    --image=HASH                   Options for displaying images and thumbnails in the terminal (deprecated after 4.27.0: use --out.img)
-h, --help                         Show this message
    --show-config                  Display parameters used for the provided action
-v, --version                      Display version
    --ui=HASH                      User interface (dot-notation: mode, web_auth, command)
    --invalid-characters=VALUE     Replacement character and invalid filename characters
    --log-level=ENUM               Log level
    --log-format=VALUE             Log formatter
    --logger=stderr|stdout|syslog  Logging method
    --log=HASH                     Logging options (dot-notation: level, type, format, secrets)
    --security=HASH                Security restrictions (dot-notation: level, allowed_paths)
    --lock-port=INT                Prevent dual execution of a command, e.g. in cron
    --once-only=yes|no             Process only new items (some commands)
    --dry-run=yes|no               Do not perform changes, only show what would be done
    --log-secrets=yes|no           Show passwords in logs
    --temp=HASH                    Temporary files (dot-notation: folder, clean, expiry)
    --clean-temp=yes|no            Cleanup temporary files on exit (deprecated after 4.27.5: use --temp.clean)
    --temp-folder=VALUE            Temporary folder (deprecated after 4.27.5: use --temp.folder)
    --pid-file=VALUE               Write process identifier to file, delete on exit
    --parser=none|json|ruby|yaml   Default parser for structured parameters and options
    --home=VALUE                   Home folder for tool
    --config-file=VALUE            Path to YAML file with preset configuration
    --secret=VALUE                 Secret for access keys
    --vault=HASH                   Secret vault configuration
    --vault-password=VALUE         Vault password
    --progress-bar=yes|no          Display progress bar
    --fpac=VALUE                   Proxy auto configuration script
    --proxy-credentials=LIST       HTTP proxy credentials for fpac: user, password
    --sql=VALUE                    SQL suffix appended to sqlite3 queries for admin subcommands (e.g. WHERE clause)
    --ascp-logs=HASH               Ascp: log analysis (dot-notation: period, zero_bytes, mtu, resolve, thresholds, timezone, health_minutes, remote_max_size)
-P, --preset=VALUE                 Load the named option preset from current config file
    --version-check-days=INT       Period in days to check new version (zero to disable)
    --plugin-folder=VALUE          Folder where to find additional plugins
    --sdk-url=VALUE                Ascp: URL to get Aspera Transfer Executables
    --locations-url=LIST           Ascp: URLs to get download locations of Aspera Transfer Daemon
    --sdk-folder=VALUE             Ascp: Path to folder with ascp (or product with "product:")
    --smtp=HASH                    SMTP email server configuration
    --notify-to=VALUE              Email: Recipient for notification of transfers
    --notify-template=VALUE        Email: ERB template for notification of transfers
    --cache-tokens=yes|no          Save and reuse OAuth tokens
    --expand-mounts=yes|no         Commands: list commands of sub-trees provided by another plugin
-N, --no-default                   Do not load default configuration for plugin
    --insecure=yes|no              HTTP/S: Do not validate any certificate
    --ignore-certificate=LIST      HTTP/S: Do not validate certificate for these URLs
    --warn-insecure=yes|no         HTTP/S: Issue a warning if certificate is ignored
    --cert-stores=LIST             HTTP/S: List of folder with trusted certificates
    --http-options=HASH            HTTP/S connection parameters for REST calls (not `ascp` WSS)
    --http-proxy=VALUE             HTTP/S: URL for proxy with optional credentials
    --query=HASH                   Additional filter for for some commands (list/delete)
    --bulk=yes|no                  Bulk operation (only some)
    --bfail=yes|no                 Bulk operation error handling
    --ts=HASH                      Override transfer spec values
    --to-folder=VALUE              Destination folder for transferred files
    --sources=VALUE                How list of transferred files is provided (@args,@ts,Array)
    --src-type=list|pair           Type of file list
    --transfer=HASH                Transfer agent type, or agent parameters with optional agent key
    --transfer-info=HASH           Parameters for transfer agent (deprecated after 4.26.2: use --transfer instead)

PLUGINS
    alee            Aspera License Entitlement Engine
    aoc             Aspera on Cloud
    ats             Aspera Transfer Service
    console         Console
    cos             IBM Cloud Object Storage
    faspex5         Faspex v5
    faspio          faspio Gateway
    httpgw          HTTP Gateway
    mcp             Model Context Protocol Server
    node            HSTS Node API
    orchestrator    Orchestrator
    preview         Preview
    server          HSTS Fasp/SSH
    shares          Shares
    transferd       Transfer Daemon

Bulk creation and deletion of resources🔗

Bulk creation and deletion of resources are possible using option bulk (yes,no(default)). In that case, the operation expects an Array of Hash instead of a simple Hash using the Extended Value syntax. This option is available only for some resources: if you need it, try and see if the entities you create or delete support this option.

Plugins🔗

ascli uses a plugin mechanism. The first level command (just after ascli on the command line) is the name of the concerned plugin which will execute the command. Each plugin usually represents commands sent to a specific application. Available plugins can be found using command:

ascli config plugins list
╭────────┬────────┬────────┬─────────────────────────────────────────────────╮
│ plugin │ detect │ wizard │ path                                            │
╞════════╪════════╪════════╪═════════════════════════════════════════════════╡
│ shares │ ✓      │ ✓      │ .../aspera-cli/lib/aspera/cli/plugins/shares.rb │
│ node   │ ✓      │ ✓      │ .../aspera-cli/lib/aspera/cli/plugins/node.rb   │
...
╰────────┴────────┴────────┴─────────────────────────────────────────────────╯

Most plugins will take the URL option: url to identify their location.

REST APIs of Aspera legacy applications (Aspera Node, Shares, Console, Orchestrator) use simple username/password authentication: HTTP Basic Authentication using options: username and password.

Aspera on Cloud and Faspex 5 rely on OAuth.

By default, plugins are looked up in folders specified by (multi-value) option plugin_folder:

ascli --show-config --fields=plugin_folder

You can create the skeleton of a new plugin like this:

ascli config plugins create foo .
Created ./foo.rb
ascli --plugin-folder=. foo

Plugins vs Transfer Agents🔗

Plugins typically represent a specific remote application on which ascli can operate, while transfer agents are the underlying components that handle the actual data transfer.

A given remote application can sometimes be both a plugin and a transfer agent. For example: node and httpgw are both plugins and transfer agents.

A plugin is invoked as the first positional argument in a command line. A Transfer Agent is used by setting the option transfer (for example, --transfer=node).

Command line parsing, Special Characters🔗

ascli is typically executed in a shell, either interactively or in a script. ascli receives its arguments on the command line. The way arguments are parsed and provided to ascli depends on the operating system and shell.

Shell parsing for Unix-like systems: Linux, macOS, AIX🔗

Linux command line parsing is well-defined: It is fully documented in the shell's documentation.

On Unix-like environments, this is typically a POSIX-like shell (bash, zsh, ksh, sh). A c-shell (csh, tcsh) or other shell can also be used. In this environment, the shell parses the command line, possibly replacing variables, and so on. See bash shell operation. The shell builds the list of arguments and then fork/exec Ruby with that list. Ruby receives the list of command line arguments from the shell and gives it to ascli. Special character handling (quotes, spaces, env vars, ...) is handled by the shell for any command executed.

Shell parsing for Windows🔗

On Windows, command line parsing first depends on the shell used (see next sections). MS Windows command line parsing differs from Unix-like systems because Windows does not provide a list of arguments to the executable (Ruby): it provides the whole command line as a single string, but the shell may interpret some special characters.

So command line parsing is not handled by the shell (cmd.exe), not handled by the operating system, but it is handled by the executable (Ruby). Typically, Windows executables use the Microsoft library for this parsing.

As far as ascli is concerned: the executable is Ruby. It has its own parsing algorithm, close to a Linux shell parsing.

Thankfully, ascli provides a command to check the value of an argument after parsing: config echo. One can also run ascli with option --log-level=debug to display the command line after parsing.

It is also possible to display arguments received by Ruby using this command:

C:> ruby -e 'puts ARGV' "Hello World" 1 2
Hello World
1
2

Note

Use pp instead of puts to display as Ruby Array.

Once the shell has dealt with the command line "special" characters for it, the shell calls Windows' CreateProcess with the whole command line as a single string. (Unlike Unix-like systems where the command line is split into arguments by the shell.)

It's up to the program to split arguments:

ascli is a Ruby program, so Ruby parses the command line (received with GetCommandLineW) into arguments and provides them to the Ruby code ($0 and ARGV). Ruby vaguely follows the Microsoft C/C++ parameter parsing rules. (See w32_cmdvector in Ruby source win32.c):

  • Space characters: split arguments (space, tab, newline)
  • Backslash: \ escapes a single special character
  • Globbing characters: *?[]{} for file globbing
  • Double quotes: "
  • Single quotes: '

Shell parsing for Windows: cmd.exe🔗

The following examples give the same result on Windows using cmd.exe:

  • Single quote protects the double quote
ascli config echo @json:'{"url":"https://..."}'
  • Triple double quotes are replaced with a single double quote
ascli config echo @json:{"""url""":"""https://..."""}
  • Double quote is escaped with backslash within double quotes
ascli config echo @json:"{\"url\":\"https://...\"}"

cmd.exe handles some special characters: ^"<>|%&. It handles I/O redirection (<>|), shell variables (%), and multiple commands (&). Eventually, all those special characters are removed from the command line unless escaped with ^ or ". " are kept and given to the program.

Shell parsing for Windows: PowerShell🔗

For PowerShell, the behavior depends on the version (5.1, 7.3+).

A difficulty is that PowerShell parses the command line for its own use and manages special characters, but then it passes the command line to the program (Ruby) as a single string, possibly without the special characters. If not using PowerShell features (for example, variables), one can use the "stop-parsing" token --%.

Details can be found here:

PowerShell 5🔗

  • Check your PowerShell version:
$psversiontable.psversion.Major
5

The following examples give the same result on Windows using PowerShell 5:

╭───────┬───────╮
│ field │ value │
╞═══════╪═══════╡
│ x     │ true  │
│ k     │ v     │
╰───────┴───────╯
  • Use PowerShell argument --% to place PowerShell in "stop-parsing" mode.
ascli config echo  --% @json:'{"k":"v","x":true}'
  • Triple double quotes are replaced with a single double quote in normal mode:
ascli config echo @json:'{"""k""":"""v""","""x""":true}'
  • To insert PowerShell variables in the JSON string, one can do:
$var="v"
ascli config echo  $('@json:{"""k""":"""' + $var + '""","""x""":true}')

PowerShell 7🔗

  • Check your PowerShell version:
$psversiontable.psversion.Major
7

The following examples give the same result on Windows using PowerShell 7:

  • Use PowerShell argument --% to place PowerShell in "stop-parsing" mode.
ascli config echo  --% @json:{"k":"v","x":true}
  • Single quote protects double quote in normal mode:
ascli config echo @json:'{"k":"v","x":true}'
  • To insert PowerShell variables in the JSON string, one can do:
$var="v"
ascli config echo  $('@json:{"k":"' + $var + '","x":true}')
  • Use PowerShell structure and then convert to JSON string:
$var="v"
ascli config echo "@json:$(@{ k = $var; x = $true } | ConvertTo-Json -Compress)"

Extended Value (JSON, Ruby, ...)🔗

Some values provided to ascli (options, Command Parameters) are expected to be Extended Values, that is, not a simple String, but a composite structure (Hash, Array).

For small structures with a few keys, the dot-path key.subkey=value notation (using @: to collect positional arguments, or --opt.key=value directly for options) is often the most convenient: it requires no quoting and reads naturally on the command line. For larger or more complex structures (deep nesting, arrays of hashes, values copied from API documentation), the @json: modifier is typically a better fit. @json: expects a JSON value; note that " is used to enclose a String in JSON, which may be difficult to specify in shells: enclose the whole argument in single quotes to avoid this.

Any option or Command Parameter expecting a Hash value accepts the special value help to display its schema. See Schema Discovery with help.

Using a shell variable, parsed by shell, in an Extended Value🔗

To be evaluated by the shell, the shell variable must not be in single quotes. Enclose the variable in double quotes so that it results in a single argument for ascli even if its value contains spaces: bash splits the result of an unquoted variable expansion into several words (zsh does not, by default).

Note

We use a simple shell variable in this example. It does not need to be exported as an environment variable.

MYVAR="Hello World"
ascli config echo @json:'{"title":"'"$MYVAR"'"}' --format=json
ascli config echo "@json:{\"title\":\"$MYVAR\"}" --format=json
{"title":"Hello World"}

Double quote in strings in command line🔗

Double quote is a shell special character. Like any shell special character, it can be protected either by preceding with a backslash or by enclosing in a single quote.

ascli config echo \"
ascli config echo '"'
"

Double quote in JSON is a little tricky because " is special both for the shell and JSON. Both shell and JSON syntax allow protecting ", but only the shell allows protection using single quote.

ascli config echo @json:'"\""' --format=text
ascli config echo @json:\"\\\"\" --format=text
ascli config echo @ruby:\'\"\' --format=text
"

Here a single quote or a backslash protects the double quote to avoid shell processing, and then an additional \ is added to protect the " for JSON. But as \ is also shell special, then it is protected by another \.

Shell and JSON or Ruby special characters in Extended Value🔗

Construction of values with special characters is done like this:

  • First select a syntax to represent the Extended Value, for example, JSON or Ruby

  • Write the expression using this syntax, for example, using JSON:

{"title":"Test \" ' & \\"}

Or, using Ruby:

{"title"=>"Test \" ' & \\"}
{'title'=>%q{Test " ' & \\}}

Both " and \ are special characters for JSON and Ruby and can be protected with \ (unless Ruby's extended single quote notation %q is used).

  • Then, since the value will be evaluated by shell, any shell special characters must be protected, either using preceding \ for each character to protect, or by enclosing in single quote:
ascli config echo @json:{\"title\":\"Test\ \\\"\ \'\ \&\ \\\\\"} --format=json
ascli config echo @json:'{"title":"Test \" '\'' & \\"}' --format=json
ascli config echo @ruby:"{'title'=>%q{Test \" ' & \\\\}}" --format=json
{"title":"Test \" ' & \\"}

Reading special characters interactively🔗

If ascli is used interactively (a user typing on terminal), values can be entered by the user:

ascli config echo @ruby:"{'title'=>gets.chomp}" --format=json

gets is Ruby's method of terminal input (terminated by \n), and chomp removes the trailing \n.

Command line arguments from a file🔗

If you need to provide a list of command line arguments from lines that are in a file, on Linux you can use the xargs command:

xargs -a lines.txt -d \\n ascli config echo

This is equivalent to execution of:

ascli config echo [line1] [line2] [line3] ...

If there are spaces in the lines, those are not taken as separator, as we provide option -d \\n to xargs.

Extended value using special characters read from environment variables or files🔗

Using a text editor or shell: create a file title.txt (and env var) that contains exactly the text required: Test " ' & \ :

export MYTITLE='Test " '\'' & \'
echo -n $MYTITLE > title.txt

Using those values will not require any escaping of characters since values do not go through shell or JSON parsing.

If the value is to be assigned directly to an option of ascli, then you can directly use the content of the file or env var using the @file: or @env: readers:

ascli config echo @file:title.txt --format=text
ascli config echo @env:MYTITLE --format=text
Test " ' & \

If the value to be used is in a more complex structure, then the @ruby: modifier can be used: it allows any Ruby code in expression, including reading from file or env var. In those cases, there is no character to protect because values are not parsed by the shell, or JSON or even Ruby.

ascli config echo @ruby:"{'title'=>File.read('title.txt')}" --format=json
ascli config echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
{"title":"Test \" ' & \\"}