Command Line Interface🔗
The command line tool is: ascli
The aspera-cli gem provides a command line interface (CLI) which interacts with Aspera Products (mostly using REST APIs):
- IBM Aspera High Speed Transfer Server (FASP and Node)
- IBM Aspera on Cloud (including ATS)
- IBM Aspera Faspex
- IBM Aspera Shares
- IBM Aspera Console
- IBM Aspera Orchestrator
- And more...
ascli provides the following features:
- Commands to Aspera server products (on-premise and SaaS)
- Any command line options (products URL, credentials or any option) can be provided on command line, in configuration file, in env var, in files, ...
- Commands, Options, and Option values shortcuts
- FASP Transfer Agents can be: local
ascp, or Connect Client, or any transfer node - Transfer parameters can be altered by modification of transfer-spec, this includes requiring multi-session
- Allows transfers from products to products, essentially at node level (using the node transfer agent)
- FASPStream creation (using Node API)
- Watchfolder creation (using Node API)
- Additional command plugins can be written by the user
- Download of Faspex and Aspera on Cloud "external" links
- Legacy SSH based FASP transfers and remote commands (
ascmd)
Basic usage is displayed by executing:
ascli -h
See Usage.
Note
Not all features of ascli are documented here: explore commands on the command line, using -h.
Command Line Arguments🔗
Command line arguments are the units of command line typically separated by spaces (the argv of C).
ascli handles the following types of command line arguments:
- Options: absolute position is not important, but relative order is, as a given option may be provided several times
- Plugins for example,
config, on first position - Resource Types for example,
users - Verbs for example,
create, to act on those resources or plugins. - Identifiers for example,
123, uniquely identifying a resource - Command Parameters arguments, for example, creation data, can be an Extended Value.
Command line arguments that are not options are referred to as Positional Arguments.
For example:
ascli plugin command verb --option-name=VAL1 VAL2
- Positional Arguments:
plugin command verb VAL2 - One Option:
option_nameand its value:VAL1
Enumeration values (positional arguments and option names/values) support prefix matching. See Enumerations for details.
For example ascli config pre ov --for=c is the same as ascli config preset overview --format=csv.
The value of Options and Command Parameters is evaluated with the Extended Value syntax.
Positional Arguments🔗
When options (beginning with --) are removed from the command line, the remaining arguments are Positional Arguments with a significant, pre-defined order.
Plugins, Resource Types and Verbs are Enumerations.
Identifiers and Command Parameters are open values, but a given type is expected, either a simple String or an Extended Value.
The general structure of positional arguments is:
ascli <PLUGIN> <RESOURCE> <COMMAND> [<ID>] [<PARAMETERS>]
If a sub-resource is used:
ascli <PLUGIN> <RESOURCE1> <SUB_RESOURCE> [<ID>1] <COMMAND2> [<ID2>] [<PARAMETERS>]
Example:
ascli config ascp info
ascliis the executable executed by the shellconfigis the name of the plugin to be usedascpis the name of the resource (singleton, so no identifier is provided)infois the verb to be performed (action on selected resource)
Resource types🔗
They identify a type of resource accessible in a given context.
A resource type can be a singleton, or have multiple instances (identified by their identifier).
A plugin is a kind of resource type singleton.
A resource type can also be a grouping of other resource types, for example admin or files, like a singleton.
Verbs🔗
Standard resource Verbs are: create, show, list, modify, delete.
Some entities also support additional verbs.
When such additional commands relate to a resource also accessible in another context, they are placed under the do command.
Such subcommands appear after the resource identifier, for example, in ascli aoc admin node do <NODE_ID> browse /, browse is a subcommand of node.
Typically, the create verb takes resource creation data as a parameter.
show, modify and delete take an identifier, unless manipulating a singleton.
list typically uses the query, select options.
list and show typically use the fields option.
Identifiers🔗
Identifiers uniquely identify a resource.
They are typically located immediately after a verb like show, modify or delete, itself placed after the resource type, for example: user show foobar.
Some resources can also be selected by a unique field other than the native identifier (typically: id), using the percent selector.
Percent selector🔗
The percent selector is a special syntax that lets you identify a resource by any unique field, instead of its default identifier (typically a numeric id).
Syntax: %<FIELD>:<VALUE>
- The leading
%marks the argument as a percent selector (not a plain identifier). <FIELD>is the name of the field to match on (e.g.name,email).<VALUE>is the expected value of that field.
The tool then looks up the resource whose <FIELD> equals <VALUE> and uses its native identifier internally.
For example, to show the user whose name field is john:
ascli aoc admin user show %name:john
Command Parameters🔗
Command Parameters are typically mandatory values for a command, such as resource creation data.
Note
A Command Parameter could also have been designed as an option.
However, since it is mandatory and such data typically do not need to be stored in a configuration file, using a Command Parameter is the better design choice.
The advantages over an option are: a shorter command line (no option name, only position), a clearly mandatory value, and a position that unambiguously indicates its role.
Extended Value syntax is supported, so it is possible to retrieve a value from the configuration file (using @preset:) or an environment variable (using @env:).
If a Command Parameter begins with -, then either use the @val: syntax (see Extended Value), or use the -- separator (see below).
A few Command Parameters are optional: they are always located at the end of the command line.
Enumerations🔗
Enumerations are arguments that must match a value from a predefined list in a given context.
The following are enumerations:
- Plugins, Resource Types, and Verbs (positional arguments)
- Option names (for example,
--log-level) - Some option values (for example,
debugfor--log-level)
Prefix matching: You can use a unique prefix instead of the full value, provided it uniquely identifies the value in that context.
Examples:
- Positional:
ascli config pre ov --for=cis the same asascli config preset overview --format=csv - Option name:
--log-l=debugis the same as--log-level=debug - Option value:
--format=cis the same as--format=csv
Note
While prefix matching works for option names, using full names is recommended for clarity.
Error handling: If an invalid or missing enumeration value is provided, an error message displays the list of valid values for that context.
Dot-path Notation🔗
ascli uses a unified dot-path notation in several places on the command line and in output.
Understanding this concept once makes it immediately usable everywhere it appears.
Dot-path notation is used in four distinct places in ascli:
| Surface | Syntax | in/out | read/write |
|---|---|---|---|
| Option names | --opt.key.sub=value |
Input | write |
| Positional arguments | @: key.sub=value ... [END] |
Input | write |
Extended value @preset: |
@preset:key.sub |
Input | read |
| Output field names | --out.fields=key.sub |
Output | read |
A dot-path is a String where segments are separated by . (dot), each segment designating a key in a nested structure:
- A string segment designates a key in a
Hash(associative array). - An integer segment designates an index in an
Array.
For example, the path a.b.0 means: key a, then key b, then the first element of an array.
When a value is assigned to the path (write with =), it is automatically converted to the simplest matching type: Boolean, Integer, Float, or String.
Values true and yes are converted to Boolean true, and values false and no to false.
For an option expecting a list of values that includes yes or no (e.g. --out.table.pivot=no), the Boolean is converted back to that value.
Note
A value of 1 will be automatically converted to an Integer.
When a specific type is required for the value, the Extended Value syntax modifiers @json: or @ruby: can be used.
For example: --opt.x=1 generates {"x": 1}.
To get a String: --opt.x=@json:\"1\" or --opt.x=@ruby:%q{1}.
Likewise, to get the String yes: --opt.x=@json:\"yes\".
Example: dot-path to JSON output
ascli config echo @: a.b=1 a.c=2 a.d.0=hello a.d.1=world --format=json
{"a":{"b":1,"c":2,"d":["hello","world"]}}
Example: JSON to dot-path output
ascli config echo @json:'{"a":{"b":1,"c":2,"d":["hello","world"]}}'
╭───────┬───────╮
│ field │ value │
╞═══════╪═══════╡
│ a.b │ 1 │
│ a.c │ 2 │
│ a.d │ hello │
│ │ world │
╰───────┴───────╯
Positional Arguments with Dot-path🔗
The @: syntax is a specialized argument type used for in-place definition of nested data structures.
It allows you to construct complex parameters (Hashes and Arrays) directly from the command line, serving as a readable alternative to providing a single, serialized JSON string.
Usage and Syntax:
The general syntax for this argument is:
@:[<MARKER>] <DOT_PATH>=<VALUE> [<DOT_PATH>=<VALUE>] ... [<MARKER>]
-
@:[<MARKER>]: A standalone argument that starts collecting dot-path assignments into a single data structure. The optional text after:sets the end marker. If it is empty, the marker isEND. -
<DOT_PATH>=<VALUE>: An assignment using the standard dot-path notation. Multiple assignments can be provided in sequence to build a complex object. The value is converted like any dot-path value, and can be an Extended Value, for examplepublic_key=@file:~/.ssh/id_rsa.pub. Options placed among the assignments are not collected: they are processed as usual, for example@: a=1 --format=json b=2. -
<MARKER>(ENDby default): An optional argument that ends the collection. -
Without the marker: All remaining positional arguments are collected, and each one must contain
=. -
With the marker: Only arguments between
@:and the marker are used for the structure. Arguments after the marker are passed to the command as normal positional arguments. -
Example with a custom marker:
@:STOP a=1 STOP.
Info
Use END whenever any positional argument must follow the object built with @: (for example, a file list, or any other subsequent positional parameter).
Without END, those arguments are collected as dot-path assignments instead of being passed to the command: an argument without = causes an error, and an argument containing = is silently added to the structure.
Example: Sending a package with a file list using @: for package information.
ascli aoc packages send @: name="<TITLE>" recipients.0=user@example.com END file1.dat file2.dat
Danger
In the above example, removing END causes an error, because file1.dat is read as a dot-path assignment and does not contain =.
Note
@: can also be used as an option value (for example, --query=@: a=b).
In that case, only positional arguments after the option's position in the original command line are consumed, so sub-commands before the option are not affected.
Use END as usual to stop collection when further positional arguments must follow: ascli aoc tier_restrictions --query=@: a=b END other_arg.
Options🔗
Command-line options, such as --log-level=debug, follow these conventions:
- Prefix:
All options begin with
--. - Naming:
Option names on command line use lowercase letters and hyphens (
-) as word separators. Option names in the configuration file use underscores (_) as word separators. Example:--log-level=debugislog_levelin config file. - Values:
An option's value is assigned using
=(for example,--log-level=debug). Alternatively, the value can be provided as a separate, space-separated token (for example,--log-level debug). Both forms are equivalent. - Prefix Usage: Options support prefix matching (see Enumerations), though full names are recommended.
- Optionality: Most options are optional; they either have a default value or do not require one.
- Order: Options can appear in any position on the command line, but their order may affect processing.
Exceptions and Special Cases:
- Short Forms
Some options have short forms.
For example, -Ptoto and -P toto are both equivalent to --preset=toto and --preset toto.
See the manual or -h for details.
- Flags
Certain options are flags and do not require a value (for example, -N).
- Option Terminator
The special option -- ends option parsing.
All subsequent arguments, including those starting with -, are treated as positional arguments.
- dot-path notation
- Cumulative Hashes
When an option of type Hash is set, the value is deep-merged with any existing value.
Setting to @json:{} starts from an empty hash (the existing value is discarded for that merge step).
Setting to @none: explicitly disables the option: it is set to nil and any preset or default value is prevented from restoring it.
This is useful, for example, to disable a vault or any Hash option that is configured in a default preset.
Example:
ascli config echo -- --sample
--sample
Note
Here, --sample is taken as an argument, and not as an option, due to --.
Options may have a (hard coded) default value.
Options can be placed anywhere on command line and are evaluated in order.
Usually the last value evaluated overrides previous values, but some options are cumulative, for example, ts.
Options are typically optional: to change the default behavior. But some are mandatory, so they can be placed in a configuration file, for example: connection information.
The value for any options can come from the following locations (in this order, last value evaluated overrides previous value):
- Configuration file
- Environment variable
- Command line
Environment variables starting with prefix ASCLI_ are taken as option values, for example, ASCLI_OPTION_NAME is for --option-name.
Option show_config dry runs the configuration, and then returns currently set values for options.
ascli --show-config outputs global options only, and ascli [plugin] --show-config outputs global and plugin default options.
In addition, option --show-config can be added at the end of any full command line, this displays the options that would be used for the command.
A command line argument is typically designed as option if:
- It is optional, or
- It is a mandatory parameter with a default value that would benefit from being set persistently (that is, in a configuration file or environment variable, for example, URL and credentials).
Interactive Input🔗
Some options and Command Parameters are mandatory and others are optional.
By default, ascli raises an error for missing mandatory options or Command Parameters.
The behavior can be controlled with:
--interactive=<yes|no>(default=no)yes: Missing mandatory parameters/arguments are asked to the user.no: Missing mandatory parameters/arguments raise an error message.--ask-options=<yes|no>(default=no)yes: Optional parameters/arguments are asked to the user.
Output🔗
Command execution will result in output (terminal, stdout/stderr). The information displayed depends on the action.
To redirect results to a file, use option --out.file.
Types of output data🔗
Depending on action, the output will contain:
| Result Type | Description |
|---|---|
single_object |
Displayed as a two-dimensional table: one line per field, first column is field name, and second is field value. Nested hashes are collapsed. |
object_list |
Displayed as a two-dimensional table: one line per item, one column per field. |
value_list |
A table with one column. |
empty |
nothing |
status |
A message. |
other_struct |
A complex structure that cannot be displayed as an array. |
With a structured format (json, jsonpp, yaml, ruby), the output is always valid in this format: a status is a string, an empty list is [], and no result is null.
Enhanced display of special values🔗
Special values are highlighted as follows in format=table:
| Value | Display |
|---|---|
nil |
<null> |
empty String |
<empty string> |
empty Array |
<empty list> |
empty Hash |
<empty dict> |
Example:
ascli config echo @json:'{"ni":null,"es":"","ea":[],"eh":{}}'
╭───────┬────────────────╮
│ field │ value │
╞═══════╪════════════════╡
│ ni │ <null> │
│ es │ <empty string> │
│ ea │ <empty list> │
│ eh │ <empty dict> │
╰───────┴────────────────╯
Filtering results🔗
The result of a command can be filtered on the server side, by the API, and then on the client side, by ascli.
Filtering on the server side reduces the data transferred, while filtering on the client side accepts any condition.
Options are evaluated in this order:
--query: parameters sent to the API.--out.secrets: secrets are hidden first, so that an expression cannot reveal them.--out.query: JMESPath expression or Ruby lambda, applied to the data as returned by the API.--out.select: selection of items (rows).--out.fields: selection of fields (columns).- Display in the format set by
--out.format.
| To | Use |
|---|---|
| Limit what the API returns | --query, with parameters max and pmax |
| Filter on any condition, reshape, count | --out.query |
| Select items by the value of fields | --out.select |
| Choose the fields displayed | --out.fields |
Option: --query: Parameters of the API🔗
The query option can generally be used to add URL parameters to commands that list resources.
It takes a Hash, corresponding to key/value pairs that appear in the query part of the request.
For example: --query=@json:'{"p1":"v1","p2":"v2"}' leads to query: ?p1=v1&p2=v2.
If the same parameter needs to be provided several times, then it is also possible to provide an Array.
For example: --query=@json:'{"p":["v1","v2"]}' leads to query: ?p=v1&p=v2.
If PHP-style arrays are expected in the API, add [] to the name of the parameter.
For example: --query=@json:'{"p[]":["v1","v2"]}' leads to query: ?p[]=v1&p[]=v2.
Commands that read a list by pages (for example, aoc admin user list) also accept parameters max and pmax.
They are processed by ascli, not sent to the API: they limit the number of items (max) and of pages (pmax) requested.
For example: --query.max=10 reads the first 10 items.
Tip
Use --query=help on any list command to display the available query parameters and their types, as documented in the API.
For example: ascli faspex5 admin accounts list --query=help
Option: --out.query: JMESPath expression🔗
Option --out.query applies a JMESPath expression to the result of a command, when it is an object or a list.
JMESPath is a standard query language for JSON, also used by option --query of the AWS and Azure CLIs.
The expression is applied to the data as returned by the API (secrets hidden), before options --out.select and --out.fields.
Its syntax is checked before the command is executed.
Results of other types (status, no result) are not changed.
Examples:
# Name and email of ATS administrators
ascli aoc admin user list --out.query='[?ats_admin].{name: name, email: email}'
# Number of users
ascli aoc admin user list --out.query='length(@)'
# Email of users, one per line, for use in a shell loop
ascli aoc admin user list --out.query='[].email' --format=text
Tip
Enclose the expression in single quotes for the shell.
In JMESPath, literal values are enclosed in backticks, for example: [?size > `1000`].
An expression with string literals in single quotes ('...') is enclosed in double quotes, for example: "[?name=='Ann']".
For processing that JMESPath does not provide (regular expressions, string manipulation, grouping), the value can be a Ruby lambda (@ruby:): it is called with the data, and returns the new data, converted to JSON data (for example, symbols and dates become strings).
It is refused at Security Level medium and high.
# Number of users per email domain
ascli aoc admin user list --out.query=@ruby:'->(users){users.group_by{|u|u["email"].split("@").last}.transform_values(&:count)}'
The result of the query is displayed according to its type: a list of objects as a table, a list of values with one value per line, a single value as is.
A list of lists (for example, [].[name, email]) is displayed only with a structured format (json, yaml, ...): to display a table, or CSV, use a multi-select hash instead, for example, [].{name: name, email: email}.
Note
Option --query gives filtering parameters to the API, while option --out.query filters and reshapes the result received.
Option --out.query is the equivalent of option --query of the AWS and Azure CLIs.
Option: --out.select: Selection of items🔗
A list of objects (object_list) can be filtered with option --out.select (or --select).
Its value is either a Hash or a Proc:
Hash: an item is selected if each key of theHashis a field of the item with the same value, and the same type:true(Boolean) is not"true"(String).Proc: a Ruby lambda called with each item (Hash), that returnstrueto select it, orfalseto remove it.
Example:
ascli aoc admin user list --out.fields=name,email,ats_admin --query=@json:'{"sort":"name"}' --out.select=@json:'{"ats_admin":true}'
╭────────────────┬─────────────────────┬───────────╮
│ name │ email │ ats_admin │
╞════════════════╪═════════════════════╪═══════════╡
│ John Curtis │ john@example.com │ true │
│ Laurent Martin │ laurent@example.com │ true │
╰────────────────┴─────────────────────┴───────────╯
In the above example, the same result is obtained with option:
--out.select=@ruby:'->(i){i["ats_admin"]}'
A key can be a dot-path, for example, directory_user.type, with all formats.
Option: --out.fields: Selection of fields🔗
Depending on the command, results may include by default all fields, or only some selected fields.
It is possible to define specific columns to be displayed, by setting option --out.fields (or --fields).
The --out.fields option is a list that can be either a comma-separated list or an Extended Value Array.
Individual elements of the list can be:
- property: add property to the current list
-property: remove property from the current listDEF: default list of fields (that is the default, when not set)ALL: all fields- A Ruby
RegEx: using@ruby:'/.../', or@re:...add those matching to the list
To display a property inside a complex structure (Hash, Array), use dot-path notation, only with --format=table or csv (see option --out.flat).
With other formats, use a JMESPath multi-select hash with option --out.query, for example: --out.query='[].{name: name, type: directory_user.type}'.
Examples:
a,b,c: the list of attributes specified as a comma-separated list (overrides the default)@list:,a,b,c:ArrayExtended Value: same as above@json:'["a","b","c"]':ArrayExtended Value: same as aboveb,DEF,-a: default property list, removeaand addbin first position@ruby:'/^server/': Display all fields whose name begins withserver
--out.select or --out.query🔗
Option --out.select is shorter for a test of equality, while option --out.query accepts any condition.
| Condition | --out.select |
--out.query |
|---|---|---|
Equal to a Boolean |
@json:'{"ats_admin":true}' |
'[?ats_admin]' |
Equal to a String |
@json:'{"name":"Ann"}' |
"[?name=='Ann']" |
| Field of a nested object | @json:'{"directory_user.type":"active_directory"}' |
"[?directory_user.type=='active_directory']" |
| Several conditions (and) | @json:'{"ats_admin":true,"name":"Ann"}' |
"[?ats_admin && name=='Ann']" |
| Negation, or | Ruby lambda (@ruby:) |
'[?!ats_admin]', or: see Recipes |
| Number comparison | Ruby lambda (@ruby:) |
'[?size > `1000000`]' |
| Substring | Ruby lambda (@ruby:) |
'[?contains(email, `@example`)]' |
| Choice of fields | With option --out.fields |
'[].{name: name, email: email}' |
| Count | Not possible | 'length(@)' |
Warning
- Values are compared with their type. With
--out.select,"true"(String) does not matchtrue(Boolean). In JMESPath,[?ats_admin == `true`]matches, but[?ats_admin == 'true']does not:'...'is a string literal. - After
--out.query, the default fields of the command no longer apply: all the fields of the result are displayed. Choose fields in the expression (multi-select hash), or with option--out.fields. - Secrets are hidden before
--out.queryis applied, so that the expression cannot reveal them.
Recipes🔗
# Users of Active Directory, with user name and email
ascli shares admin user all list --out.query="[?directory_user.type=='active_directory'].{username: username, email: email}"
# Users without directory (field missing or null)
ascli shares admin user all list --out.query='[?directory_user == null].username'
# Number of ATS administrators
ascli aoc admin user list --out.query='length([?ats_admin])'
# Users who are ATS administrators, or have no email
ascli aoc admin user list --out.query='[?ats_admin || !email].name'
# Emails of a domain, one per line
ascli aoc admin user list --out.query='[?ends_with(email, `@example.com`)].email' --format=text
# Largest file of a folder
ascli node browse / --out.query='max_by([?type==`file`], &size).path'
# Files larger than 1 MB
ascli node browse / --out.query='[?size > `1000000`].{path: path, size: size}'
# Files modified since a date
ascli node browse / --out.query='[?mtime >= `"2025-06-01"`].path'
Note
The JMESPath implementation of ascli (gem jmespath) also compares strings with <, <=, > and >=, which the JMESPath specification defines only for numbers.
Dates in ISO 8601 format, in the same time zone, are compared in chronological order.
The result in JSON (--format=json) can also be processed by other tools, such as jq, whose language differs from JMESPath: for example, jq 'map(select(.directory_user.type == "active_directory"))'.
Other filters🔗
Some commands provide their own filters:
- Percent selector
%<field>:<value>: selects a single resource by the value of one of its fields. findof pluginnode(gen4) andaoc files find: recursive search of files on the client side, with a glob or a Ruby lambda (see Operationfind).faspex5 packages list: a Ruby lambda as last argument (see Faspex 5: List packages).console transfer: query parameterfilter(see Transfer filter).preview: options--filterand--skip-folders(see Candidate detection).
Option: --out.format🔗
The style of output can be set using option --out.format (or --format):
format |
Output formatting |
|---|---|
table |
Text table (default) |
text |
Value as String |
ruby |
Ruby code |
json |
JSON code |
jsonpp |
JSON pretty printed |
yaml |
YAML |
csv |
Comma Separated Values |
image |
URL or data for a picture/video |
nagios |
Suitable for Nagios |
By default, results of type single_object and object_list are displayed using format table.
Option: --out.table🔗
The way format: table and csv are generated can be customized with option --out.table which expects a Hash.
For format=table, options are the ones described in gem terminal-table.
For example, to display a table with thick Unicode borders:
ascli config preset overview --out.table=@ruby:'{border: :unicode_thick_edge}'
Note
Other border styles exist, not limited to: :unicode, :unicode_round.
By default, if the terminal supports Unicode (see --out.utf8), then border=unicode_round is used.
A special parameter is defined: str_lst_sep (String), default is \n.
It defines how lists of strings are displayed.
Alternatively, set to ,.
When the output is a terminal, the table is fitted to the terminal width: columns with text (e.g. description) are wrapped, the widest first.
If not enough, other columns (names, identifiers) are wrapped too, preferably after a punctuation (-, _, ., /).
If the table cannot fit, columns are narrowed to their minimum width (20 characters for text, 10 for others, or heading).
Parameter width (Integer) sets the maximum table width (e.g. --out.table.width=100), also when the output is not a terminal.
For format=csv, options are described in gem csv.
For example, to display a CSV with headers and quotes:
ascli config echo @json:'[{"name":"foo","id":1},{"name":"bar","id":8}]' --format=csv --out.table=@json:'{"headers":true,"force_quotes":true}'
Option: --out.flat: Single level Hash🔗
This option controls how object fields are displayed for complex objects.
Effective only when format is table or csv to display single_object or object_list.
If value is no, then object's field names are only the first level keys of the Hash result and values that are Hash are displayed as such in Ruby syntax.
If value is yes (default), then objects are "flattened" using dot-path notation with a variation:
- Final arrays are displayed as a list of values, one per line (see
str_lst_sepin--out.tableabove) ArrayofHashwith onlynamekeys are displayed as comma separated list of valuesArrayofHashwith onlynameandvaluekeys are displayed like aHashwith value ofnameas key.
Example: The result of the command is a single object:
ascli config echo @json:'{"A":"a","B":[{"name":"B1","value":"b1"},{"name":"B2","value":"b2"}],"C":[{"C1":"c1"},{"C2":"c2"}],"D":{"D1":"d1","D2":"d2"}}'
╭────────┬───────╮
│ field │ value │
╞════════╪═══════╡
│ A │ a │
│ B.B1 │ b1 │
│ B.B2 │ b2 │
│ C.0.C1 │ c1 │
│ C.1.C2 │ c2 │
│ D.D1 │ d1 │
│ D.D2 │ d2 │
╰────────┴───────╯
For the same command, adding option --out.flat=no:
╭───────┬────────────────────────────────────────────────────────────────────────╮
│ field │ value │
╞═══════╪════════════════════════════════════════════════════════════════════════╡
│ A │ a │
│ B │ [{"name" => "B1", "value" => "b1"}, {"name" => "B2", "value" => "b2"}] │
│ C │ [{"C1" => "c1"}, {"C2" => "c2"}] │
│ D │ {"D1" => "d1", "D2" => "d2"} │
╰───────┴────────────────────────────────────────────────────────────────────────╯
Option: --out.table.pivot🔗
This option controls how result fields are displayed as columns or lines, when option format is set to table.
Values are false (default), true or single.
On command line, no and yes can be used as well (converted to Boolean, see dot-path), but a structured value (e.g. --out=@json:'{"table":{"pivot":true}}') expects a Boolean.
There are two types of results that are affected by this option:
| Result | Description |
|---|---|
single_object |
A single item with multiple fields. |
object_list |
A list of items, each with multiple fields. |
An item (object) is displayed in one of those 2 ways:
| Display | rows | columns |
|---|---|---|
| Simple | Fields | field and value |
| Transposed | Items | Fields |
The display of result is as follows:
| Result | false |
true |
single |
|---|---|---|---|
single_object |
Simple | Simple | Simple |
object_list |
Transposed | Simple (Multiple objects) |
Simple if 1 object. transposed if 2+ objects. |
This parameter can be set as a global default with:
ascli config preset set GLOBAL out.table.pivot single
Examples:
Simulate a result by executing this command:
ascli config echo @json:'<JSON_VALUE>' --out.table.pivot=<no|yes|single>
Example 1: A list of one object
[{"user":{"id":1,"name":"toto"},"project":"blash"}]
Display with no (Transposed):
╭─────────┬───────────┬─────────╮
│ user.id │ user.name │ project │
╞═════════╪═══════════╪═════════╡
│ 1 │ toto │ blash │
╰─────────┴───────────┴─────────╯
Display with yes and single (Simple):
╭───────────┬───────╮
│ field │ value │
╞═══════════╪═══════╡
│ user.id │ 1 │
│ user.name │ toto │
│ project │ blash │
╰───────────┴───────╯
Example 2: A list of two objects:
[{"id":1,"speed":111},{"id":"2","speed":222}]
Display with no and single (Transposed):
╭────┬───────╮
│ id │ speed │
╞════╪═══════╡
│ 1 │ 111 │
│ 2 │ 222 │
╰────┴───────╯
Display with yes (multiple Simple):
╭───────┬───────╮
│ field │ value │
╞═══════╪═══════╡
│ id │ 1 │
│ speed │ 111 │
╰───────┴───────╯
╭───────┬───────╮
│ field │ value │
╞═══════╪═══════╡
│ id │ 2 │
│ speed │ 222 │
╰───────┴───────╯
Option: --out.level: Verbosity of output🔗
Output messages are categorized into three types:
infooutput contains additional information, such as the number of elements in a tabledataoutput contains the actual output of the command (object, or list of objects)erroroutput contains error messages
The option --out.level controls the level of output:
infodisplays all messages:info,data, anderrordatadisplaysdataanderrormessageserrordisplays only error messages
Option: --out.secrets: Hide or show secrets in results🔗
- If value is
no(default), then secrets are redacted from command results: replaced with🔑, or***if the terminal does not support Unicode (see--out.utf8). - If value is
yes, then secrets are shown in clear in results.
Terminal rendering: Colors and UTF-8🔗
By default, ascli detects the capabilities of the terminal:
| Option | Effect when yes |
Auto-detection (default) |
|---|---|---|
--out.colors |
ANSI colors and styles in results, messages and logs | no if NO_COLOR is set or results are written to a file (--out.file), else yes if CLICOLOR_FORCE is set and not 0, else yes if both stdout and stderr are terminals, TERM is not dumb and CLICOLOR is not 0 |
--out.utf8 |
Unicode characters: table borders, check marks, hidden secrets, symbols in messages, half blocks in images (else ASCII equivalents) | yes if stdout is a terminal and the locale is UTF-8, or on Windows, the terminal is Windows Terminal, VS Code, Git Bash, ConEmu, or sets TERM to xterm* |
Set either option to yes or no to override detection, for example to keep colors when piping to less -R, or to get plain ASCII output in a terminal:
ascli config preset overview --out.colors=no --out.utf8=no
Note
Logs issued before the option is processed (e.g. at startup) use the detected value.
Exit status🔗
The exit code of ascli indicates the category of error, so that scripts can react to it:
| Code | Description |
|---|---|
0 |
Success |
1 |
Other error |
2 |
Command line error: unknown command or option, invalid or missing argument |
3 |
Authentication or authorization failure: SSH, HTTP 401 or 403, OAuth grant |
4 |
Resource not found, or several resources match the identifier, HTTP 404 |
5 |
Transfer failure |
6 |
Network or TLS failure: name resolution, connection, timeout, certificate |
7 |
Other error returned by a remote API |
130 |
Interrupted by user (Ctrl-C) |
With format nagios, the exit code is the status of the check: 0 (OK), 1 (WARNING), 2 (CRITICAL), and 3 (UNKNOWN) if the command fails, following the Nagios plugin guidelines.
Example:
ascli aoc files download /report.csv
case $? in
0) echo 'Downloaded' ;;
6) echo 'Network failure, retry later' ;;
*) echo 'Failed' ;;
esac
Extended Value Syntax🔗
An Extended Value is a structured value - typically a Hash (key/value dictionary) or Array, potentially nested - that is passed on the command line wherever a simple string is not sufficient.
Most options and arguments are specified by a simple string (for example, username or url).
However, some situations require a complex structure:
- An option expects a structured value (
Hash,Array) rather than a plain string. - The value is too long or complex to type inline (for example, a PEM private key, a JSON body).
- The value must be read from a file, an environment variable, or a URL at runtime rather than embedded in the command.
- The value needs type coercion: for example, an integer
1instead of the string"1", or a booleantrue.
The Extended Value syntax addresses all of these needs with a uniform, composable mechanism. It lets you specify what the value is and how to obtain or decode it, entirely on the command line.
Syntax and Decoders🔗
The syntax is:
<0 or more decoders><some text value or nothing>
Decoders act like a function with its parameter on right-hand side and are recognized by the prefix: @ and suffix :
@<NAME>:
Available Decoders🔗
The following decoders are supported:
| Decoder | Parameter | Returns | Description |
|---|---|---|---|
base64 |
String |
String |
Decode a base64 encoded string. |
csvt |
String |
Array |
Decode a titled CSV value. |
env |
String |
String |
Read from the named environment variable. For example, --password=@env:MYPASSVAR |
file |
String |
String |
Read value from specified file (prefix ~/ is replaced with the user's home folder). For example, --key=@file:~/.ssh/mykey |
json |
String |
Any | Decode JSON values. Convenient to provide complex structures. |
lines |
String |
Array |
Split a string in multiple lines and return an Array. |
list |
String |
Array |
Split a string in multiple items taking first character as separator and return an Array. |
none |
None | Nil | A null value. |
path |
String |
String |
Performs path expansion on specified path (prefix ~/ is replaced with the user's home folder). For example, --config-file=@path:~/sample_config.yml |
preset |
String |
Any | Get value from configuration file using dot-path notation: a whole preset (Hash) or a single value. |
extend |
String |
String |
Evaluates embedded Extended Value syntax in string. |
re |
String |
Regexp |
Ruby Regular Expression (short for @ruby:/.../) |
ruby |
String |
Any | Execute specified Ruby code. |
s |
Any | String |
Converts argument to String. |
secret |
String |
String |
Ask password interactively: on terminal (hides input), or on a web page if option ui is graphical. Argument is the prompt. |
stdin |
String |
String |
Read from stdin in text mode. Argument: <empty>, bin or chomp. |
uri |
String |
String |
Read value from specified URL. Supported schemes: http:, https:, data:, file:. For example, --fpac=@uri:http://serv/f.pac or --key=@uri:file:/path/to/key.pem |
val |
String |
String |
Prevent decoding by the decoders on the right. For example, --key=@val:@file:foo sets the option key to value @file:foo. |
vault |
String |
String |
Read a secret from the Secret Vault: <label>.<field>, or a reference //<store>/<path>[?<options>][#<field>]. For example, --password=@vault:my_label.password |
yaml |
String |
Any | Decode YAML. |
zlib |
String |
String |
Decompress data using zlib. |
<empty> |
String |
Any | The dot-path modifier: argument @: collects the following positional arguments as key.subkey=value assignments and builds a Hash or Array using dot-path notation. Shell-friendly alternative to @json: for structured values.Argument is the end marker: <empty> for END, or a custom marker (e.g. @:STOP). Use the end marker to stop collection when further positional arguments must follow.See Positional Arguments with Dot-path for full syntax. |
Note
A few commands support a value of type Proc (lambda expression).
For example, the Extended Value @ruby:'->(i){i["attr"]}' is a lambda expression that returns the value for key attr of the Hash parameter named i.
Chaining rule: When multiple decoders are combined, they are applied right-to-left: the rightmost decoder runs first on the literal string, and each decoder to its left receives the output of the one to its right.
To display the result of an Extended Value, use the config echo command.
The extend decoder is useful to evaluate embedded Extended Value syntax in a string.
It expects a @ to close the embedded Extended Value syntax.
Option parser allows definition of a default parser when the positional parameter or option expects a Hash or Array.
For example, with --parser=json, the parameter {} will be parsed as an empty JSON Hash, even without prefix @json:.
Syntax Equivalences🔗
There are four main ways to express a structured value (for example, a Hash with two keys).
They are all equivalent and produce the same result:
| Syntax | Example | Best for |
|---|---|---|
@json: |
@json:'{"name":"alice","age":30}' |
Scripts, API payloads, copy-paste from docs |
@yaml: |
@yaml:@stdin: + here-doc |
Multi-line or deeply nested structures |
@ruby: |
@ruby:'{"name"=>"alice","age"=>30}' |
Advanced use: lambdas, file reads, logic |
dot-path (key.subkey=value) via @: |
@: name=alice age=30 |
Interactive CLI, simple structures, no quoting |
All four examples above produce the same Hash: {"name" => "alice", "age" => 30}.
ascli config echo @json:'{"name":"alice","age":30}' --format=json
ascli config echo @ruby:'{"name"=>"alice","age"=>30}' --format=json
ascli config echo @: name=alice age=30 --format=json
{"name":"alice","age":30}
Note
The dot-path form (key.subkey=value assignments via @:) does not need quotes for simple string values and is the most shell-friendly.
The @json: form requires enclosing in single quotes to protect " from the shell.
The @ruby: form accepts both single and double quotes and supports arbitrary Ruby expressions.
The @yaml: form is most convenient when combined with a shell here-document (<<EOF).
Equivalences: Positional Parameter vs Option🔗
The same Hash value {"a":1,"b":"two"} can be expressed differently depending on whether it is supplied as a positional parameter or as an option.
As a positional parameter (argument to a command):
# JSON form
some_command @json:'{"a":1,"b":"two"}'
# Dot-path form (END is optional when no positional argument follows)
some_command @: a=1 b=two
some_command @: a=1 b=two END
As an option named opt:
# JSON form
--opt=@json:'{"a":1,"b":"two"}'
# Dot-path form via option name (each key as a separate option flag)
--opt.a=1 --opt.b=two
# Dot-path form via option value (consumes following positional arguments)
--opt=@: a=1 b=two END
Note
--opt.a=1 --opt.b=two and --opt=@json:'{"a":1,"b":"two"}' and --opt=@: a=1 b=two all produce the same Hash assigned to option opt.
The --opt.key=value form is the most concise when only a few keys need to be set.
Use END with @: when further positional arguments follow on the command line; it is optional otherwise.
Which Syntax to Choose?🔗
When in doubt: start with dot-path (--opt.key=value for options, @: key=value for positional arguments).
It requires no quoting, auto-converts types, and reads naturally on the command line.
Switch to @json: when you already have a JSON payload from an API doc or external tool.
| Context | Recommended syntax | Reason |
|---|---|---|
| Interactive CLI, simple values | dot-path key.subkey=value via @: or --opt.key=value |
No quoting needed, auto-typed, readable |
| Shell scripts, API payloads | @json: |
Standard, portable, easy to copy from API docs |
| Multi-line or nested structures in scripts | @yaml: + @stdin: heredoc |
Readable, no escaping |
| Dynamic values, lambdas, file reads | @ruby: |
Full Ruby expressiveness |
Common Usage Examples🔗
Example: Create a Hash value - for a small structure, the dot-path key.subkey=value form (via @:) is simpler; for larger or API-sourced structures, use @json::
ascli config echo @: key1=value1 key2=value2
ascli config echo @json:'{"key1":"value1","key2":"value2"}'
Example: read the content of the specified file, then, base64 decode, then unzip:
ascli config echo @zlib:@base64:@file:myfile.dat
Example: Create a Hash value with one key and the value is read from a file:
ascli config echo @ruby:'{"token_verification_key"=>File.read("mykey.txt")}'
Example: read a CSV file and create an Array of Hash for bulk provisioning:
cat test.csv
name,email
lolo,laurent@example.com
toto,titi@tutu.tata
ascli config echo @csvt:@file:test.csv
╭──────┬─────────────────────╮
│ name │ email │
╞══════╪═════════════════════╡
│ lolo │ laurent@example.com │
│ toto │ titi@tutu.tata │
╰──────┴─────────────────────╯
Example: create a Hash with values coming from a preset named config
ascli config echo @json:@extend:'{"hello":true,"version":"@preset:config.version@"}'
╭─────────┬────────╮
│ field │ value │
╞═════════╪════════╡
│ hello │ true │
│ version │ 4.21.1 │
╰─────────┴────────╯
Example: Create a Hash from YAML provided as shell Here document:
ascli config echo @yaml:@stdin: --format=json<<EOF
key1: value1
key2:
- item1
- item2
key3:
key4: value4
key5: value5
EOF
{"key1":"value1","key2":["item1","item2"],"key3":{"key4":"value4","key5":"value5"}}
Schema Discovery with help🔗
Any option or Command Parameter that expects a Hash value accepts the special value help.
Instead of performing the action, ascli displays the schema of the expected Hash as a table (field name, type, description), then exits with an error.
When a mandatory Hash argument is missing, ascli automatically hints to use help:
ascli aoc packages send
ERRR Missing: Missing argument: package (Hash)
HINT:Give `help` as argument to retrieve the schema of the missing argument.
Following the hint and passing help as the argument displays the schema:
ascli aoc packages send help
INFO Schema: argument: package (Hash)
╭───────────────────────┬────────┬──────────┬────────────────────────────────────────────────────────────────────────────────────────────╮
│ name │ type │ required │ description │
╞═══════════════════════╪════════╪══════════╪════════════════════════════════════════════════════════════════════════════════════════════╡
│ bcc_recipients │ array │ false │ <empty string> │
│ bcc_recipients[].id │ string │ true │ The ID of the recipient. │
│ bcc_recipients[].type │ enum │ false │ The entity type of the recipient. │
│ │ │ │ Allowed: user, group │
│ name │ string │ true │ Package name. Required for POST. Optional for PUT. │
│ note │ string │ false │ The sender's message to recipients to include with the package. Maximum characters: 65535. │
│ recipients │ array │ false │ <empty string> │
│ recipients[].id │ string │ true │ The ID of the recipient. │
│ recipients[].type │ enum │ false │ The entity type of the recipient. │
│ │ │ │ Allowed: user, group │
...
╰───────────────────────┴────────┴──────────┴────────────────────────────────────────────────────────────────────────────────────────────╯
The same applies to options: display the schema of the transfer-spec option ts:
ascli --ts=help
INFO Schema: option: ts
╭─────────────────────┬─────────┬──────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ name │ type │ required │ description │
╞═════════════════════╪═════════╪══════════╪════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
│ apply_local_docroot │ boolean │ false │ Apply local docroot to source paths. │
│ cipher │ enum │ false │ In transit encryption algorithms. │
│ │ │ │ Allowed: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm │
│ authentication │ string │ false │ Set to `token` for SSH bypass keys, else password asked if not provided. │
...
╰─────────────────────┴─────────┴──────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
This works for any Hash option or positional parameter that has a defined schema.
Schema Validation🔗
The value of an option or Command Parameter with a schema defined by ascli (e.g. options ts, transfer, out, http_options, argument of orchestrator workflows start) is validated against it before the action.
An invalid value is rejected with the path of the invalid element and the reason:
ascli config echo 1 --ts=@json:'{"direction":"sideways"}'
ERRR Argument: Option ts: value at `/direction` is not one of: ["send", "receive"] (use --ts=help for schema)
Use option -h to get help.
Option values are merged from several sources (presets, command line), so mandatory fields are not checked for options.
Request bodies of product APIs (e.g. create and modify commands) are not validated by ascli: the API validates them.
Testing Extended Value🔗
Two complementary commands help verify that a value is parsed as expected:
| What to check | Tool | How |
|---|---|---|
| Value of a positional parameter | config echo |
Pass the expression as the sole argument |
| Value of an option | --show-config |
Append --show-config to any command line |
Checking a positional parameter with config echo:
config echo takes exactly one argument which can use the Extended Value syntax, and displays its parsed value.
Unprocessed command line arguments are shown in the error message.
ascli config echo @json:'{"a":1,"b":"two"}' --format=json
ascli config echo @: a=1 b=two --format=json
{"a":1,"b":"two"}
Note
The value is shown after both shell command-line parsing and ascli Extended Value parsing.
Example: the shell parses three arguments (1, 2, 3), but config echo only processes the first:
ascli config echo 1 2 3
1
ERRR Argument: unprocessed values: ["2", "3"]
Checking an option value with --show-config:
Adding --show-config to any command line performs a dry run and displays the resolved value of all options that would be used, without executing the command.
To display a specific option, add --fields=<option_name>.
Add --out.flat=no when the option holds a structured value (Hash, Array) to display it as-is rather than flattened into dot-path keys:
ascli --opt=@json:'{"a":1,"b":"two"}' some_plugin --show-config --fields=opt --out.flat=no
ascli --opt.a=1 --opt.b=two some_plugin --show-config --fields=opt --out.flat=no
Both lines above display the same resolved value for option opt.
In the following examples (using a POSIX shell, such as bash), several equivalent commands are provided.
For all examples, most special character handling is not specific to ascli:
It depends on the underlying syntax: shell, JSON, and so on.
Depending on the case, a different format option is used to display the actual value.
For example, in the simple string Hello World, the space character is special for the shell, so it must be escaped so that a single value is represented.
Double quotes are processed by the shell to create a single string argument.
For POSIX shells, single quotes can also be used in this case, or protect the special character (space) with a backslash.
ascli config echo "Hello World" --format=text
ascli config echo 'Hello World' --format=text
ascli config echo Hello\ World --format=text
Hello World
Security Level🔗
On the command line, ascli can do anything that the user running it is allowed to do.
Option security restricts the features that access local resources or execute code, for example, when commands come from another program, such as an MCP client (see MCP Security).
It is a Hash with the following keys:
| Key | Type | Default | Description |
|---|---|---|---|
level |
string |
low |
low, medium, or high |
allowed_paths |
Array<string> |
- | With level medium: files and folders that @file: can read |
writable_paths |
Array<string> |
- | With levels medium and high: folders where files can be written |
| Level | @file: |
@ruby:, @uri:, and @env: |
Secret processed by another extended value | Write files | Features that execute code |
|---|---|---|---|---|---|
low |
Allowed | Allowed | Allowed | Allowed | Allowed |
medium |
Only files in allowed_paths |
Refused | Refused | Only in writable_paths |
Refused |
high |
Refused | Refused | Refused | Only in writable_paths |
Refused |
With level medium, the real path of the file is checked: a symbolic link leading outside of allowed_paths is refused.
With levels medium and high, files are written only in writable_paths (none by default): downloaded files (config download, and transfers that receive files), because a file written where the client chooses could replace a file that is used later, for example a plugin or a shell startup file.
The real path of the folder is checked: neither .. nor a symbolic link leading outside of writable_paths is accepted.
Note
Reading files is not restricted for transfers: a command that sends files (for example server upload) reads the files that it is given, whatever the level.
The security level restricts extended values, not the paths of transfers.
The same applies to @file:<path> in the parameters of stores of option vault, and in option vault_password (for example, token), although the file is read when used, not when the option is evaluated.
A secret (read by @secret:, @vault:, or @preset:) is hidden in results by its value: processed by another extended value, it could be revealed (for example, split by @list:@vault:..., or with a short value in a text by @extend:).
With levels medium and high, a secret must be the whole value of an option or argument, for example @preset:<PRESET_NAME>.password for option password, and it cannot be a parameter name of config preset set.
Features that execute code (own, or downloaded), load code, or write a file chosen by the client, are available only at level low:
| Feature | Why |
|---|---|
Option notify_template |
ERB template: Ruby code |
Option fpac |
Proxy auto configuration: JavaScript code |
Option ui, key command |
Command executed to open a URL |
Options sdk_url and locations_url |
The SDK, including ascp, is downloaded from this location, then executed |
Option out, key file (and output), option pid_file |
Write (or truncate, or delete) a file chosen by the client |
Option home |
Configuration folder: its presets, and its folder plugins (Ruby code) are used |
Option plugin_folder |
Plugins are Ruby code |
Plugins of the folder plugins of the configuration folder, and of option plugin_folder |
Ruby code: not loaded at other levels, also if the file was there before |
Command config plugins create |
Creates Ruby code, loaded as a plugin |
Command faspex5 postprocessing |
Executes scripts, or Ruby code |
At other levels, an option is refused whatever the way it is set (command line, or preset), and a command is not listed by config commands.
Values from the environment of the process (for example ASCLI_HOME) are not restricted: they are set by the one who starts the process, not by a client.
A sub-option of an option is restricted in the same way: it is marked x-security: low in the schema of the option (for example, file of option out), and refused when given, by dot notation (--out.file=...) or in a Hash.
A preset is used later, maybe at level low: at level medium and high, config preset set, update and initialize refuse such an option, and a value that contains an extended value that the level refuses (for example @val:@ruby:...).
Options are evaluated in the order of the command line: place --security before options that use extended values.
ascli config echo @file:/etc/hosts --security.level=high
ERRR Argument: Evaluation of @file: for argument: value: @file: extended value is disabled by security level high: use level medium with the file in security.allowed_paths (option security), or give the value itself
Main, configuration and Persistency Folder🔗
ascli looks for configuration and persistency files (token cache, file lists, persistency files) in the folder specified using option home.
The default value is [User's home folder]/.aspera/ascli.
Note
The [User's home folder] is determined using Ruby's Dir.home method.
Primary source: The HOME environment variable.
On Windows: Ruby also checks %HOMEDRIVE%%HOMEPATH% and %USERPROFILE% (via rb_w32_home_dir).
Additionally, ascli sets the %HOME% environment variable to the value of %USERPROFILE% if it exists and is valid.
Therefore, on Windows, %USERPROFILE% is preferred because it is generally more reliable than %HOMEDRIVE%%HOMEPATH%.
The configuration folder can be displayed using:
ascli config folder
/Users/kenji/.aspera/ascli
Note
This is equivalent to displaying the value of the home option.
ascli --show-config --fields=home
Example (Windows):
set ASCLI_HOME=C:\Users\Kenji\.aspera\ascli
ascli config folder
C:\Users\Kenji\.aspera\ascli
When OAuth is used (AoC, Faspex 5), ascli keeps a cache of generated bearer tokens in folder persist_store located in the configuration folder by default.
Option cache_tokens (yes/no) allows controlling if OAuth tokens are cached on file system, or generated for each request.
The command config tokens flush clears that cache.
Tokens are kept on disk for a maximum of 30 minutes and garbage collected after that.
When a token has expired, then a new token is generated, either using a refresh_token if it is available, or by the default method.
Configuration file🔗
On the first execution of ascli, an empty configuration file is created in the configuration folder (ascli config folder).
There is no mandatory information required in this file.
Its use is optional, as any option can be provided on the command line.
Although the file is a standard YAML file, ascli provides commands to read and modify it using the config command.
All options for ascli can be set on command line, or by env vars, or using Option Preset in the configuration file.
A configuration file provides a way to define default values, especially for authentication options, thus avoiding having to always specify those options on the command line.
The default configuration file is: $HOME/.aspera/ascli/config.yaml (this can be overridden with option config_file).
The configuration file is a catalog of named lists of options, called: Option Preset.
Then, instead of specifying some common options on the command line (for example, address, credentials), it is possible to invoke the ones of an Option Preset (for example, <PRESET_NAME>) using the option preset: --preset=<PRESET_NAME> or its shortcut: -P<PRESET_NAME>.
Format of configuration file🔗
The configuration file is a Hash in a YAML file.
Example:
config:
version: 0.3.7
default:
config: cli_default
server: demo_server
cli_default:
interactive: no
demo_server:
url: ssh://demo.asperasoft.com:33001
username: asperaweb
password: <PASSWORD>
We can see here:
- The configuration was created with
ascliversion 0.3.7 - The default Option Preset to load for
serverplugin is:demo_server - The Option Preset
demo_serverdefines some options: the URL and credentials - The default Option Preset to load in any case is:
cli_default
Two Option Presets are reserved:
configcontains a single value:versionshowing the version used to create the configuration file. It is used to check compatibility.defaultis reserved to define the default Option Preset name used for known plugins.
The user may create as many Option Presets as needed. For instance, a particular Option Preset can be created for a particular application instance and contain URL and credentials.
Values in the configuration also follow the Extended Value syntax.
Note
If the user wants to keep the Extended Value syntax inside the configuration file, the user shall use the @val: prefix.
Example:
ascli config preset set <AOC_ORG> private_key @val:@file:"$HOME/.aspera/ascli/<PKEY_NAME>"
This creates the Option Preset:
<AOC_ORG>:
private_key: "@file:/Users/laurent/.aspera/ascli/<PKEY_NAME>"
So, the key file is read only at execution time and is not embedded in the configuration file.
Note
The main use of the configuration file is to store collections of options.
Nevertheless, it can be used to store any type of information, not only option values.
For example, the special preset: default.
Option Preset🔗
An Option Preset is a collection of options and their associated values in a named section in the configuration file.
A named Option Preset can be modified directly using ascli, which will update the configuration file:
ascli config preset <show|delete|get|unset|set|initialize|update|ask> <PRESET_NAME> ...
The syntax of all config preset commands, including list and overview, is displayed with ascli config commands config preset.
The command initialize allows setting several options at once, but it deletes an existing configuration instead of updating it, and expects a Hash Extended Value.
ascli config preset initialize demo_server @json:'{"url":"ssh://demo.asperasoft.com:33001","username":"asperaweb","password":"<PASSWORD>","ts":{"precalculate_job_size":true}}'
One can also use positional arguments with dot-path:
ascli config preset initialize demo_server @: url=ssh://demo.asperasoft.com:33001 username=asperaweb password=<PASSWORD> ts.precalculate_job_size=true
The command update creates an Option Preset from options provided on the command line, for example:
ascli config preset update demo_server --url=ssh://demo.asperasoft.com:33001 --username=asperaweb --password=<PASSWORD> --ts=@json:'{"precalculate_job_size":true}'
This creates an Option Preset demo_server with all provided options.
Note
update takes ALL options provided on the command line (starting with -- with a value).
The command set allows setting individual options in an Option Preset:
ascli config preset set <PRESET_NAME> <PARAMETER_NAME> <PARAMETER_VALUE>
Example:
ascli config preset set demo_server password <PASSWORD>
The parameter name supports dot-notation to set nested hash options. If an intermediate hash already exists, the new value is deep-merged into it (existing sibling keys are preserved).
ascli config preset set GLOBAL out.table.pivot single
ascli config preset set GLOBAL out.level data
The parameter value is automatically coerced to its natural type: integers, floats and booleans (true/yes/false/no) are stored as native YAML types rather than strings.
To delete a key from a preset, pass @none: as the value (evaluates to nil):
ascli config preset set GLOBAL out.table.pivot @none:
A full terminal based overview of the configuration can be displayed using:
ascli config preset overview
The list of Option Presets can be displayed using:
ascli config preset list
Any key starting with _ (underscore) in a preset is treated as a comment and is ignored when the preset is applied.
This is useful to annotate presets directly in the configuration file:
demo_server:
_comment: "Demo server - public credentials"
url: ssh://demo.asperasoft.com:33001
username: asperaweb
password: <PASSWORD>
Note
Comment keys are preserved and visible in config preset overview for inspection, but are never injected as CLI options.
By convention, _comment holds a short description of the preset.
The shell completion of preset names (e.g. -P <tab>) shows it next to each name.
The wizard sets it for the presets it creates: _comment: "wizard: <application>".
A good practice is to not manually edit the configuration file and use modification commands instead. If necessary, the configuration file can be opened in a text editor with:
ascli config open
Note
This starts the editor specified by env var EDITOR if defined.
It is possible to load an Option Preset from within another Option Preset using the preset option.
For example if pcommon is a preset with common options, and pspecific is a preset with specific options, then pspecific can load pcommon using:
ascli config preset set pspecific preset pcommon
When pspecific is loaded, then cumulative option preset will be set, and it will also load pcommon.
Special Option Preset: config🔗
This preset name is reserved and contains a single key: version.
This is the version of ascli which created the file.
Special Option Preset: default🔗
This preset name is reserved and contains a Hash, where the key is the name of a plugin, and the value is the name of another preset.
Usually, Option presets are used to contain pre-defined options and values, but this preset contains names of presets to be used by default for plugins.
When a plugin is invoked, the preset associated with the name of the plugin is loaded, unless the option --no-default (or -N) is used.
Note
The special plugin name config in the default preset points to a preset containing global default options
(such as --log-level, --vault, --interactive), loaded before any plugin is invoked.
To set global default options, use the GLOBAL keyword (see Plugin: config).
Operations on this preset are done using regular config operations:
ascli config preset set default _plugin_name_ _default_preset_for_plugin_
Note
default is not a command, it is the name of the special preset.
ascli config preset get default _plugin_name_
"_default_preset_for_plugin_"
Handling invalid filename characters🔗
Some commands of ascli may create files or folders based on input that may contain invalid characters for the local file system.
The option invalid_characters allows specifying a replacement character for a list of characters that are invalid in filenames on the local file system and replaces them with the specified character.
The first character specifies the replacement character, and the following characters are the invalid ones.
This is used when a folder or file is created from a value that potentially contains invalid characters.
For example, using the option package_folder, a package name may contain characters not allowed, such as /.
The default value is _<>:"/\|?*, corresponding to replacement character _ and characters not allowed on Windows.
Note
This option is different from the replace_illegal_chars parameter in aspera.conf, which applies to transfers only.
Managing temporary files🔗
Some temporary files may be needed during runtime, for example the file list of ascp.
The composite option temp (use --temp=help to display this schema) controls them:
| Field | Type | Description |
|---|---|---|
clean |
Bool |
Delete temporary files created by the command when it ends. false keeps them, e.g. for debugging. Files left by previous commands are still deleted according to expiry.Default: true. |
expiry |
String, Number, Bool |
Age after which file lists of ascp left by previous commands (e.g. interrupted) are deleted, when a command starts. They are in folder filelists of the main folder (option home). A duration, e.g. 5d, 12h, 1w, 1h 30m, 1d 02:00:00, a plain number is seconds. false or null (@none:): never delete them. 0: delete all of them, including file lists of transfers in progress in other executions of ascli. Such a transfer fails if ascp reads its file list again: next session of a multi-session transfer, or resume after an error. Use 0 only when no other transfer is in progress, not in a permanent configuration.Default: 5d. |
folder |
String |
Folder for temporary files, e.g. certificate for WSS transfers, logs of transferd.- @sys: temporary folder of the system for the current user (Ruby Etc.systmpdir).- @env: from environment variables, e.g. TMPDIR (Ruby Dir.tmpdir).Default: @sys. |
A plain value is the folder, for example: --temp=/data/tmp.
Temporary files of a command are deleted when it ends, unless clean is false.
If a command is interrupted (for example, killed), its ascp file lists remain: a later command deletes them when they are older than expiry.
Warning
--temp.expiry=0 also deletes the file lists of transfers in progress in other executions of ascli, which may then fail: use it only when no other transfer is in progress.
Tip
Options temp_folder and clean_temp are deprecated aliases for temp.folder and temp.clean.
Plugin: config: Configuration🔗
Plugin config provides general commands for ascli:
- Option Preset operations (configuration file)
wizardvaultascptransferd(deprecated, use plugintransferd)
The default preset for config is read for any plugin invocation: this allows setting global options, such as --log-level or --interactive.
When ascli starts, it looks for the default Option Preset and checks the value for config.
If set, it loads the options independently of the plugin used.
Note
If default.config is not set, ascli automatically uses global_common_defaults as the global preset name
and saves that mapping in the configuration file the first time the GLOBAL keyword is used.
Tip
If you do not know the name of the global preset, you can use GLOBAL to refer to it.
Show current default (global) Option Preset (config plugin):
ascli config preset get default config
global_common_defaults
Set a global parameter:
ascli config preset set GLOBAL version_check_days 0
If you want to use a custom name instead of global_common_defaults:
ascli config preset set default config <GLOBAL_OPTIONS_NAME>
ascli config preset set GLOBAL version_check_days 0
Tested commands for config🔗
Note
Add ascli config in front of the following commands:
agents jobs cleanup
agents jobs list
agents jobs status <CONF_ASYNC_NODE_UPLOAD>
agents list
agents parameters connect
agents parameters direct
agents parameters node
agents show connect
agents show direct
ascp errors
ascp info --sdk-folder=sdk_test_dir
ascp products list
ascp schema --format=jsonpp
ascp show
ascp spec
check_update
coffee
coffee --log-level=trace2 --log-format=caller
coffee --ui=text
coffee --ui=text --out.img.text=true
coffee --ui=text --out.img=@json:'{"text":true,"double":false}'
commands
commands aoc --expand-mounts=yes
commands aoc files
commands server
detect @: url=app.example.com
detect @: url=https://f5.example.com/path
detect @: url=https://f5.example.com/path plugin=faspex5
detect @: url=https://node.example.com/path
detect @: url=https://server1.example.com/path
detect @: url=https://shares.example.com/path plugin=shares
detect @: url=https://tst.example.com/path plugin=faspio
detect @: url=https://tst.example.com/path plugin=httpgw
detect @: url=my_org plugin=aoc
doc
doc github transfer-parameters
doc local quick-start --ui=text
doc toc
echo '<svg viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg"><circle cx="50" cy="50" r="50" fill="#069"/></svg>' --format=image --out.img.text=true
echo -- --special-string
echo @base64:SGVsbG8gV29ybGQK
echo @csvt:@stdin:
echo @env:USER
echo @json:'[{"email":"a@x.com"},{"email":"b@x.com"},{"email":"c@y.org"}]' --out.query=@ruby:'->(users){users.group_by{|u|u["email"].split("@").last}.transform_values(&:count)}' --format=json
echo @json:'[{"name":"alice","size":10},{"name":"bob","size":2000}]' --out.query='[?size > `100`].name' --format=text
echo @json:'[{"user":{"id":1,"name":"foo"},"project":"bar"}]' --out.table.pivot=single
echo @json:'[{"user":{"id":1,"name":"foo"},"project":"bar"}]' --out.table.pivot=yes
echo @json:'{"empty":"","bool":true}' --out.colors=yes --out.utf8=yes
echo @lines:@stdin:
echo @list:,1,2,3
echo @secret: --ui=text --out.secrets=yes
echo @stdin:bin
echo @uri:/etc/hosts
echo @uri:file:/etc/hosts
echo @uri:http://ifconfig.me
echo @uri:https://ifconfig.me
echo @uri:https://ifconfig.me/ip --cert-stores=@ruby:[OpenSSL::X509::DEFAULT_CERT_DIR,OpenSSL::X509::DEFAULT_CERT_FILE] --http-options.ssl_options=@list:,CIPHER_SERVER_PREFERENCE,-NETSCAPE_CA_DN_BUG --insecure=yes
echo @vault:my_preset.password
echo @zlib:@stdin:
echo hello
email_test --notify-to=my_email_external
file
file --log-level=debug --log-format=@ruby:'->(s, d, p, m){"#{d.strftime("%Y-%m-%d %H:%M:%S")} - #{p} - #{s} - #{m}\n"}'
file --log-level=debug --log-format=default
file --log-level=debug --log-format=standard
file --log-level=debug --logger=syslog
folder
gem name
gem path
gem version
genkey my_key
genkey my_key 4096
image https://eudemo.asperademo.com/wallpaper.jpg
initdemo
open
options aoc
options server
options server --select=@json:'{"option":"--display"}' --fields=replacement
plugins create my_command .
plugins list
preset delete conf_name
preset initialize conf_name @json:'{"p1":"v1","p2":"v2"}'
preset list
preset overview
preset set GLOBAL sdk_folder 'product:IBM Aspera Connect'
preset set GLOBAL vault @: default=local stores.local.type=file stores.local.file=/secure/vault_file
preset set GLOBAL vault_password <PASSWORD>
preset set GLOBAL version_check_days 0
preset set conf_name param value
preset set default shares conf_name
preset show conf_name
preset unset conf_name param
preset update conf_name --p1=v1 --p2=v2
proxy_check --fpac=@file:proxy.pac https://eudemo.asperademo.com --proxy-credentials=@list:,user,pass
pubkey @file:my_key
remote_certificate chain https://node.example.com/path
remote_certificate name https://node.example.com/path
remote_certificate only https://node.example.com/path
smtp_settings
sync spec
tokens flush
tokens list
vault check @vault://local/my_label
vault create @: label=my_label password=<PASSWORD> 'description=my secret'
vault delete foo --vault.type=system
vault delete my_label
vault field my_label
vault ids
vault info
vault list
vault overview
vault show my_label
wizard @: url=https://console.example.com/path plugin=console override=yes --username=my_username --password=<PASSWORD>
wizard @: url=https://f5.example.com/path plugin=faspex5 override=yes --username=my_username --private-key=@file:<PRIVATE_KEY_FILE> --client-id=<CLIENT_ID> --client-secret=<CLIENT_SECRET>
wizard @: url=https://node.example.com/path plugin=node --username=test --password=<PASSWORD>
wizard @: url=https://orch.example.com/path plugin=orchestrator --username=test --password=<PASSWORD>
wizard @: url=https://server1.example.com/path plugin=server --username=my_username --password=<PASSWORD>
wizard @: url=https://shares.example.com/path plugin=shares --username=test --password=<PASSWORD>
wizard @: url=https://tst.example.com/path plugin=faspio override=yes --username=my_username --password=<PASSWORD>
wizard @: url=https://tst.example.com/path plugin=httpgw
wizard @: url=my_org plugin=aoc --username=my_user_email --private-key=@file:<PRIVATE_KEY_FILE> --use-generic-client=yes
wizard @: url=my_org plugin=aoc preset=my_preset --username=my_user_email --private-key=@file:<PRIVATE_KEY_FILE>
Evaluation order of options🔗
Some options are global, others are available only for some plugins (the plugin is the first-level command).
Options are loaded using this algorithm:
- If option
--no-default(or-N) is specified, then no default value is loaded for the plugin - Else it looks for the name of the plugin as key in section
default, the value is the name of the default Option Preset for it, and loads it. - If option
--preset=<NAME>is specified (or-P<NAME>), this reads the Option Preset specified from the configuration file by name. - If option
--preset=<EXTENDED_VALUE_HASH>is specified, it is used as option values (Hashof option/value pairs). - Environment variables are evaluated.
- Command line options are evaluated.
Options are evaluated in the order of command line.
To avoid loading the default Option Preset for a plugin, use: -N
On command line, words in option names are separated by a dash (-).
In the configuration file, the separator is an underscore.
For example, --xxx-yyy on command line gives xxx_yyy in configuration file.
The main plugin name is config, so it is possible to define a default Option Preset for the main plugin with:
ascli config preset set cli_default interactive no
ascli config preset set default config cli_default
An Option Preset value can be removed with unset:
ascli config preset unset cli_default interactive
Example: Define options using command line:
ascli -N --url=<URL> --password=<PASSWORD> --username=<USERNAME> node --show-config
Example: Define options using a Hash:
ascli -N --preset=@json:'{"url":"<URL>","password":"<PASSWORD>","username":"<USERNAME>"}' node --show-config
Shell Completion🔗
ascli supports shell tab-completion for Bash, Zsh, Fish, and PowerShell (Windows PowerShell 5.1 and PowerShell 7, on any OS).
The command ascli config completion <shell> displays the completion script for the given shell.
The default shell is Zsh on macOS, Bash on most Linux distributions, and PowerShell on Windows (its prompt starts with PS).
In Bash and Zsh, echo $0 displays the name of the running shell.
To activate completion, add the line for your shell to its startup file:
# Bash, in ~/.bashrc
eval "$(ascli config completion bash)"
# Zsh, in ~/.zshrc, after compinit
eval "$(ascli config completion zsh)"
# Fish, in ~/.config/fish/config.fish
ascli config completion fish | source
# PowerShell, in $PROFILE (edit with notepad $PROFILE)
ascli config completion powershell | Out-String | Invoke-Expression
The startup file then executes ascli each time a shell starts.
Alternatively, save the script once in the completion folder of the shell (and save it again after an upgrade of ascli):
# Bash, with package bash-completion
ascli config completion bash > ~/.local/share/bash-completion/completions/ascli
# Zsh (the folder must be in $fpath before compinit)
ascli config completion zsh > ~/.zsh/completions/_ascli
# Fish
ascli config completion fish > ~/.config/fish/completions/ascli.fish
PowerShell has no completion folder: save the script, and dot-source it in $PROFILE:
ascli config completion powershell | Set-Content "$HOME\ascli_completion.ps1"
Add-Content -Path $PROFILE -Value ". '$HOME\ascli_completion.ps1'"
In PowerShell, the words proposed are kept for the session, so only the first Tab at a given position executes ascli.
Once active, press Tab to complete:
ascli <Tab> # plugins and global options: aoc, server, node, ..., --format, ...
ascli server <Tab> # sub-commands and options of plugin: upload, download, ls, ..., --url, ...
ascli aoc admin <Tab> # sub-commands at any depth: user, node, ...
ascli server ls --u<Tab> # options, after any command: --url, --username, ...
ascli --format=<Tab> # values of option: --format=json, --format=table, ...
ascli --ts.<Tab> # keys of option, from its schema: --ts.direction=, --ts.target_rate_kbps=, ...
ascli --out.t<Tab> # a sub-structure is followed by a dot: --out.table.
ascli --ts.direction=<Tab> # values of key: --ts.direction=send, --ts.direction=receive
ascli --transfer=<Tab> # plain value of option, key agent: --transfer=direct, --transfer=node, ...
ascli faspex5 admin accounts list --query.<Tab> # keys of --query, from the command: --query.limit=, ...
ascli server upload <Tab> # local files
ascli server download --to-folder <Tab> # local folders
ascli config genkey <Tab> # local files, where a local path is expected
ascli --temp.folder=<Tab> # local folders, for a key of option
ascli --ts=@<Tab> # extended value modifiers: @json:, @file:, @preset:, @:, ...
ascli --private-key=@file:<Tab> # local files, after @file: or @path:
ascli -P <Tab> # presets: also for config preset show, @preset:
ascli config preset get myconf <Tab> # parameters of preset: url, username, ...
ascli config agents jobs status <Tab> # asynchronous transfer jobs
ascli node access_keys do <Tab> # type of expected value, not inserted: <identifier>
ascli aoc packages send <Tab> # type, and keys of structure (required ones with *): <Hash> name*, workspace_id*, ...
ascli aoc packages send @: <Tab> # keys of structure built with @:, from its schema: name=, workspace_id=, recipients.0., ...
ascli aoc packages send @: archived=<Tab> # values of key: archived=true, archived=false
ascli aoc packages send @: name=x <Tab> # end marker of structure built with @: END, and the other keys
No space is inserted after a word that continues: key=, key., @file:, or a folder (Bash 4 and later).
Zsh, Fish and PowerShell also display the description of commands, options, keys and extended value modifiers.
In PowerShell, a word starting with @ is inserted with quotes (e.g. '@file:README.md'), as required by PowerShell.
Zsh lists options once - is typed (zsh style prefix-needed).
The type of an expected value is displayed by Bash and Zsh, and by PowerShell in the list of MenuComplete (Ctrl+Space).
The scripts call ascli config completion candidates -- [<word>...] <current word>, which lists the candidates for the word at the cursor, with their description, followed by a directive for the shell: :files or :folders (local paths, followed by the beginning of the word that is not part of the path, if any, e.g. :files --private-key=@file:), :hint <text> or :.
The words are not evaluated (e.g. @env: is not replaced).
The command ascli config completion words [<word>...] lists the words that can follow the given ones.
It can be used directly to inspect available commands:
ascli config completion words aoc admin
Wizard🔗
The wizard is a command that asks the user for information and creates an Option Preset with the provided information for a given application.
It takes one optional argument: a Hash, usually given with @: as key=value items, in any order:
| Key | Description | If not provided, ascli |
|---|---|---|
url |
URL or hostname of the application | asks for it. |
plugin |
The plugin name to limit search to | tries to detect known plugins from the URL |
preset |
The preset name to save in config file | generates a unique preset name |
default |
Set the preset as default configuration for the plugin (yes/no) |
yes |
override |
Override the existing default preset of the plugin, if it exists (yes/no) |
no |
Options of the plugin (for example --username, --password, --client-id, --private-key), see the plugin, are given as usual on the command line: they are added to the Option Preset created by the wizard.
Options that the wizard needs, but that are not provided, are asked.
An unknown key in the Hash is refused.
For example:
ascli config wizard @: url=<URL> plugin=<PLUGIN> preset=<PRESET>
The command config detect takes the same argument, with keys url and plugin, and shows the detected applications, without creating a preset.
Use ascli config wizard help to show the schema of the argument.
The simplest invocation (without argument: the URL is asked) is:
ascli config wizard
If the application requires a private key, the user can provide it with option --private-key (the key, e.g. @file:<path>, not only its path), or let the wizard generate one.
The user is told where to place the associated public key PEM in the application.
Example of configuration for a plugin🔗
For Faspex 5, Shares, Node (including ATS, Aspera Transfer Service), Console, only username/password and URL are required (either on command line, or from configuration file). Those can be provided on the command line:
ascli shares files browse / --url=https://10.25.0.6 --username=john --password=<PASSWORD>
This can also be provisioned in a configuration file:
- Build Option Preset
ascli config preset set shares06 url https://10.25.0.6
ascli config preset set shares06 username john
ascli config preset set shares06 password <PASSWORD>
This can also be done with one single command:
ascli config preset initialize shares06 @json:'{"url":"https://10.25.0.6","username":"john","password":"<PASSWORD>"}'
Or:
ascli config preset update shares06 --url=https://10.25.0.6 --username=john --password=<PASSWORD>
- Define this Option Preset as the default Option Preset for the specified plugin (
shares)
ascli config preset set default shares shares06
- Display the content of the configuration file in table format
ascli config preset overview
- Execute a command on the Shares application using default options
ascli shares files browse /
Secret Vault🔗
Secrets, for example, passwords, keys, are needed when connecting to applications. Those secrets are usually provided as command options: on the command line, in env vars, in files, and so on.
For security reasons, those secrets shall not be exposed in clear, either:
- On terminal during input
- In logs
- In command output
Instead, they shall be hidden (logs, output) or encrypted (configuration).
Terminal output (command result) secret removal is controlled by option --out.secrets (default: no).
Log secret removal is controlled by option log_secrets (default: no).
Secrets are identified by the name of their field (for example, password, token), and by their value: values of secret options and of presets, of Extended Values @secret: and @vault:, tokens obtained or generated, and secrets of transfer specs are hidden wherever they appear, also in field names.
In presets, a secret made of digits only (for example, a password not quoted in the YAML file) is converted to a string, so that it is identified as a secret.
A value shorter than 8 characters is hidden only when it is the whole value, not inside a text, except in responses of mcp server, where it is also hidden as a word (not next to a letter or digit).
Mandatory command line options can be requested interactively (for example, password) using option interactive.
It is possible to use Extended Value @secret:[name] to ask for a secret interactively.
If option ui is graphical (see Graphical Interactions), the secret is entered on a web page opened in the browser, served by a local web server on 127.0.0.1, else on terminal.
In MCP server mode, the secret is entered on a web page, see Secrets entered in a web page.
It is also possible to enter an option as an environment variable, for example, ASCLI_PASSWORD for option password and read the env var like this:
read -s ASCLI_PASSWORD
export ASCLI_PASSWORD
Another possibility is to retrieve values from a secret vault.
Option vault declares named secret stores (stores), and the name of the default store (default):
@vault:<label>.<field>reads field<field>of secret<label>of the default store, managed with commandconfig vault(see Vault: Operations).@vault://<store>/<path>[?<options>][#<field>]reads a secret of any store (see Vault: Secret references).
| Field | Type | Description |
|---|---|---|
cache_ttl |
Number |
Time in seconds a value read by reference is kept in memory, 0 for no cache.Default: 60. |
default |
String |
Name of the default store, in stores: read by @vault:<label>.<field>, managed by config vault, unlocked with option vault_password. No default store if not set. |
stores |
Hash |
Name of store (letters, digits, _, ., -) → parameters of store, with type. Secrets are read by reference with @vault://<store>/<path>[?<options>][#<field>] from any store.Required. |
For example, to use as default store an encrypted file vault.bin in the configuration folder (see Vault: Encrypted file):
ascli config preset set GLOBAL vault @json:'{"default":"local","stores":{"local":{"type":"file","file":"vault.bin"}}}'
Parameters of a store, with type (see sections below):
| Field | Type | Description |
|---|---|---|
| type | String |
vault: IBM HashiCorp Vault secret store. Labels of config vault and @vault:<label>.<field> are in mount secret (KV v2). Reference @vault://<store>/<mount>/<path>#<field> (field required) in any KV v2 mount, options version=<n>, mount=<mount> (when the mount contains /). |
allowed_prefixes |
Array[String] |
Allowed prefixes of <mount>/<path> of references: a prefix ending with / is a folder, else a path or a folder. All paths are allowed if not set. |
approle |
Hash |
AppRole authentication: the token is obtained by login, renewed, and obtained again when refused. |
approle.mount |
String |
Mount path of the AppRole auth method. Default: approle. |
approle.role_id |
String |
Role ID. Required. |
approle.secret_id |
String |
Secret ID (if the role requires one): the value, or @file:<path>. |
ca_file |
String |
File with the CA certificates that verify the certificate of the server (HTTPS). |
namespace |
String |
Namespace (Vault Enterprise). |
token |
String |
The authentication token. Its value, or @file:<path> (e.g. written and renewed by Vault Agent), read again when modified. One of token or approle is required. For the default store, defaults to the value of vault_password, unless approle is set.Example: <TOKEN>. |
type |
Enum[String] |
Vault type selector. Required. Allowed values: vault. |
url |
String |
URL of the HashiCorp Vault server. Required. Example: http://127.0.0.1:8200. |
| type | String |
system: macOS system keychain secret store. |
name |
String |
Name of the keychain to use (file <name>.keychain-db in the keychain search list).Default: ascli. |
type |
Enum[String] |
Vault type selector. Required. Allowed values: system. |
| type | String |
secret_service: Linux Secret Service secret store (GNOME Keyring, KWallet, KeePassXC...), using secret-tool. |
name |
String |
Name of the group of secrets (attribute service of the items).Default: ascli. |
type |
Enum[String] |
Vault type selector. Required. Allowed values: secret_service. |
| type | String |
file: Encrypted file secret store. Reference @vault://<store>/<label>[#<field>], field password by default. |
file |
String |
Path to the encrypted vault file, created when the first secret is added. Relative paths are resolved from the configuration folder. Read when used, and again when modified. Default: vault.bin. |
name |
String |
Same as file (compatibility). |
password |
String |
Password of the vault file: the value, or @file:<path>, read again when modified. For the default store, defaults to the value of vault_password. |
type |
Enum[String] |
Vault type selector. Required. Allowed values: file. |
| type | String |
1password: 1Password secret store (Connect REST API or CLI). |
account |
String |
Account shorthand for op (used when source is cli). Uses the default account if omitted. |
source |
Enum[String] |
Backend source. Use api for the Connect REST API, cli for the op CLI.Required. Allowed values: api, cli.Default: api. |
token |
String |
Bearer token for the Connect API (required when source is api). For the default store, defaults to the value of vault_password. |
type |
Enum[String] |
Vault type selector. Required. Allowed values: 1password. |
url |
String |
Base URL of the 1Password Connect server (required when source is api).Example: http://localhost:8080. |
vault |
String |
Name or ID of the 1Password vault to use with the CLI (used when source is cli). Uses the default vault if omitted. |
vault_id |
String |
ID of the 1Password vault to use (required when source is api). |
| type | String |
keeper: Keeper Secrets Manager secret store (Keeper Vault), using gem keeper_secrets_manager. A secret is a record shared with the application: its label is its title or its UID. Reference @vault://<store>/<title or uid>[#<field>], field password by default: standard field by type (login, password, url...), custom field by label, or notes. |
config |
String |
Configuration of the client device (base64 or JSON, e.g. from ksm profile export): the value, or @file:<path>. For the default store, defaults to the value of vault_password. |
folder_uid |
String |
UID of the shared folder where config vault create adds records (not needed to read). |
type |
Enum[String] |
Vault type selector. Required. Allowed values: keeper. |
Note
The parameters of one store, without stores (for example, {"type":"file","name":"vault.bin"}), are deprecated: they define a store named after its type (for example, file), which is the default store.
Set the store in stores, and its name in default instead: {"default":"local","stores":{"local":{"type":"file","file":"vault.bin"}}}.
Secrets used to access a store (token, approle.secret_id, password) are the value, or @file:<path> (for example, a credential of systemd, or a token written by Vault Agent): the file is read when used, and again when it is modified.
As for extended value @file:, reading the file is restricted by option security (see Security Level).
In a YAML file, quote the value: @ cannot start a plain scalar.
The secret of the default store (password of type file, token of type vault without approle, of type 1password with source api, and config of type keeper), if not in its parameters, is the value of option vault_password.
To avoid exposing the vault password in the shell history, provide it via an environment variable:
export ASCLI_VAULT_PASSWORD=<YOUR_PASSWORD>
Tip
Set it interactively using read so that it is not recorded in the shell history:
read -rs ASCLI_VAULT_PASSWORD && export ASCLI_VAULT_PASSWORD
Vault: IBM HashiCorp Vault🔗
https://developer.hashicorp.com/vault
Quick start macOS:
brew tap hashicorp/tap
brew install hashicorp/tap/vault
vault server -dev -dev-root-token-id=<TOKEN>
--vault=@json:'{"default":"corp","stores":{"corp":{"type":"vault","url":"http://127.0.0.1:8200"}}}' --vault-password=<TOKEN>
With a token in a file (for example, written and renewed by Vault Agent), read again when the file is modified:
--vault=@json:'{"default":"corp","stores":{"corp":{"type":"vault","url":"https://vault.example.com:8200","token":"@file:/run/vault/token"}}}'
With AppRole authentication, the token is obtained by login, and renewed:
--vault=@json:'{"default":"corp","stores":{"corp":{"type":"vault","url":"https://vault.example.com:8200","approle":{"role_id":"<ROLE_ID>","secret_id":"@file:/etc/vault/secret_id"}}}}'
Vault: System keychain🔗
Warning
macOS only
It is possible to manage secrets in macOS keychain.
Commands list, ids and delete are not supported: use the Keychain Access app.
--vault=@json:'{"default":"keychain","stores":{"keychain":{"type":"system","name":"ascli"}}}'
Vault: Linux Secret Service🔗
Warning
Linux only
It is possible to manage secrets in the Linux Secret Service (GNOME Keyring, KWallet, KeePassXC...), using the secret-tool command of libsecret (for example, package libsecret-tools on Debian and Ubuntu).
A session with an unlocked keyring is required.
Secrets are grouped with attribute service, which is parameter name (default: ascli).
--vault=@json:'{"default":"keyring","stores":{"keyring":{"type":"secret_service","name":"ascli"}}}'
Vault: Encrypted file🔗
It is possible to store and use secrets encrypted in a file, with a store of type file:
{"default":"local","stores":{"local":{"type":"file","file":"vault.bin"}}}
To configure the encrypted file vault as the default store, set the vault option in the global preset:
ascli config preset set GLOBAL vault @json:'{"default":"local","stores":{"local":{"type":"file","file":"vault.bin"}}}'
The vault file is created automatically when the first secret is added - no explicit initialization is needed. It is read when used, and again when it is modified.
Warning
The vault_password option should not be stored in the config file: doing so would protect secrets with a password that is itself stored in plain text, defeating the purpose of the vault.
Use an environment variable as described above, or parameter password with @file:<path> (for example, a credential of systemd).
Vault: 1Password🔗
Use type: 1password with a source parameter to select the backend.
Source: API (Connect REST API)🔗
https://developer.1password.com/docs/connect/
1Password Connect exposes a local REST API backed by a 1Password vault.
Deploy the Connect server with Docker, then point ascli at it:
docker run -d --name op-connect \
-p 8080:8080 \
-v /path/to/1password-credentials.json:/home/opuser/.op/1password-credentials.json \
1password/connect-api:latest
--vault=@json:'{"default":"op","stores":{"op":{"type":"1password","source":"api","url":"http://localhost:8080","vault_id":"<VAULT_ID>"}}}' \
--vault-password=<CONNECT_TOKEN>
Tip
The vault_id is the alphanumeric ID shown in the 1Password web app under Vault Settings.
The vault_password holds the Connect bearer token generated when creating the Connect server credentials.
Note
The 1Password Service Accounts API is also supported: use url=https://api.1password.com with a service account token as vault_password.
No local server is required in that case.
Source: Cli (op CLI)🔗
https://developer.1password.com/docs/cli/
Requires the op CLI to be installed and signed in.
No server to deploy: authentication is handled by the 1Password desktop app (biometric unlock) or by op signin.
--vault=@json:'{"default":"op","stores":{"op":{"type":"1password","source":"cli"}}}'
# or, to target a specific vault and account:
--vault=@json:'{"default":"op","stores":{"op":{"type":"1password","source":"cli","vault":"<VAULT_NAME>","account":"<ACCOUNT_SHORTHAND>"}}}'
Tip
Install op on macOS with brew install 1password-cli, then connect it to the desktop app:
op signin
Referencing items🔗
An item can be referenced by its title (label) or by its 1Password ID, in @vault:<item>.<field>:
--password=@vault:<ITEM_ID>.password
--password=@vault:<ITEM_TITLE>.password
Note
If several items share the same title, use the item ID (or the vault parameter to restrict the search).
Vault: Keeper🔗
https://docs.keeper.io/keeperpam/secrets-manager/overview
Keeper Secrets Manager (KSM) gives an application access to the records of shared folders of a Keeper vault, without the credentials of a user.
It requires the optional gem keeper_secrets_manager:
gem install keeper_secrets_manager
Create an application and a client device in Keeper (see the Quick Start Guide), share the folders of secrets with the application, and get the configuration of the device (base64 or JSON, for example with ksm profile export).
Parameter config is the value, or @file:<path>:
--vault=@json:'{"default":"keeper","stores":{"keeper":{"type":"keeper","config":"@file:<KSM_CONFIG_FILE>"}}}'
If config is not set in the default store, it is the value of option vault_password.
A secret is a record of the vault: its label is its title, or its UID.
The field of a reference is password by default: standard field of the record (login, password, url...), custom field by its label, or notes.
Field username is an alias of login.
--password=@vault:<RECORD_TITLE>.password
--password=@vault://keeper/<RECORD_TITLE>#<FIELD>
To create records with config vault create (type login), set parameter folder_uid: UID of the shared folder where records are added.
Reading does not need it.
Note
If several records share the same title, use the UID.
Deleting a record with config vault delete requires that the application has edit permission.
Vault: Operations🔗
Secrets of the default store can be manipulated using the config vault command:
info: Show vault informationids: List secret labels in the vaultlist: List all secrets with full detailsoverview: List labels of all secrets with their field names, without valuesfield: List field names of a secret by label, without valuesshow: Show a secret by label (or id)create: Add a new secret to the vaultdelete: Delete a secret by label (or id)password: Set a new password for the vaultimport: Import secrets from a JSON array (supports--bulk)check: Check that a Secret Reference can be read, without its value
To add a new password entry in the vault for label <NAME>:
ascli config vault create @: label=<NAME> password=@secret:password description='for this account'
Vault: Migration between vaults🔗
To migrate all secrets from one vault backend to another (for example, from the encrypted file vault to 1Password), use vault list piped into vault import.
Note
Use list (not ids) as the source: ids returns only labels, while list returns the full secret details needed for import.
ascli config vault list --format=json --out.level=data --out.secrets=yes \
--vault=@json:'{"default":"local","stores":{"local":{"type":"file","file":"<SOURCE_VAULT_FILE>"}}}' \
--vault-password=<SOURCE_PASSWORD> | \
ascli config vault import @json:@stdin: --bulk=yes \
--vault=@json:'{"default":"op","stores":{"op":{"type":"1password","url":"<CONNECT_URL>","vault_id":"<VAULT_ID>"}}}' \
--vault-password=<CONNECT_TOKEN>
Tip
Use --out.level=data on the list command so that only the raw JSON array is written to stdout, with no table headers or status lines, and --out.secrets=yes so that secrets are not replaced by a placeholder.
The import command accepts a JSON array where each element is a vault secret object (same schema as create).
--bulk makes each entry reported individually in the result table; omit it to get a single-line summary.
On error (for example a duplicate label), the default behavior is to stop (--bfail=yes).
Pass --bfail=no to skip failures and continue with the remaining entries.
Vault: Secret references🔗
Secrets can also be read by reference, from any store of option vault:
@vault://<store>/<path>[?<options>][#<field>]
<store>: name of a store instoresof optionvault.<path>: path of the secret in the store, and<field>: field of the secret. Both are percent-decoded (for example,%20for a space). For most types, the path is the label, and the field ispasswordby default. For typevault, the path is<mount>/<path>in a KV v2 secrets engine, and the field is required.<options>: options of the store, as a URL query (for example,version=2).
References only read secrets: they are managed with the tools of the store (for example, vault kv put for HashiCorp Vault), or with config vault for the default store.
Compared to @vault:<label>.<field>, several stores can be used in the same command, and a HashiCorp Vault store reads any KV v2 mount, a given version, and limits the paths that can be read by reference (allowed_prefixes).
Values read by reference are kept in memory for cache_ttl seconds.
The configuration of stores is the same as in applications that use the library with references secret://<store>/<path> (for example, IBM Aspera Orchestrator).
For example, a HashiCorp Vault server with AppRole authentication, limited to secrets under kv/aspera/, and the encrypted file of config vault as default store, unlocked with option vault_password:
--vault=@json:'{"default":"local","stores":{"corp":{"type":"vault","url":"https://vault.example.com:8200","approle":{"role_id":"<ROLE_ID>","secret_id":"@file:/etc/vault/secret_id"},"allowed_prefixes":["kv/aspera/"]},"local":{"type":"file","file":"vault.bin"}}}'
References in a preset: to field password of secret aspera/faspex in mount kv, and to field password (default field of type file) of label <VAULT_LABEL> in the encrypted file:
ascli config preset update <PRESET_NAME> --url=... --username=... --password=@val:@vault://corp/kv/aspera/faspex#password
ascli config preset update <PRESET_NAME> --url=... --username=... --password=@val:@vault://local/<VAULT_LABEL>
To check that a reference can be read, without its value (store, version, time of update):
ascli config vault check @vault://corp/kv/aspera/faspex#password
Configuration Finder🔗
When a secret is needed by a sub command, the command can search for existing configurations in the configuration file by setting option secret to the special value PRESET.
The lookup is done by comparing the service URL and username (or access key).
Securing passwords and secrets🔗
A password can be saved in clear in an Option Preset together with other account information (URL, username, and so on). Example:
ascli config preset update <PRESET_NAME> --url=... --username=... --password=...
For more secure storage, first create a vault entry (see Vault: Operations), then refer to it in the preset:
ascli config preset update <PRESET_NAME> --url=... --username=... --password=@val:@vault:<VAULT_LABEL>.password
Note
Use @val: in front of @vault: so that the Extended Value is not evaluated.
To migrate existing clear-text passwords from presets to the vault in one step, use the secure command.
The vault must already be configured (option vault, see Secret Vault).
ascli config preset secure
This scans all presets (or a single one if a name is provided) and, for every option whose name ends with password or secret, moves the value into the vault and replaces it with a @vault: reference.
ascli config preset secure <PRESET_NAME>
When a vault is configured, the update, set, and ask preset commands automatically move secret values (options ending with password or secret) into the vault as they are written, so no manual migration is needed.
To disable this behavior for a single command, pass --vault=@none:.
Private Key🔗
Some Aspera applications allow the user to be authenticated using Public Key Cryptography:
- For SSH: Server
- For OAuth JWT: AoC, Faspex 5, faspio Gateway
It consists of using a pair of associated keys: a private key and a public key.
The same pair can be used for multiple applications.
The file containing the private key (key pair) can optionally be protected by a passphrase.
If the key is protected by a passphrase, then the passphrase is prompted when the key is used.
Some plugins support option passphrase.
Keys of type ed25519 and OpenSSH encoded keys (default on modern Linux) are supported.
See section: Private key type ed25519.
For OpenSSH encoded keys protected by a passphrase, install gem bcrypt_pbkdf (not available for JRuby):
gem install bcrypt_pbkdf
The following sample commands use the shell variable KEY_PAIR_PATH.
Set it to the desired safe location of the private key.
Typically, located in folder $HOME/.ssh or $HOME/.aspera/ascli.
For example:
KEY_PAIR_PATH=~/.aspera/ascli/<PKEY_NAME>
Several methods can be used to generate a key pair.
The format expected for keys is PEM.
If another format is used, such as DER, it can be converted to PEM, for example, using openssl.
ascli for key generation🔗
The generated key is of type RSA, by default: 4096 bit.
For convenience, the public key is also extracted with extension .pub.
Files are PEM encoded.
The key is not passphrase protected.
ascli config genkey ${KEY_PAIR_PATH} 4096
To display the public key of a private key:
ascli config pubkey @file:${KEY_PAIR_PATH}
ssh-keygen🔗
Both private and public keys are generated, option -N is for passphrase.
ssh-keygen -t rsa -b 4096 -m PEM -N '' -f ${KEY_PAIR_PATH}
openssl🔗
To generate a key pair with a passphrase, the following can be used on any system:
openssl genrsa -passout pass:<PASSPHRASE> -out ${KEY_PAIR_PATH} 4096
openssl rsa -pubout -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.pub
openssl is sometimes compiled to support option -nodes (no DES, that is, no passphrase, for example, on macOS).
In that case, add option -nodes instead of -passout pass:<PASSPHRASE> to generate a key without passphrase.
If option -nodes is not available, the passphrase can be removed using this method:
openssl rsa -passin pass:<PASSPHRASE> -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.no_des
mv ${KEY_PAIR_PATH}.no_des ${KEY_PAIR_PATH}
To change (or add) the passphrase for a key:
openssl rsa -des3 -in ${KEY_PAIR_PATH} -out ${KEY_PAIR_PATH}.with_des
mv ${KEY_PAIR_PATH}.with_des ${KEY_PAIR_PATH}
Using an application to generate a key pair🔗
Many applications are available, including on the internet, to generate key pairs. For example: https://cryptotools.net/rsagen
Warning
Private keys are sensitive information, and shall be kept secret (like a password), so using online tools is risky.
Web service🔗
ascli is a command-line tool: it runs in a shell, with the credentials and presets of the user who starts it, and it assumes that only this user uses it.
It is not intended to be a public-facing web server.
When commands come from another program, for example an MCP client, see Security Level.
Some commands start a web server, for a specific client:
| Command | Client | Protection required when url is not loopback |
|---|---|---|
mcp server, http transport |
MCP client | auth_token, see MCP Security |
faspex5 postprocessing |
Faspex 5 web hooks | secret_token or allowed_ips, see Faspex 4-style post-processing |
faspex5 gateway, aoc gateway |
Faspex 4 client applications | username and password, see Faspex 4 Gateway |
node emulator |
Node API clients, for tests | username and password, see Node emulator |
By default, the server listens on a loopback address: localhost, an address of 127.0.0.0/8, or ::1.
It can be reached only from the same host.
Warning
Loopback restricts the host, not the user: on a host shared with other users, they can also reach the server. On such a host, also set the protection of the command.
When the host of url is not a loopback address, for example 0.0.0.0 (all interfaces), the server can be reached from the network: the command refuses to start without its protection.
On a network, use HTTPS (https:// in url) or a reverse proxy with TLS: with HTTP, passwords and tokens are sent in clear.
Other features listen on a loopback address only, to receive input from the browser of the user:
- Web login flow: the result of the login, on
redirect_uri. @secret:, with optionuiset tographical(see Secret Vault), or withmcp serverand thestdiotransport (see Secrets entered in a web page): the secret entered by the user.
The server serves HTTP, or HTTPS with a certificate. The following parameters are supported:
| Parameter | Type | Default | Description |
|---|---|---|---|
url |
String |
http://localhost:8080 |
Base URL on which requests are received; a path can be included. |
cert |
String |
- | (HTTPS) Path to certificate file (with ext. .pfx or .p12 for PKCS12). |
key |
String |
- | (HTTPS) Path to private key file (PEM), or passphrase for PKCS12. |
chain |
String |
- | (HTTPS) Path to certificate chain (PEM only). |
Parameter url (base URL) defines:
- If
httporhttpsis used - The local port number (default 443 for HTTPS, 80 for HTTP)
- The base path, that is, the path under which requests are received (useful for routing when a reverse proxy is used).
A web page open in a browser can send requests to a local web server, also on localhost.
To refuse them, the server checks each request:
- When
urlis a loopback address (localhost,127.0.0.0/8,::1), headerHostshall be a loopback name or address, else HTTP 403: a host name that resolves to127.0.0.1is refused (DNS rebinding). - Header
Origin, sent by browsers, shall be the origin of the server, else HTTP 403. - A JSON body shall have
Content-Type: application/json, else HTTP 415: a web page cannot send it to another site without the agreement of that site.
Clients that are not browsers send no Origin, and the Host of the URL they call.
They shall send the type of a JSON body: for example, with curl, add -H 'Content-Type: application/json'.
Image and video thumbnails🔗
ascli can display thumbnails for images and videos in the terminal, using iTerm2 inline image protocol or through colorized text.
This feature can be used:
- In the
thumbnailcommand ofnodewhen using gen4/access key API. - When using the
showcommand ofpreviewplugin. coffeeandimagecommands ofconfigplugin.- Any displayed value which is a URL to image can be displayed with option
formatset toimage
The following options can be specified in option --out.img:
| Field | Type | Description |
|---|---|---|
double |
Bool |
Use double vertical resolution via half-block characters (Unicode). Default is true if the terminal supports Unicode (see utf8). |
font_ratio |
Number |
Terminal font aspect ratio (height divided by width). Default: 2.3. |
reserve |
Integer |
Number of terminal lines reserved for non-image output. Default: 3. |
text |
Bool |
Display the image as colorized text instead of using the iTerm2 inline protocol. Default: false. |
Examples:
- Display image as colorized text (requires
rmagick)
ascli config image https://eudemo.asperademo.com/wallpaper.jpg --ui=text --out.img.text=true
- Display image from byte stream as image in terminal (requires iTerm2-compatible terminal)
curl -so - https://eudemo.asperademo.com/wallpaper.jpg | ascli config image @stdin:bin
- Display image from file (requires iTerm2-compatible terminal)
ascli config image @stdin:bin < A-team.jpg
Graphical Interactions: Browser and Text Editor🔗
Some actions may require the use of a graphical tool:
- A browser for Aspera on Cloud authentication (web auth method)
- A browser to enter a secret requested with
@secret: - A text editor for editing the configuration file
By default, ascli assumes that a graphical environment is available on Windows and macOS; on other systems, it relies on the presence of the DISPLAY environment variable.
The composite option ui (use --ui=help to display this schema) selects the mode, the way a web login is received, and the browser command:
| Field | Type | Description |
|---|---|---|
command |
Array[String] |
Command that opens a URL, instead of the system default. First item is the executable (full path, or name of an executable found in PATH), then its arguments. The URL is added as last argument. When set, it is used to open URLs, whatever the mode. |
mode |
Enum[String] |
- auto: browser on Windows and macOS, or if env var DISPLAY is set, else terminal.- terminal: URLs to open are displayed on terminal, and secrets are entered on terminal.- browser: a browser opens URLs, and secrets are entered on a web page.text is an alias for terminal, and graphical is an alias for browser.Files to edit are always opened with a text editor, whatever the mode.Allowed values: auto, terminal, browser, text, graphical.Default: auto. |
web_auth |
Enum[String] |
How the result of a web login (--auth=web, SSO) is received.- port: ascli listens on the loopback port of redirect_uri, the browser is redirected to it.- paste: no port is opened. After login, the browser shows a connection error on redirect_uri: copy the full URL from the address bar of the browser and paste it in the terminal. Use it when the browser cannot reach the host that runs ascli and a SSH tunnel is not possible. For shares, Shares displays a code (a long text of letters, digits, - and _) instead: paste it in the terminal.Allowed values: port, paste.Default: port. |
A plain value is the mode:
--ui=browser(orgraphical) forces a graphical environment: a browser is opened for URLs or secrets.--ui=terminal(ortext) forces a text environment: the URL to open is displayed on terminal, and secrets are entered on terminal.--ui=autoselects the mode from the environment (default).
The browser is opened, by priority, with: --ui.command, or the system default.
The system default is opened with the following command:
| OS | Command |
|---|---|
| macOS | open <url> |
| Windows | start explorer "<url>" |
| Linux | xdg-open <url> |
To edit a file (for example config open), whatever the mode, the editor is, by priority: the command in the environment variable EDITOR, else notepad.exe on Windows, else the system default above (or --ui.command if defined).
When a command is defined, it is used to open URLs whatever the mode.
The command is an array: the URL is appended as last argument.
The executable is either a full path, or the name of an executable found in the PATH.
For example, to use Firefox:
ascli config echo @secret: --ui=browser --ui.command=@json:'["firefox","--private-window"]'
Web login flow🔗
Authentication methods that need a user login in a browser (--auth=web, including for Shares) all work the same way: ascli listens on a local port, the browser is redirected to it after login.
ascligenerates a randomstateand starts a local HTTP server on the loopback address ofredirect_uri(default:http://localhost:12345).ascliopens the login URL in the browser (see Graphical Interactions): the authorization endpoint withredirect_uriandstateforweb, or, for Shares, the SAML login page withRelayState. If no graphical environment is available, the URL is displayed, to be opened on any browser of the same host.- The user logs in on the server, possibly redirected to an identity provider.
- The server redirects the browser to
redirect_uri, with the OAuth code (web), or, for Shares, the tokens, and thestate. - The browser sends this request to
ascli, which checks thatstateis the one it generated, and stops listening. asclirequests the token to the server: exchange of the code (web) or, for Shares, of the refresh token, and saves it in the cache. Next calls use the cached token, renewed if needed without a browser.
The code or tokens never go to another host than the local one: the host of redirect_uri shall be loopback.
Option ui.web_auth🔗
Option ui.web_auth (part of the composite option ui, see Graphical Interactions) selects how the result of a web login is received:
| Value | Local port | Use when |
|---|---|---|
port (default) |
Yes: ascli listens on the loopback address of redirect_uri |
The browser runs on the same system as ascli, or an SSH tunnel is possible. |
paste |
No: redirect_uri is only used in the login URL |
The browser cannot reach the system that runs ascli, and no tunnel is possible. |
With paste, after login the user gives the result to ascli through the terminal (nothing is displayed, as it contains a code or tokens):
- the code displayed by Shares (a long text of letters, digits,
-and_), or the code shown byascli config web_loginon the workstation, - or the full URL (or only the query) from the address bar of the browser, which shows a connection error on
redirect_uri: this is expected.
In all cases the state is checked.
Details are in the following sections.
Example:
ascli <PLUGIN> <COMMAND> --ui=terminal --ui.web_auth=paste
Web login when the browser is on another system🔗
The browser sends the result of the login to redirect_uri, on the loopback address of the system where the browser runs.
If ascli runs on another system than the browser (for example, ascli is installed on a server, and the user is connected with SSH from a workstation), the redirection does not reach ascli.
To solve this, forward the port of redirect_uri from the workstation to the server with an SSH tunnel, and open the login URL in the browser of the workstation:
- On the workstation, open the SSH session with a tunnel (same port on both sides, default
12345):
ssh -L 127.0.0.1:12345:127.0.0.1:12345 <SERVER_USER>@<SERVER_HOST>
- In this session, run the command with
--ui=terminal, so that the login URL is displayed instead of opening a browser on the server (this is the default if the server has no graphical environment):
ascli <PLUGIN> <COMMAND> --ui=terminal
- Copy the displayed URL in the browser of the workstation, and log in.
- The browser is redirected to
http://localhost:12345: the tunnel sends the request toasclion the server, and the flow continues as described above.
The tunnel only exists as long as the SSH session is open.
If no tunnel is possible (for example, a jump server between the workstation and the server), use --ui.web_auth=paste: ascli then opens no local port, and the result of the login is given through the terminal.
There are two ways, depending on whether ascli is also installed on the workstation.
With --ui.web_auth=port (default), in an SSH session with --ui=terminal, a warning reminds this option.
With ascli on the workstation🔗
ascli on the workstation opens the login page, receives the redirection of the browser, and shows a code to paste in the terminal of the server.
Nothing else is needed on the workstation: no configuration, no preset, no access to the server of the application.
The exchange of the code for a token is done by ascli on the server: the secret of the client does not leave it.
- In the SSH session, run the command with
--ui=terminal --ui.web_auth=paste:
ascli <PLUGIN> <COMMAND> --ui=terminal --ui.web_auth=paste
asclidisplays a message that proposes the two ways to log in, with the login URL for the first one, and a command to run on the workstation for the second one: it contains a code made of letters, digits,-and_. Thenascliwaits:
USER ACTION: To continue, choose one of these two ways:
1. With any browser (ascli is not needed on the workstation):
- Open this URL in a browser and log in:
https://...
- The browser shows a connection error on http://localhost:12345: this is expected.
- Copy the full URL from the address bar of the browser, and paste it in the prompt below.
2. If ascli is installed on the workstation that has the browser:
- Run there:
ascli config web_login <CODE>
- Log in in the page that opens.
- Copy the code shown by the browser, and paste it in the prompt below.
The command is displayed only if ascli does not start the browser itself (--ui=terminal without --ui.command).
If the browser is started by ascli (--ui=graphical, or --ui.command), it is on the same host as ascli, and the redirection may be immediate (user already logged in): only the steps to log in, copy the URL from the browser and paste it are displayed.
- On the workstation, run this command.
The first time for a login host,
asclidisplays the host of the login page, and asks to trust it (yes): it opens a URL that comes from another system. Trusted hosts are saved; without a terminal, an unknown host is refused.asclithen opens the browser on the login page and listens onredirect_uri. - Log in. The browser shows a page "Login Complete" with the code to paste, hidden by default:
- The button Copy puts the code in the clipboard.
- The eye button shows or hides the code, to select it manually.
- Paste the code in the terminal of the server (nothing is displayed).
asclion the server checks thestate, and the flow continues as described above, from step 6.
The command on the workstation:
- accepts only a login page in
https, and aredirect_urion a loopback address; the host shown for confirmation is the one of the URL, not a label given in the code; - is available only at security level
low; - ends once the browser has received the page with the code; the port of
redirect_urishall be free on the workstation (the URL is registered for the login and cannot be changed); - gives to the server only the parameters it needs (the
codeand thestate).
The codes are compressed and encrypted with a key that is not secret (derived from a constant and the current time, in slots of 15 minutes): they are hard to read and are valid for 15 to 30 minutes (the clocks of both computers must be correct), but they are not confidential: do not keep them, nor send them. The code shown by Shares also uses the random state of the login in its key.
The codes are authenticated: a code that is incomplete, mistyped or modified is detected.
Spaces and line breaks are ignored (letter case matters): a code displayed on several lines can be pasted.
Without ascli on the workstation🔗
If ascli is not installed on the workstation, the URL of the error page is used:
- In the SSH session, run the command with
--ui=terminal --ui.web_auth=paste. - Copy the displayed login URL in the browser of the workstation, and log in.
- The browser is redirected to
redirect_uri(for examplehttp://localhost:12345/?code=...&state=...) and shows a connection error: this is expected. - Copy the full URL from the address bar of the browser, and paste it in the terminal (nothing is displayed, as it contains a code or tokens).
The query alone (
code=...&state=...) is also accepted. Thestateis mandatory: it is checked like in the standard flow, then the flow continues from step 6.
Both ways accept, at the prompt, the code, the URL or the query. If standard input is not a terminal, the line is read from it. The size of a pasted line may be limited by the terminal: the code of the reply contains only what is needed.
Notes:
- For Shares with
--auth=web,pasteis simpler: Shares itself displays the code, so neither the tunnel, norascli config web_login, nor the URL of the error page is used (see Authentication of SAML users). - The port must be free on the workstation.
To use another port, change it in the tunnel and in
--redirect-uri, for example--redirect-uri=http://localhost:23456and-L 127.0.0.1:23456:127.0.0.1:23456. For Aspera on Cloud, thisredirect_urishall also be declared in the API client (see API Client Registration). - The tunnel listens on
127.0.0.1: if the browser resolveslocalhostto::1, use--redirect-uri=http://127.0.0.1:12345. - On Windows,
sshis the OpenSSH client provided with the system; with PuTTY, add a local port forwarding (Connection → SSH → Tunnels) from127.0.0.1:12345to127.0.0.1:12345. - Another solution is X11 forwarding: run an X server on the workstation (for example
VcXsrvor X410 on Windows) and connect withssh -X. The browser then runs on the server and is displayed on the workstation, so no tunnel is needed. It is heavier (X server to install, slower display), and the browser is the one of the server. - Without interaction (automation), prefer a method that needs no browser, such as
--auth=jwtfor Aspera on Cloud.
Logging, Debugging🔗
The gem is equipped with traces, mainly for debugging and learning APIs.
The composite option log groups all logging parameters (use --log=help to display this schema):
| Field | Type | Description |
|---|---|---|
format |
Enum[String] |
Log line formatter. Use @ruby: for a custom lambda.Allowed values: default, caller, standard.Default: default. |
level |
Enum[String] |
Minimum log level displayed. Allowed values: trace2, trace1, debug, info, warn, error, fatal, unknown.Default: warn. |
secrets |
Bool |
Show passwords and secrets in logs. Default: false. |
type |
Enum[String] |
Log output destination. Allowed values: stderr, stdout, syslog.Default: stderr. |
Tip
Individual options log_level, log_format, logger, and log_secrets are also available as aliases for log.level, log.format, log.type, and log.secrets respectively.
log_level and log_secrets🔗
To increase debug level, use option log_level (for example, using command line --log-level=xx, env var ASCLI_LOG_LEVEL, or an Option Preset).
Note
When using the direct agent (ascp), additional transfer logs from ascp can be activated using ascp options and ascp_args, see direct.
By default, passwords and secrets are redacted from logs.
Set option log_secrets to yes to include secrets in logs.
log_format🔗
Option log_format (alias for log.format) supports a few pre-defined formatters or a custom one using @ruby:.
A custom formatter is a lambda that takes 4 arguments; see Ruby Formatter: severity, time, progname, msg.
The default formatter is:
->(s, _d, _p, m){"#{s[0..2]}#{s[-1]} #{m}\n"}
Logging examples🔗
- Display debugging log on
stdout:
ascli config preset overview --log-level=debug --logger=stdout
Or equivalently using dot-path notation:
ascli config preset overview --log.level=debug --log.type=stdout
- Log errors to
syslog:
ascli config preset overview --log-level=error --logger=syslog
Or using the composite option in a preset:
log:
level: error
type: syslog
Note
When ascli is used interactively in a shell, the shell itself will usually log executed commands in the history file (history | grep ascli).
Learning Aspera Product APIs (REST)🔗
ascli uses mainly REST APIs to interact with Aspera applications.
To get traces of execution, with dump of API calls, use argument: --log-level=debug.
To display HTTP/S traffic set option log_level to trace2: --log-level=trace2.
It will display the exact content of HTTP requests and responses.
HTTP socket parameters🔗
To ignore SSL certificate for any address/port, use option: insecure, that is, --insecure=yes.
To ignore SSL certificate for a list of specific address/port, use option ignore_certificate, set to a URL or an Array of URLs for which the certificate is ignored (only the address and port are matched), for example, --ignore-certificate=https://127.0.0.1:9092 or --ignore-certificate=@list:,https://127.0.0.1:9092,https://127.0.0.1:9093
Note
Ignoring certificate also applies to ascp WSS.
Ignoring a certificate is not recommended.
It is preferable to add the certificate to the trusted store.
When certificate validation is skipped, a warning is displayed.
To disable this warning, set the option warn_insecure to no.
HTTP connection parameters (not ascp WSS) can be adjusted using option http_options:
| Field | Type | Description |
|---|---|---|
download_partial_suffix |
String |
File suffix appended to the target filename while a download is in progress. (Rest) Default: .http_partial. |
keep_alive_timeout |
Number |
Maximum time in seconds to keep an idle persistent connection open. (Net::HTTP) Default: 2. |
open_timeout |
Number |
Maximum time in seconds to wait for a connection to be established. (Net::HTTP) Default: 60. |
read_timeout |
Number |
Maximum time in seconds to wait for data to be read from the server. (Net::HTTP) Default: 60. |
retry_max |
Integer |
Maximum number of retry attempts. (Rest) Default: 1. |
retry_on_error |
Bool |
Retry the request on any HTTP error response. (Rest) Default: false. |
retry_on_timeout |
Bool |
Retry the request on connection timeout. (Rest) Default: true. |
retry_on_unavailable |
Bool |
Retry the request on HTTP 503 Service Unavailable. (Rest) Default: true. |
retry_sleep |
Integer |
Delay in seconds between retry attempts. (Rest) Default: 4. |
ssl_options |
Array[String] |
List of OpenSSL option flags to set on the HTTP session (e.g. IGNORE_UNEXPECTED_EOF). (SSL) |
token_cache_max_age |
Integer |
Maximum age in seconds of a cached OAuth token before it is discarded. (OAuth) Default: 1800. |
token_refresh_threshold |
Integer |
Remaining token lifetime in seconds below which the token is proactively refreshed. (OAuth) Default: 120. |
user_agent |
String |
Value sent in the User-Agent HTTP request header. (Rest) |
write_timeout |
Number |
Maximum time in seconds to wait for data to be written to the server. (Net::HTTP) Default: 60. |
Time values are set in seconds and can be of type either Integer or Float.
Any additional key is forwarded as a setter to the Net::HTTP session.
For a full list, refer to the Ruby library: Net::HTTP.
Like any other option, those can be set either on command line, or in configuration file, either in a global preset or server-specific one.
Example:
ascli aoc admin package list --http-options=@json:'{"read_timeout":10.0}'
The option ssl_options corresponds to a list of options as listed in man SSL_CTX_set_options.
The default initial value is the default of Ruby as specified in openssl/ssl.rb.
Each option can be specified as a String with the same name as in the OpenSSL library by removing the prefix: SSL_OP_, or an Integer (for example, 0 resets to no option).
If the name appears in the list, the option is set.
If the name appears in the list prefixed with a hyphen (-), the option is cleared.
For example to enable option SSL_OP_CIPHER_SERVER_PREFERENCE, add it to the list as CIPHER_SERVER_PREFERENCE.
To disable option SSL_OP_SAFARI_ECDHE_ECDSA_BUG, add it as -SAFARI_ECDHE_ECDSA_BUG.
Example:
{"ssl_options":["CIPHER_SERVER_PREFERENCE","-SAFARI_ECDHE_ECDSA_BUG"]}
Proxy🔗
There are several types of network connections, each of them uses a different mechanism to define a (forward) proxy:
- REST calls (APIs) and HTTP Gateway
ascpWSS and Legacy Aspera HTTP/S FallbackascpSSH and UDP (Aspera FASP)
See the following sections.
Proxy for REST and HTTP Gateway🔗
REST API calls and transfers based on HTTP Gateway both use Ruby's Net::HTTP class.
See Ruby find proxy.
When Ruby HTTP is used, there are two possibilities to define an HTTP proxy to be used.
The http_proxy environment variable (lower case) can be set to the URL of the proxy (with optional credentials).
Syntax is: (http|https)://[user:password@]host:port.
For example, http://myproxy.org.net:3128.
Note
Ruby expects a URL and myproxy.org.net:3128 alone is not valid.
Credentials for proxy are optional but can also be specified:
export http_proxy=http://<USERNAME>:<PASSWORD>@proxy.example.com:3128
Option http_proxy does the same (set env var) but on command line:
ascli --http-proxy=http://<USERNAME>:<PASSWORD>@host:port ...
Alternatively, the fpac option (function for proxy auto config) can be set to a Proxy Auto Configuration (PAC) JavaScript value.
Note
Proxy credentials are not supported in PAC files.
To read the script from a URL (http:, https: and file:), use prefix: @uri:.
A minimal script can be specified to define the use of a local proxy:
ascli --fpac='function FindProxyForURL(url, host){return "PROXY localhost:3128"}' ...
The result of a PAC file can be tested with command: config proxy_check.
Example, using command line option:
ascli config proxy_check --fpac='function FindProxyForURL(url, host) {return "PROXY proxy.example.com:3128;DIRECT";}' http://example.com
proxy://proxy.example.com:3128
ascli config proxy_check --fpac=@file:./proxy.pac http://www.example.com
proxy://proxy.example.com:8080
ascli config proxy_check --fpac=@uri:http://server/proxy.pac http://www.example.com
proxy://proxy.example.com:8080
If the proxy found with the PAC requires credentials, then use option proxy_credentials with username and password provided as an Array:
ascli --proxy-credentials=@json:'["<USERNAME>","<PASSWORD>"]' ...
ascli --proxy-credentials=@list::<USERNAME>:<PASSWORD> ...
Proxy for Legacy Aspera HTTP/S Fallback🔗
Only supported with the direct agent: To specify a proxy for legacy HTTP fallback, use ascp native option -x and ascp_args: --transfer=@json:'{"ascp_args":["-x","<PROXY_URL>"]}'.
FASP proxy (forward) for transfers🔗
To specify a FASP proxy (forward), set the transfer-spec parameter: proxy (only supported with the direct agent).
For example, for an Aspera forward proxy not encrypted (HTTP) without authentication running on port 9091, the option would be:
--ts=@json:'{"proxy":"dnat://proxy.example.org:9091"}'
Or, alternatively, (prefer transfer spec like above, generally):
--transfer=@json:'{"ascp_args":["--proxy","dnat://proxy.example.org:9091"]}'
FASP configuration🔗
ascli uses one of the transfer agents to execute transfers.
By default, it uses the direct agent, which is a local ascp.
Nevertheless, ascli does not come with ascp installed.
This is the reason why it is advised to install the Aspera Transfer Daemon during installation (ascli transferd install).
By default, ascli uses the ascp binary found in well known locations, that is, typical Aspera product installation paths.
The config plugin allows finding and specifying the location of ascp.
It provides the following commands for ascp sub-command:
show: Show the path ofascpusedinfo: Show information onascpand its environmentinstall: Install the Transfer SDK (deprecated, same astransferd install)spec: List transfer spec parameters supported byascpschema: Show the JSON schema of the transfer specerrors: List knownascperrors and whether they are retry-ableproducts list: List Aspera transfer products available locally
Selection of ascp location for direct agent🔗
Option: sdk_folder is used to specify the location of ascp.
By default, ascli uses ascp from the Transfer SDK installed in its configuration folder (see config ascp install).
To override and use an alternate ascp path, use option sdk_folder (--sdk-folder=).
For a permanent change, set a global default.
For example, <INSTALL_DIR> could be ~/my_install_dir on Linux, or C:\Users\admin\.aspera\ascli\sdk on Windows.
Using a POSIX shell:
ascli config preset set GLOBAL sdk_folder <INSTALL_DIR>
INFO Updated: global_common_defaults: sdk_folder <- <INSTALL_DIR>
INFO Saving config file: /home/john/.aspera/ascli/config.yaml
If the path has spaces, read section: Shell and Command line parsing.
A special value product:<PRODUCT_NAME> can be used for option sdk_folder.
It specifies to use ascp from the given product name.
A special value for product name is FIRST, which means: use the first product found in the internal list.
In that case, other files (SSH keys, aspera.conf, transferd) are still taken from the default SDK folder.
Locally installed Aspera products can be listed with:
ascli config ascp products list
╭───────────────────────────────────────┬──────────────────────────────────────╮
│ name │ app_root │
╞═══════════════════════════════════════╪══════════════════════════════════════╡
│ IBM Aspera Transfer SDK │ /Users/laurent/.aspera/sdk │
│ IBM Aspera for Desktop │ /Applications/IBM Aspera.app │
│ IBM Aspera Connect │ /Applications/IBM Aspera Connect.app │
│ IBM Aspera High-Speed Transfer Server │ /Library/Aspera │
╰───────────────────────────────────────┴──────────────────────────────────────╯
To permanently use the ascp of a product:
ascli config preset set GLOBAL sdk_folder 'product:IBM Aspera Connect'
INFO Updated: default: config <- global_common_defaults
INFO Updated: global_common_defaults: sdk_folder <- product:IBM Aspera Connect
INFO Saving config file: /home/john/.aspera/ascli/config.yaml
To show the path of currently used ascp:
ascli config ascp show
/Users/laurent/.aspera/sdk/ascp
ascli config ascp info
╭─────────┬──────────────────────────────────────────────────╮
│ field │ value │
╞═════════╪══════════════════════════════════════════════════╡
│ ascp │ /Users/john/.aspera/sdk/ascp │
...
╰─────────┴──────────────────────────────────────────────────╯
Transfer Clients: Agents🔗
Some actions on Aspera Applications lead to file transfers (upload and download) using the FASP protocol (ascp).
Transfers will be executed by a transfer client, here called Transfer Agent.
The following agents are supported and selected with option transfer:
transfer |
location | Description of agent |
|---|---|---|
direct |
local | direct execution of ascp |
transferd |
local | Aspera Transfer Daemon |
connect |
local | Aspera Connect Client (to be deprecated in 2027, use desktop) |
desktop |
local | Aspera for Desktop |
node |
remote | Aspera Transfer Node |
httpgw |
remote | Aspera HTTP Gateway |
The list of agents and their short identifier can be displayed with:
ascli config agents list
To display the configurable parameters of a specific agent (with required fields, default values, enumerated values):
ascli config agents parameters <agent name>
For example:
ascli config agents parameters direct
ascli config agents parameters node
To display the description of an agent:
ascli config agents show <agent name>
Note
All transfer operations are seen from the point of view of the agent. For example, an agent executing an upload, or package send operation will effectively push files to the related server from the system where the agent runs.
All above agents (including direct) receive transfer parameters as a transfer-spec.
Parameters in transfer-spec can be modified with option ts.
Specific options for agents are provided with option transfer (as a Hash, optionally with an agent key).
Note
Parameters in transfer are specific for each agent type and are described in the agents respective sections.
Asynchronous Transfer Mode🔗
Adding asynchronous: true to the transfer option makes ascli return immediately after
submitting the transfer, without waiting for completion.
The returned job_id can be used at any time to check progress:
# Start a download and return immediately
ascli server download /remote/big-file.bin --to-folder=/tmp \
'--transfer=@json:{"agent":"desktop","asynchronous":true}'
# Poll status later
ascli config agents jobs status <job_id>
# List all pending/completed async transfers
ascli config agents jobs list
# Remove completed/failed entries
ascli config agents jobs cleanup
Note
config transfer (list, status, cleanup) is deprecated after 4.28.0: use config agents jobs. It still works, with a warning.
All transfer agents support asynchronous mode:
| Agent | Status source | Persisted across restart |
|---|---|---|
desktop |
JSON-RPC get_transfer (auto-discovered URL) |
Yes |
node |
REST ops/transfers/{id} |
Yes |
connect |
REST transfers/info/{id} (auto-discovered URL) |
Yes |
transferd |
gRPC monitor_transfers |
Yes |
direct |
In-process thread state (re-queryable while process lives, e.g. MCP mode) | No: returns unknown after restart |
httpgw |
In-process thread state (re-queryable while process lives, e.g. MCP mode) | No: returns unknown after restart |
For direct and httpgw, the transfer runs as a Ruby thread inside the ascli process.
The job_id is persisted on disk but the live thread state is only available as long as the same process is running.
If the process is restarted, config agents jobs status returns unknown for those jobs.
Parameters of the agent are persisted, except secrets (e.g. password of agent node): config agents jobs status takes them from the current transfer option, if it is for the same agent (e.g. same option on command line, or in a preset).
Note
asynchronous and MCP - When ascli is used as an MCP server, an AI assistant calling
a transfer command may time out or cancel the request and retry, causing duplicate transfers.
Using asynchronous: true eliminates this risk: the transfer is submitted immediately,
the job_id is returned, and the AI can poll
config agents jobs status <job_id> to track progress without any risk of duplication.
This works with all agents, including direct and httpgw.
Agent: Direct🔗
The direct agent directly executes a local ascp in ascli.
This is the default agent for ascli (option --transfer=direct).
ascli will locally search installed Aspera products, including SDK, and use ascp from that component.
See FASP.
Agent: Direct: transfer🔗
The transfer option accepts the following optional parameters to control multi-session, Web Socket Session, Resume policy and add any argument to ascp:
| Field | Type | Description |
|---|---|---|
ascp_args |
Array[String] |
List of native ascp command-line arguments.Default: []. |
client_ssh_key |
Enum[String] |
SSH key type to use for token-based transfers. Allowed values: rsa, dsa_rsa, per_client.Default: rsa. |
file_list |
Bool |
If true, source paths are written to a temp file passed to ascp via --file-list or --file-pair-list.If false, source paths are placed directly on the ascp command line.Default: true. |
monitor |
Bool |
Enable use of the ascp management port for transfer monitoring.Default: true. |
multi_incr_udp |
Bool |
Multi session - Increment UDP port for each session. If true, each session uses a different UDP port starting at fasp_port (default: 33001).If false, all sessions use the same fasp_port (or ascp default).Default: true on Windows, false on other platforms. |
quiet |
Bool |
Suppress the ascp progress bar display.If false, the progress bar of option progress_bar is not displayed, unless that option is set.Default: true. |
resume |
Hash |
Configuration for automatic transfer resume on interruption. |
resume.iter_max |
Integer |
Maximum number of retry attempts on error. Default: 7. |
resume.sleep_factor |
Integer |
Multiplier applied to sleep duration between consecutive retry attempts. Default: 2. |
resume.sleep_initial |
Integer |
Initial sleep duration (in seconds) before first retry. Default: 2. |
resume.sleep_max |
Integer |
Maximum sleep duration (in seconds) between retry attempts. Default: 60. |
spawn_delay_sec |
Number |
Multi session - Delay (in seconds) between starting each ascp session.Default: 2. |
spawn_timeout_sec |
Number |
Multi session - Maximum time (in seconds) to verify that ascp is running.Default: 2. |
trusted_certs |
Array[String] |
List of trusted certificate repositories. |
wss |
Bool |
Enable Web Socket Session when available. Default: true. |
In case of transfer interruption, the agent will resume a transfer up to iter_max times.
Sleep between iterations is given by the following formula where iter_index is the current iteration index, starting at 0:
min( sleep_max, sleep_initial * sleep_factor ^ iter_index )
By default, the progress bar of ascli is displayed (see Transfer progress bar): it aggregates all sessions of a multi-session transfer.
To display the native progress bar of ascp instead, set parameter quiet to false:
--transfer.quiet=false
In that case, the progress bar of ascli is not displayed, unless option progress_bar is set.
For multi-session transfers, each ascp process displays its own progress bar.
To skip usage of management port (which disables the progress bar of ascli), set option monitor to false.
By default, Ruby's root CA store is used to validate any HTTPS endpoint used by ascp (for example, WSS).
To use a custom certificate store, use the trusted_certs option of direct agent's option transfer.
To use ascp's default, use option:
--transfer.trusted_certs=@none:
Some transfer errors are considered retry-able (for example, timeout) and others are not (for example, wrong password). The list of known protocol errors and retry level can be listed:
ascli config ascp errors
Error codes of sessions (errcode in ascp logs) are a different numbering, listed with their category and the corresponding error above:
ascli config ascp errors session
Examples:
ascli ... --transfer.wss=true --transfer.resume.iter_max=20
ascli ... --transfer.spawn_delay_sec=2.5 --transfer.multi_incr_udp=false
Parameter ascp_args can also be used to activate logging using option -L of ascp.
For example, to activate debug level 2 for ascp (DD), and display those logs on the terminal (-):
--transfer=@json:'{"ascp_args":["-DDL-"]}'
This is useful to debug if a transfer fails.
To store ascp logs in file aspera-scp-transfer.log in a folder, use --transfer=@json:'{"ascp_args":["-L","/path/to/folder"]}'.
To analyze ascp log files (ascp 4.0 and later), give log files, or folders with log files, to config ascp logs:
ascli config ascp logs summary /path/to/folder
ascli config ascp logs sessions /path/to/folder/aspera-scp-transfer.log
| Command | Result |
|---|---|
summary |
Statistics of sessions, errors by category, maximum throughput, findings. |
health |
Health check for monitoring in Nagios format (failed sessions and low efficiency over last N minutes). |
sessions |
One row per session, with diagnostics. |
diagnostics |
One row per diagnostic of a session: id, severity (error, warning, info), message and recommendation (action to take, when the diagnostic has one). |
multi_sessions |
Multi-session transfers: sessions grouped by transfer, with aggregate rate. |
snapshots |
Aggregate rates per period. |
snapshot_sessions |
Rates of each session per period. |
peers |
Sessions and errors by peer address. |
transfers |
Transferred files. |
licenses |
Licenses of local and peer systems. |
overdrives |
Sessions with loss due to overdriving of network or storage. |
records |
Log lines parsed into records: samples, sessions or transfers (see below). |
fields |
Description of the fields of records: samples, sessions or transfers. |
Log files compressed with gzip (e.g. rotations aspera.log-20261101.gz, aspera-scp-transfer.1.log.gz) are read directly, in a folder or given as argument; gzip is detected from the first bytes of the file, not from its name. Argument - reads the logs from the standard input, plain or compressed with gzip:
ascli config ascp logs summary /var/log/aspera.log-20261101.gz
zcat /var/log/aspera.log-2026*.gz | ascli config ascp logs summary -
To analyze the logs of an HSTS read over SSH, instead of files of the local machine, use command logs of the plugin server.
A file must contain ascp log lines (FASP messages, or lines of process ascp in syslog): in a folder, other files are ignored; a file given as argument that is not an ascp log is refused.
When logs of both sides of a session are analyzed (client and server), each session includes the status, error and diagnostics of the other side (peer_side, peer_diagnostics).
For example, to list only errors:
ascli config ascp logs diagnostics /path/to/folder --select=@json:'{"severity":"error"}'
Besides the default columns, sessions rows have context columns to filter sessions (show them with --fields=ALL, or select with --fields=session,product_name,transport,authentication, or filter with --select/--query); a column is absent when the log does not give it:
| Column | Value |
|---|---|
product_name |
Product that wrote the log, from the line <product> version <version> (e.g. Aspera High-Speed Transfer Server, Aspera Transfer SDK, Aspera Connect), without IBM. |
transport |
ssh if Session Start has tcp_port, wss if it has wss_port. |
authentication |
token if ascp logs that ASPERA_AUTH_TOKEN is set, else node if it logs the storage of a node access key. Never the token, the access key or the storage root. |
proxy |
Address of the proxy (proxy_ip of Session Params), absent if there is none. |
fips |
true or false (fips_enabled of Session Params). |
Diagnostics of a session include: category of error (e.g. udp_setup, storage, authentication) with a hint, retry of a failed session, parameters set by the server (cipher, resume policy, virtual link), efficiency and loss of control messages, decline of the rate during the transfer (rate_decline: second half of the samples below 70% of the first half, not evaluated if the peer changes the rate, if another transfer session starts meanwhile, or if the session failed), and for a sender, which limit of the receiver (network, disk, virtual link) or reading from storage limits the rate. Other diagnostics: no packet received for some seconds, by the receiver (reception_interruption) or by the sender, which gets no feedback of the receiver (feedback_interruption); overflow of the retransmission timer (rex_timer_overflow: the retransmission RTT is very large); maximum rate of the license of ascp below the target rate (license_rate_limit); license that expires within license_expiry_days (default 30) of the start of the session, or has expired (license_expiry); and many small files transferred (small_files): at least small_files_min_count (default 100) files with a mean size below small_files_max_mean (default 1 MB); path MTU measured by ascp below small_mtu_max (default 1400, ascp measures 1492, 1006, 508 or 296), which suggests a VPN or a tunnel (small_mtu). Two configuration messages, which have no effect on the transfer, are not diagnostics but only appear in findings of summary: OpenSSL modules folder not found (ossl-modules not found), and a schedule of virtual link capacity that could not be registered (unable to submit change event).
The recommendation of a diagnostic is only in the list diagnostics (not in sessions, where diagnostics is a list of messages, and not in summary): it tells what to check or change, with the ascp options and aspera.conf parameters concerned (e.g. UDP port and firewalls, target and minimum rate, storage write, SSH). Diagnostics without action (e.g. information, or cancelled by the user) have no recommendation; the diagnostic of the category of a failed session gives its hint in message.
Option ascp_logs (dot-notation) sets period (snapshot period in minutes, default 1), zero_bytes (include sessions without transferred bytes), mtu (when not found in logs), resolve (host names of peers), timezone, health_minutes (window in minutes for health check command, default 60), remote_max_size (maximum total size in bytes of the log files read from a server with server logs, default 268435456), and thresholds (diagnostic thresholds, including rate_decline_ratio, rate_decline_min_samples, rate_decline_delay, rate_decline_end_fraction, license_expiry_days, small_files_min_count, small_files_max_mean and small_mtu_max), e.g. --ascp-logs.period=5 or --ascp-logs.thresholds.efficiency=95.
Times of log lines are read in the time zone of the analyst, and displayed in it. To read the log of another system without shifting it, set the time zone of the log with timezone: UTC or an offset +HH:MM, e.g. --ascp-logs.timezone=+02:00. Syslog lines (Oct 25 00:02:20) have no year: it is deduced from the modification date of the log file (the current date for the standard input), the log being before this date. So a log from December to January, analyzed in March, has its December lines in the previous year. Keep the modification date of the file when copying it (e.g. cp -p, scp -p).
Command records gives the log lines parsed into records, with the field names of the log lines, to filter them with --out.query (JMESPath):
| Record | Log lines |
|---|---|
samples |
Receiver bl or Sender bl line, with the DS Qs line that follows it in field DS. |
sessions |
Session Start, Session Params, Session Stop and Session Statistics lines of a session, and lines of the process about it (see below). |
transfers |
Transfer Start and Transfer Stop lines of a file. |
A group of a log line becomes an object: for example, rate t/m/c/n/vl/vr/r=... becomes rate.t, rate.m, rate.c, etc.
Values are numbers when the log value is a number: rates in bps, durations in ms, s or µs, and percentages without %.
Session Stop and Transfer Stop have the fields of Session Start and Transfer Start, with their final values.
When several log files are read, field log_file gives the log file of each record.
Field cookie of Session Params is never output.
A session record also has the fields of these lines of the process (ascp), when the log has them:
| Field | Log line |
|---|---|
fasp_version, license_max_rate, license_expires |
Initializing FASP version <version>, license expires on <date>, max rate=<N> Mbps, ...: license_max_rate (bps) and license_expires are absent if the license is unlimited or has no expiry date. account no. and license no. are never output. |
pMTU, start_brtt |
Measured pMTU: <N> Bytes, start_brtt: <N> ms. |
datagram_size, block_size, path_mtu |
datagram size <N>B, block size <N>B, path MTU <N>B (no path_mtu in old versions). |
peer_rate_params |
List of Peer requested rate params adopted lines, in the order of the log: time, target_rate, min_rate and transfer_policy of each change. |
For example, samples where the calculated rate is lower than the target rate, and sessions with loss of control messages from the receiver:
ascli config ascp logs records samples /path/to/aspera-scp-transfer.log --out.query='[?rate.c < rate.t].{time: time, session: uuid, calculated: rate.c, network: rate.n, lost: bl2.l}'
ascli config ascp logs records sessions /path/to/folder --out.query='[?rcvr_ctl.lost_pct > `1`]'
Command fields gives the type and description (with unit) of each field of a kind of record (samples, sessions or transfers), with its path as used in --out.query (e.g. rate.c):
ascli config ascp logs fields samples
ascli config ascp logs fields sessions
ascli config ascp logs fields transfers
Note
When transfer agent direct is used, the list of files to transfer is provided to ascp using either --file-list or --file-pair-list and a temp file list, unless --file-list or --file-pair-list is already provided via transfer parameter ascp_args.
To place source paths directly on the ascp command line instead of using a temp file, set file_list to false in transfer: --transfer.file_list=false.
In addition to standard methods described in section File List, it is also possible to specify the list of files using transfer parameter ascp_args:
--sources=@ts --transfer=@json:'{"ascp_args":["--file-list","myfilelist"]}'
Note
File lists are shown here; similar options exist for file pair lists.
Note
This method avoids the creation of a copy of the file list: if the standard options --sources=@lines:@file:... --src-type=... are used, then the file list is read and parsed, and a new file list is created in a temporary folder.
Note
This method has limitations: it applies only to the direct transfer agent (that is, local ascp) and not to Aspera on Cloud.
Agent: Direct: Management messages🔗
By default, ascli gets notifications from ascp on its management port.
This can be disabled with parameter: monitor=false of transfer.
It is also possible to send messages to ascp using this management port.
A typical use is to change the target rate of a running transfer.
The communication is done through a JSON file that shall be created in ascli's config folder as displayed with:
ascli config folder
The name of the file shall be: send_<PID>, where <PID> is the process ID of the running ascli.
If there is only one ascli running, one can get the PID like this:
ps -axo pid,command|grep ascli|grep -v grep|cut -f1 -d' '
Example to change the target rate:
echo '{"type":"RATE","rate":300000}' > ~/.aspera/ascli/send_67470
When ascli detects this file, it uses it during a transfer and then deletes it.
Note
The JSON's keys use snake case, that is, lower case with _ as word separator.
The list of message type values can be found in aspera/ascp/management.rb: OPERATIONS.
The list of parameters is PARAMETERS (native names are capitalized, keys in the JSON file are in snake case).
Agent: Direct: aspera.conf: Virtual Links🔗
This agent supports a local configuration file, aspera.conf, where Virtual links can be configured.
On a server (HSTS), the following commands can be used to set a global virtual link:
asconfigurator -x 'set_trunk_data;id,1;trunk_name,in;trunk_capacity,100000;trunk_on,true'
asconfigurator -x 'set_trunk_data;id,2;trunk_name,out;trunk_capacity,100000;trunk_on,true'
asconfigurator -x 'set_node_data;transfer_in_bandwidth_aggregate_trunk_id,1'
asconfigurator -x 'set_node_data;transfer_out_bandwidth_aggregate_trunk_id,2'
This command is not available on clients: instead, edit the file aspera.conf (its location is shown by ascli config ascp info --fields=aspera_conf) and modify the sections default and trunks like this for a global 100 Mbps virtual link (capacity is in kbps):
<?xml version='1.0' encoding='UTF-8'?>
<CONF version="2">
<default>
<transfer>
<in>
<bandwidth>
<aggregate>
<trunk_id>1</trunk_id>
</aggregate>
</bandwidth>
</in>
<out>
<bandwidth>
<aggregate>
<trunk_id>2</trunk_id>
</aggregate>
</bandwidth>
</out>
</transfer>
</default>
<trunks>
<trunk>
<id>1</id>
<name>in</name>
<on>true</on>
<capacity>
<schedule format="ranges">100000</schedule>
</capacity>
</trunk>
<trunk>
<id>2</id>
<name>out</name>
<capacity>
<schedule format="ranges">100000</schedule>
</capacity>
<on>true</on>
</trunk>
</trunks>
</CONF>
It is also possible to set a schedule with different time and days, for example for the value of schedule:
start=08 end=19 days=mon,tue,wed,thu capacity=900000;1000000
Agent: Direct: aspera.conf: File name with special characters🔗
By default, ascp replaces file system disallowed characters in transferred file names with _.
On Windows, it concerns: * : | < > " ' ?.
Replacing illegal characters in transferred file names is a built-in feature of Aspera transfers.
This behavior can be customized with the aspera.conf configuration file.
For example, to replace illegal character | with an underscore _:
- First, locate the configuration file with:
ascli config ascp info --fields=aspera_conf
Typically, it is located at $HOME/.aspera/sdk/aspera.conf
- Edit this file, and add the following line inside the XML section
CONF.default.file_system:
<replace_illegal_chars>_|</replace_illegal_chars>
The result should look like this:
<?xml version='1.0' encoding='UTF-8'?>
<CONF version="2">
<default>
<file_system>
<resume_suffix>.aspera-ckpt</resume_suffix>
<partial_file_suffix>.partial</partial_file_suffix>
<replace_illegal_chars>_|</replace_illegal_chars>
</file_system>
</default>
</CONF>
According to the documentation, the parameter works as follows:
- The first character in the value is the replacement character.
- All other characters listed after it are the illegal ones to be replaced.
So in this example: | will be replaced with _.
If the mounted target file system is Windows but ascli runs on Linux, there are several additional illegal characters to handle.
In that case, set the parameter as shown below. XML special characters such as <, >, " and ' must be escaped properly:
<replace_illegal_chars>_*:|<>"'?</replace_illegal_chars>
In this example:
_is the replacement character.- The list of characters that will be replaced is:
*:|<>"'?
So, for example:
report|final?.txtbecomesreport_final_.txtdata*backup"2025".csvbecomesdata_backup_2025_.csv
Agent: Connect Client🔗
Note: IBM Aspera Connect will be deprecated in 2027. Use IBM Aspera for Desktop instead:
--transfer=desktop.asclidisplays a warning when theconnectagent is used.
By specifying option: --transfer=connect, ascli will start transfers using the locally installed IBM Aspera Connect Client.
There are no specific parameters for option transfer.
Agent: Desktop Client🔗
By specifying option: --transfer=desktop, ascli will start transfers using the locally installed IBM Aspera Desktop Client.
There are no specific parameters for option transfer.
Agent: Node API🔗
By specifying option: --transfer=node, ascli starts transfers in an Aspera Transfer Server using the Node API, either on a local or remote node.
This is especially useful for direct node-to-node transfers.
Parameters provided in option transfer are:
| Field | Type | Description |
|---|---|---|
password |
String |
Password, secret, or bearer token. Required. |
root_id |
String |
Root file ID. Mandatory only when password is a bearer token. |
url |
String |
URL of the Node API. Required. |
username |
String |
Node API user or access key. Required. |
Like any other option, transfer can get its value from a pre-configured Option Preset:
--transfer=@preset:<PRESET_NAME> --transfer.agent=node
It can also directly use the Extended Value syntax:
--transfer=@json:'{"agent":"node","url":"https://...","username":"<USERNAME>","password":"<PASSWORD>"}'
If transfer is not specified and a default node has been configured (name in node for section default) then this node is used by default.
If the password value begins with Bearer then the username is expected to be an access key and the parameter root_id is mandatory and specifies the file ID of the top folder to use on the node using this access key.
It can be either the access key's root file ID, or any authorized file ID underneath it.
Agent: HTTP Gateway🔗
The Aspera HTTP Gateway is a service that allows sending and receiving files using HTTPS.
By specifying option: --transfer=httpgw, ascli will start transfers using the Aspera HTTP Gateway.
Parameters provided in option transfer are:
| Field | Type | Description |
|---|---|---|
api_version |
Enum[String] |
Force use of a specific API version. Allowed values: v1, v2.Default: v2. |
synchronous |
Bool |
Wait for acknowledgment after each message. Default: false. |
upload_chunk_size |
Integer |
Size in bytes of each upload chunk. Default: 64000. |
url |
String |
URL of the HTTP Gateway. Required. |
Example:
ascli faspex5 packages receive 323 --transfer=httpgw --transfer.url=https://asperagw.example.com:9443/aspera/http-gwy
Note
The gateway only supports transfers authorized with a token.
If the application, for example, AoC or Faspex 5, is configured to use the HTTP Gateway, then ascli will automatically use the gateway URL if --transfer=httpgw is specified, so transfer URL becomes optional.
Agent: Transfer Daemon🔗
Another possibility is to use the Transfer Daemon (transferd).
Set option transfer to transferd.
Options for transfer are:
| Field | Type | Description |
|---|---|---|
api_key |
String |
API key of an existing daemon (start is false) that has authentication enabled in its configuration. It is sent in the metadata api_key of each call. |
ca_file |
String |
Path to the certificate authority file (PEM) that signed the certificate of an existing daemon (start is false) that has tls enabled in its configuration. A self-signed certificate is its own authority. |
start |
Bool |
Start a new daemon instance. Default: true. |
stop |
Bool |
Stop the daemon when ascli exits.Default: true. |
url |
String |
Address and port listened by the daemon (e.g. :55002). If port is 0, the daemon listens on a random available port. If no address is given, 127.0.0.1 is used.Default: :0. |
Note
If port zero is specified in the URL, then the daemon will listen on a random available port.
If no address is specified, then 127.0.0.1 is used.
For example, to use an external, already running transferd, use option:
--transfer=@json:'{"agent":"transferd","url":":55002","start":false,"stop":false}'
The gem grpc is not part of default dependencies, as it requires compilation of a native part.
So, to use the Transfer Daemon you should install this gem:
gem install grpc
If the execution complains about incompatible libraries, then force recompilation of the native part:
gem uninstall grpc
gem install grpc --platform ruby
On Windows the compilation may fail for various reasons (3.1.1):
cannot find -lx64-ucrt-ruby310
→ copy the file [Ruby main dir]\lib\libx64-ucrt-ruby310.dll.a to (remove the dll extension) [Ruby main dir]\lib\libx64-ucrt-ruby310.a
conflicting types for 'gettimeofday'
→ edit the file [Ruby main dir]/include/ruby-[version]/ruby/win32.h and change the signature of gettimeofday to gettimeofday(struct timeval *, void *), that is, change struct timezone to void
Transfer Daemon as a system service🔗
The Transfer Daemon (transferd) can run permanently as a systemd service, for applications that use an existing daemon (see Agent: Transfer Daemon, start set to false).
ascli installs the SDK, writes the configuration file of the daemon (/etc/transferd/transferd.json) and the systemd unit, then enables and starts the service.
Commands are in transferd service (plugin transferd):
| Command | Action |
|---|---|
spec |
Show the schema of the configuration file of the daemon (transferd.json). Works on any system. |
install |
Install the SDK if it is missing, then do configure. |
configure |
Merge the given configuration into the existing file, validate it, write the file and the unit, reload systemd, enable and restart the service, and wait until the daemon listens. |
status |
Show the version, the state of the service, the endpoint and whether it listens. The API key is never shown. |
uninstall |
Stop and remove the service. With purge set to true in service, also remove the configuration file and the log folder. |
use |
Switch to an installed version of the SDK and restart the service (see below). |
versions |
List the installed versions of the SDK (see below). |
Except spec and versions, commands take the optional last positional argument CONFIG: a Hash (Extended Value) with two optional sections:
service: how the service is installed.config: content oftransferd.json.
{
"service": {
"user": "asperadaemon"
},
"config": {
"address": "127.0.0.1",
"port": 55002,
"log_directory": "/var/log/aspera-transferd",
"workers": {"max_concurrent": 10},
"authentication": {"enabled": true}
}
}
Parameters of section service:
| Field | Type | Description |
|---|---|---|
after |
Array[String] |
Units that shall be started before the daemon (After= in the unit).Default: ["network.target"]. |
environment |
Hash |
Environment variables set in the unit. None is set unless given here. TRANSFERD_ADDRESS, TRANSFERD_PORT, TRANSFERD_MGMT_PORT and TRANSFERD_MGMT_ADDRESS override the configuration file. |
group |
String |
Group that runs the daemon. Defaults to the primary group of user. |
name |
String |
Name of the service. The unit file is <name>.service.Default: aspera-transferd. |
purge |
Bool |
Used by uninstall. Also remove the configuration file and the log folder.Default: false. |
unit_dir |
String |
Folder where the unit file is created. Default: /etc/systemd/system. |
user |
String |
Account that runs the daemon. It must exist, it is not created. Default: asperadaemon. |
version |
String |
Version of the SDK to install, used by install when the SDK folder is named current (versions side by side). Defaults to the SDK version of this tool. |
Section config follows the JSON schema of the daemon, displayed with:
ascli transferd service spec
| Field | Type | Description |
|---|---|---|
$schema |
String |
Optional URI or path of this schema, used by editors for completion and validation. Ignored by transferd. |
address |
String |
IP address or host name on which the gRPC API listens. Use 0.0.0.0 to listen on all IPv4 interfaces (for example in a container); when not listening on a loopback address, consider enabling tls and authentication. Overridden by environment variable TRANSFERD_ADDRESS, and by option --address (-a), which takes precedence. Default: 127.0.0.1.Example: 127.0.0.1. |
api_time_settings |
Hash |
Keepalive and connection lifetime settings of the gRPC API server, in seconds. |
api_time_settings.max_conn_age |
Integer |
Maximum lifetime of a client connection, with a random variation of +/-10%; the client must then reconnect. 0 or omitted: no limit. Default: 0.Example: 3600. |
api_time_settings.max_conn_age_grace |
Integer |
Additional time given to ongoing calls once max_conn_age is reached, before the connection is forcibly closed. 0 or omitted: no limit. Default: 0.Example: 10. |
api_time_settings.max_conn_idle |
Integer |
Time after which a client connection with no ongoing call is closed. 0 or omitted: 86400 (24 hours). Default: 86400.Example: 120. |
api_time_settings.time |
Integer |
Time without activity from a client after which the server pings it to check that the connection is still alive. 0 or omitted: 7200 (2 hours). Default: 7200.Example: 60. |
api_time_settings.timeout |
Integer |
Time the server waits for activity after a keepalive ping before closing the connection. 0 or omitted: 20. Default: 20.Example: 3. |
authentication |
Hash |
API key authentication of clients of the gRPC API. |
authentication.api_key |
String |
Value that clients must send in gRPC request metadata entry api_key. It is stored in clear text: restrict read access to this file. Example: change-me-to-a-long-random-value. |
authentication.enabled |
Bool |
Requires every call to carry the API key; other calls are rejected with status UNAUTHENTICATED. api_key is then required. Default: false. |
fasp_management |
Hash |
Management port on which the ascp processes started by transferd report transfer progress. |
fasp_management.address |
String |
IP address on which the management server listens. Change it only when ascp must reach transferd through an interface other than loopback. Overridden by environment variable TRANSFERD_MGMT_ADDRESS, and by option --mgmtaddress (-A), which takes precedence. Default: 127.0.0.1.Example: 127.0.0.1. |
fasp_management.port |
Integer |
TCP port of the management server. 0 or omitted: a free port is selected at startup. Overridden by environment variable TRANSFERD_MGMT_PORT, and by option --mgmtport (-P), which takes precedence. Default: 0.Example: 23456. |
fasp_management.promiscuous_mode_enabled |
Bool |
When true, transferd also reports transfers of ascp processes that it did not start. So that those processes find the management port, transferd writes it in file transferd. Default: false. |
fasp_runtime |
Hash |
FASP runtime (ascp, async, license, aspera.conf) used for transfers. Omitted: the runtime embedded in the SDK is used. When this section is present, set use_embedded explicitly: if it is not true, user_defined is required. |
fasp_runtime.extra_config |
Array[Hash] |
Extra command line options added to the executables started for transfers. Does not apply to async. |
fasp_runtime.extra_config[].command |
String |
End of the path of the executable to which options are added, for example ascp (ascp.exe on Windows). Only the first matching entry is used. Required. Example: ascp. |
fasp_runtime.extra_config[].options |
Array[String] |
Options added to the command line, one argument per item, before the transfer arguments. Example: ["--save-before-overwrite"]. |
fasp_runtime.force_version |
String |
Accepted for backward compatibility. Has no effect. |
fasp_runtime.log |
Hash |
Default logging of ascp, for transfers whose request configuration does not set localLog or logLevel. Does not apply to async. |
fasp_runtime.log.dir |
String |
Directory where ascp writes its log (ascp option -L). Empty or omitted: default location of ascp. Example: /var/log/aspera. |
fasp_runtime.log.level |
Integer |
Level of detail of the ascp log: 0 for basic information (files transferred, transfer rate), 1 to 4 for increasingly detailed debug messages (ascp options -D to -DDDD). Default: 0. |
fasp_runtime.use_embedded |
Bool |
true: use the FASP runtime embedded in the SDK, and ignore user_defined. false: use the runtime located by user_defined. Omitted while the fasp_runtime section is present: false. Example: true. |
fasp_runtime.user_defined |
Hash |
Location of a separately installed FASP runtime. Required when use_embedded is not true, ignored otherwise. Both directories must exist at startup. |
fasp_runtime.user_defined.bin |
String |
Directory containing the ascp and async executables. A relative path is resolved from the working directory of transferd. Required. Example: /opt/aspera/bin. |
fasp_runtime.user_defined.etc |
String |
Directory containing the license file aspera-license and the configuration file aspera.conf. A relative path is resolved from the working directory of transferd. Required. Example: /opt/aspera/etc. |
log_directory |
String |
Directory of the transferd log file transferd.log (rotated at 10 MiB, 10 files kept). A relative path is resolved from the directory of the transferd executable. The directory is created if missing. Empty or omitted: logs are written to standard output. Example: /var/log/aspera. |
log_format |
Enum[String] |
Format of the transferd log: text for human-readable lines, json for one JSON object per line. Any value other than text gives json. Allowed values: text, json.Default: text. |
log_level |
Enum[String] |
Level of detail of the transferd log, from most (trace) to least (panic) verbose. Omitted or unrecognized value: info. Allowed values: trace, debug, info, warning, error, fatal, panic.Default: info. |
port |
Integer |
TCP port on which the gRPC API listens. 0 or omitted: a free port is selected at startup; it is written in the log and in file transferd. Default: 0.Example: 55002. |
reflection |
Bool |
Enables the gRPC server reflection service, so that tools such as grpcurl or grpcui can list and call the API without the transferd.proto file. Useful for debugging; leave disabled otherwise.Default: false. |
runtime_dir |
String |
Directory for runtime files. When port is 0, transferd writes the selected port in file transferd. Example: /var/run/aspera. |
tls |
Hash |
TLS for the gRPC API. When enabled, clients must connect using TLS. |
tls.certificate |
String |
Path of the PEM server certificate file, which may include intermediate certificates. A relative path is resolved from the working directory of transferd. Example: /etc/transferd/server.crt. |
tls.enabled |
Bool |
Enables TLS. certificate and key are then required. Default: false. |
tls.key |
String |
Path of the PEM private key file matching certificate. A relative path is resolved from the working directory of transferd. Example: /etc/transferd/server.key. |
tls.min_tls_version |
Enum[String] |
Minimum TLS version accepted from clients. Omitted or other value: TLSv12. Allowed values: TLSv12, TLSv13.Default: TLSv12. |
workers |
Hash |
Transfer execution and retention. |
workers.max_concurrent |
Integer |
Maximum number of transfers running at the same time. Additional transfers wait in a queue until a running one ends. 0 or omitted: 100. Default: 100.Example: 10. |
workers.transfer_cleaning_period |
Integer |
Interval, in seconds, at which transferd removes ended transfers older than transfer_max_age. 0 or omitted: 3600 (1 hour). Default: 3600. |
workers.transfer_max_age |
Integer |
Time, in seconds, during which an ended transfer remains known to transferd (status queries, events) before it can be removed. Removal happens at the next cleaning, so a transfer may remain up to transfer_max_age + transfer_cleaning_period. 0 or omitted: 7200 (2 hours). Default: 7200.Example: 86400. |
The configuration is checked before anything is written:
- Keys not given keep their value in the existing
transferd.json(deep merge), and a key set tonullis removed. - An unknown key, a value of a wrong type, or an inconsistent configuration (for example
tls.enabledwithoutcertificateandkey) is refused, with the name of the key and of the file if the faulty key is in the existing file. The daemon itself ignores unknown keys silently. - The section
fasp_runtimeis computed from the SDK folder, and can be overridden. - If
authentication.enabledistrueand noapi_keyis given, the key of the existing file is kept, else a random one is generated. The file is written in clear text with mode0640, owneruser:group. To get the key from the vault, write{"config":{"authentication":{"api_key":"@vault:transferd.api_key@"}}}in an Extended Value@json:@extend:'...'.
Requirements:
systemdon Linux only. On other systems, or withoutsystemctl, commands fail with a message (onlyspecand--dry-runwork everywhere).- Run as
root, except with--dry-run=yes(option is a boolean), which shows the actions and the files without any change, andspecandversions. - The account (
asperadaemonby default) is not created byascli. If it does not exist, the command fails and shows the command to create it:useradd --system --no-create-home --shell /usr/sbin/nologin asperadaemon. This is the account used by other Aspera products on the same host, so that files have the same owner. - If the version of the SDK installed is different from the version of the schema, then a warning is displayed (it is not an error).
Example:
sudo ascli transferd service install --sdk-folder=/opt/aspera/transferd/current @json:'{"config":{"port":55002,"authentication":{"enabled":true}}}'
ascli transferd service configure --dry-run=yes @json:'{"config":{"log_level":"debug"}}'
sudo ascli transferd service status
sudo ascli transferd service uninstall @json:'{"service":{"purge":true}}'
Versions side by side and rollback🔗
If the SDK folder (option sdk_folder) is named current, e.g. --sdk-folder=/opt/aspera/transferd/current, then ascli uses the layout <prefix>/<version> with a link current that points to the version in use:
/opt/aspera/transferd/1.1.9
/opt/aspera/transferd/1.1.10
/opt/aspera/transferd/current -> 1.1.10
service installinstalls the version given inservice.version(default: version tested with this release ofascli) in<prefix>/<version>, and sets the linkcurrent. If the installation fails, then the link is set back to the previous version.- The unit and
transferd.jsonrefer tocurrent: switching the link and restarting the service is enough to change the version. service versionslists the installed versions and the one in use.service use <VERSION>switches the link and restarts the service. If the service does not start, then the link is set back to the previous version and the service is restarted.
Any other folder name is used as is, with no link.
Transfer Specification🔗
Some commands lead to file transfer (upload/download). All parameters necessary for this transfer are described in a transfer-spec (Transfer Specification), such as:
- Server address
- Transfer username
- Credentials
- File list
- And so on
ascli builds the transfer-spec internally as a Hash.
It is not necessary to provide additional parameters on the command line for a transfer.
It is possible to modify or add any of the supported transfer-spec parameters using the ts option.
The ts option accepts a Hash Extended Value containing one or several transfer-spec parameters.
Multiple ts options on command line are cumulative, and the Hash value is deeply merged.
To remove a (deep) key from transfer spec, set the value to null.
Note
Default transfer spec values can be displayed with command:
ascli config ascp info --fields=ts --out.flat=no
It is possible to specify ascp options when the transfer option is set to direct using transfer option parameter: ascp_args.
Example:
--transfer=@json:'{"ascp_args":["-l","100m"]}'
Or an equivalent (using dotted expression):
--transfer.ascp_args=@list:' -l 100m'
This is especially useful for ascp command line parameters not supported in the transfer spec.
The use of a transfer-spec instead of ascp command line arguments has the following advantages:
- It is common to all Transfer Agents
- It does not depend on command line limitations (special characters, and so on)
Transfer Parameters🔗
All standard transfer-spec parameters can be specified. A transfer-spec can also be saved/overridden in the configuration file.
References:
- Aspera Node API Documentation → /opt/transfers
- Aspera Transfer Daemon Documentation → Guides → API Ref → Transfer Spec V1
- Aspera Connect SDK → search
The parameters for starting a transfer.
Parameters can be displayed with commands:
ascli config ascp spec
An optional parameter can be specified to display only the parameters supported by a specific transfer agent, or overview for one column per agent indicating which agents support each parameter:
ascli config ascp spec direct
ascli config ascp spec overview
ascli config ascp spec overview --fields=name,a,c,d,h,n,t
A JSON Schema can be generated with command:
ascli config ascp schema --format=jsonpp
An optional parameter can be specified to display the schema for a specific transfer agent:
ascli config ascp schema transferd --format=jsonpp
The description gives the supported agents, if not all, and the corresponding ascp argument or environment variable.
Transfer Specification Reference🔗
| ID | Name |
|---|---|
| A | Direct |
| C | Connect |
| D | Desktop |
| H | Httpgw |
| N | Node |
| T | Transferd |
| Field | Type | Description |
|---|---|---|
apply_local_docroot |
Bool |
Apply local docroot to source paths. (A, T) ( --apply-local-docroot) |
authentication |
String |
Set to token for SSH bypass keys, else password asked if not provided.(C) |
cipher |
Enum[String] |
In transit encryption algorithms. Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm.Default: none.( -c (conversion){enum}) |
cipher_allowed |
Enum[String] |
Returned by node API. Valid literals include aes-128 and none.Allowed values: none, aes-128, aes-192, aes-256, aes-128-cfb, aes-192-cfb, aes-256-cfb, aes-128-gcm, aes-192-gcm, aes-256-gcm.(C) |
content_protection |
Enum[String] |
Enable client-side content protection (CSEAR, encryption-at-rest). For uploads, set to encrypt to transfer encrypted files and store them on the server with the extension .aspera-env. (aspera.conf parameter transfer_encryption_content_protection_extension). To download and decrypt encrypted files, set to decryptcontent_protection_password must be specified if this option is set.Allowed values: encrypt, decrypt.( --file-crypt={enum}) |
content_protection_password |
String |
Password for encryption/decryption of transferred assets. (env: ASPERA_SCP_FILEPASS) |
cookie |
String |
Metadata for transfer specified by application. (env: ASPERA_SCP_COOKIE) |
create_dir |
Bool |
Create target directory if it doesn't already exist. If all the following conditions are met, then the destination_root specifies a filename instead of destination folder:- create_dir is false- A single source file is given on command line - The target folder specified by destination_root does not existIn all other cases, destination_root specifies a folder, and it is created if it does not already exist. I.e. if any of those conditions is met:- create_dir is true- Multiple source files are provided - List of source files are provided in a file (list or pair), default for Node API and ascli.- The target folder exists ( -d) |
delete_before_transfer |
Bool |
Before transfer, delete files that exist at the destination but not at the source. The source and destination arguments must be directories that have matching names. Objects on the destination that have the same name but different type or size as objects on the source are not deleted. ( --delete-before-transfer) |
delete_source |
Bool |
Remove transferred source files after transfer success. Equivalent to remove_after_transfer + remove_empty_directories + remove_empty_source_directory. Take precedence over those.(A, N, T) |
destination_root |
String |
Destination root directory. Required. |
destination_root_id |
String |
The file ID of the destination root directory. Required when using Bearer token auth for the destination node. (T) |
dgram_size |
Integer |
UDP datagram size in bytes. ( -Z {integer}) |
direction |
Enum[String] |
Direction of transfer (on client side). Required. Allowed values: send, receive.( --mode=(conversion){enum}) |
exclude_newer_than |
String |
Exclude files, but not directories, from the transfer if they are newer than the specified number of seconds added to the source computer's epoch. e.g. -86400 for newer than a day back.( --exclude-newer-than={string}) |
exclude_older_than |
String |
Exclude files, but not directories, from the transfer if they are older than the specified number of seconds added to the source computer's epoch. e.g. -86400 for older than a day back.( --exclude-older-than={string}) |
fail_bad_filepass |
Bool |
Fail on bad file decryption passphrase. (A, T) ( --fail-bad-filepass) |
fasp_port |
Integer |
Specifies fasp (UDP) port. Default: 33001.( -O {integer}) |
fasp_proxy |
Hash |
Proxy for communications between the remote server and the (local) client. (T) |
file_checksum |
Enum[String] |
Enable checksum reporting for transferred files by specifying the hash to use. Allowed values: sha-512, sha-384, sha-256, sha1, md5, none.Default: none.(A, N) ( --file-checksum={enum}) |
http_fallback |
Bool, String |
When true(1), attempts to perform an HTTP transfer if a FASP transfer cannot be performed. ( -y (conversion){boolean\|string}) |
http_fallback_port |
Integer |
Specifies HTTP port when no cipher is used. ( -t {integer}) |
https_fallback_port |
Integer |
Specifies HTTPS port when cipher is used. ( -t {integer}) |
icos |
Hash |
Configuration parameters for IBM Cloud Object Storage (ICOS). (T) |
keepalive |
Bool |
The session is running in persistent session mode. (A, T) ( --keepalive) |
lock_min_rate |
Bool |
n/a (C) |
lock_min_rate_kbps |
Bool |
If true, lock the minimum transfer rate to the value set for min_rate_kbps.If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.(C, T) |
lock_rate_policy |
Bool |
If true, lock the rate policy to the default value.(C, T) |
lock_target_rate |
Bool |
n/a (C) |
lock_target_rate_kbps |
Bool |
If true, lock the target transfer rate to the default value set for target_rate_kbps.If false, users can adjust the transfer rate up to the value set for target_rate_cap_kbps.(C, T) |
min_rate_cap_kbps |
Integer |
The highest minimum rate that an incoming transfer can request, in kilobits per second. Client minimum rate requests that exceed the minimum rate cap are ignored. The default value of unlimited applies no cap to the minimum rate. (Default: 0) (C, T) |
min_rate_kbps |
Integer |
Set the minimum transfer rate in kilobits per second. Default: 0.( -m {integer}) |
move_after_transfer |
String |
Move source files to the specified archive-dir directory after they are transferred correctly.Available as of 3.8.0. Details in ascp manual.Requires write permissions on the source. If src_base is specified, files are moved to archive-dir/path-relative-to-srcbase.archive-dir must be in the same file system (or cloud storage account) as the source files being transferred.archive-dir is subject to the same docroot restrictions as source files.move_after_transfer and remove_after_transfer are mutually exclusive options.After files have been moved to the archive, the original source directory structure is left in place. Empty directories are not saved to archive-dir. To remove empty source directories after a successful move operation, also set remove_empty_directories to true. When using remove_empty_directories, empty directory removal examination starts at the srcbase and proceeds down any subdirectories. If no srcbase is used and a file path (as opposed to a directory path) is specified, then only the immediate parent directory is examined and removed if it is empty following the move of the source file.(A, N, T) ( --move-after-transfer={string}) |
multi_session |
Integer |
Use multi-session transfer. max 128. Each participant on one host needs an independent UDP (-O) port. Large files are split between sessions only when transferring with resume_policy=none.(special: -C {integer}) |
multi_session_threshold |
Integer |
Split files across multiple ascp sessions if their size in bytes is greater than or equal to the specified value.(0=no file is split) (A, N, T) ( --multi-session-threshold={integer}) |
obfuscate_file_names |
Bool |
HTTP Gateway obfuscates file names when set to true.(H) |
overwrite |
Enum[String] |
Overwrite files at the destination with source files of the same name based on the policy: - always : Always overwrite the file.- never : Never overwrite the file. If the destination contains partial files that are older or the same as the source files and resume is enabled, the partial files resume transfer. Partial files with checksums or sizes that differ from the source files are not overwritten.- diff : (default) Overwrite the file if it is different from the source, depending on the compare method (default is size). If the destination is object storage, diff has the same effect as always. If resume is not enabled, partial files are overwritten if they are different from the source, otherwise they are skipped. If resume is enabled, only partial files with different sizes or checksums from the source are overwritten; otherwise, files resume.- diff+older : Overwrite the file if it is older and different from the source, depending on the compare method (default is size). If resume is not enabled, partial files are overwritten if they are older and different from the source, otherwise they are skipped. If resume is enabled, only partial files that are different and older than the source are overwritten, otherwise they are resumed.- older : Overwrite the file if its timestamp is older than the source timestamp.If you set an overwrite policy of diff or diff+older, difference is determined by the value set for resume_policy:none : The source and destination files are always considered different and the destination file is always overwrittenattributes : The source and destination files are compared based on file attributes sparse_checksum : The source and destination files are compared based on sparse checksums, (currently file size)full_checksum : The source and destination files are compared based on full checksumsAllowed values: never, always, diff, older, diff+older.Default: diff.( --overwrite={enum}) |
password |
String |
Password for local Windows user when transfer user associated with node API user is not the same as the one running asperanoded.Allows impersonating the transfer user and have access to resources (e.g. network shares). Windows only, node API only. (N) |
paths |
Array |
Array of path to the source (required) and a path to the destination (optional). Required. |
precalculate_job_size |
Bool |
Specifies whether to precalculate the job size. ( --precalculate-job-size / --precalculate-disable) |
preserve_access_time |
Bool |
Preserve the source-file access timestamps at the destination. Because source access times are updated by the transfer operation, the timestamp that is preserved is the one just before to the transfer. ( --preserve-access-time) |
preserve_acls |
Enum[String] |
Preserve access control lists. Allowed values: none, native, metafile.Default: none.(A, T) ( --preserve-acls={enum}) |
preserve_creation_time |
Bool |
Preserve source-file creation timestamps at the destination. Only Windows systems retain information about creation time. If the destination is not a Windows computer, this option is ignored. ( --preserve-creation-time) |
preserve_extended_attrs |
Enum[String] |
Preserve the extended attributes. Allowed values: none, native, metafile.Default: none.(A, T) ( --preserve-xattrs={enum}) |
preserve_file_owner_gid |
Bool |
Preserve the group ID for a file owner. (A, T) ( --preserve-file-owner-gid) |
preserve_file_owner_uid |
Bool |
Preserve the user ID for a file owner. (A, T) ( --preserve-file-owner-uid) |
preserve_modification_time |
Bool |
Set the modification time, the last time a file or directory was modified (written), of a transferred file to the modification of the source file or directory. Preserve source-file modification timestamps at the destination. ( --preserve-modification-time) |
preserve_remote_acls |
Enum[String] |
Preserve remote access control lists. Allowed values: none, native, metafile.Default: none.(A, T) ( --remote-preserve-acls={enum}) |
preserve_remote_extended_attrs |
Enum[String] |
Preserve remote extended attributes. Allowed values: none, native, metafile.Default: none.(A, T) ( --remote-preserve-xattrs={enum}) |
preserve_source_access_time |
Bool |
Preserve the time logged for when the source file was accessed. (A, T) ( --preserve-source-access-time) |
preserve_times |
Bool |
Preserve file timestamps. (A, N, T) ( -p) |
proxy |
String |
Specify the address of the Aspera high-speed proxy server.dnat(s)://[user[:password]@]server:portDefault ports for DNAT and DNATS protocols are 9091 and 9092. Password, if specified here, overrides the value of environment variable ASPERA_PROXY_PASS.(A) ( --proxy={string}) |
rate_policy |
Enum[String] |
The transfer rate policy to use when sharing bandwidth. Allowable values: - high : When sharing bandwidth, transfer at twice the rate of a transfer using a fair policy.- fair : (Default) Share bandwidth equally with other traffic.- low : Use only unused bandwidth.- fixed : Transfer at the target rate, regardless of the actual network capacity. Do not share bandwidth. Aspera recommends that you do not use this setting except under special circumstances, otherwise the destination storage can be damaged.Allowed values: low, fair, high, fixed.( --policy={enum}) |
rate_policy_allowed |
Enum[String] |
Specifies most aggressive rate policy that is allowed. Returned by node API. Allowed values: low, fair, high, fixed.(C) |
remote_access_key |
String |
The access key ID of the access key that was used to construct the bearer token that is used to authenticate to the remote node. (T) |
remote_host |
String |
IP or fully qualified domain name (FQDN) of the remote server. Required. ( --host={string}) |
remote_password |
String |
SSH session password. (env: ASPERA_SCP_PASS) |
remote_user |
String |
Remote user. Default value is xfer on node or connect.Required. ( --user={string}) |
remove_after_transfer |
Bool |
Remove SRC files after transfer success. (A, N, T) ( --remove-after-transfer) |
remove_empty_directories |
Bool |
Specifies whether to remove empty directories. (A, N, T) ( --remove-empty-directories) |
remove_empty_source_dir |
Bool |
Remove empty source subdirectories and remove the source directory itself, if empty. (T) |
remove_empty_source_directory |
Bool |
Remove empty source subdirectories and remove the source directory itself, if empty. (A) ( --remove-empty-source-directory) |
remove_skipped |
Bool |
Must also have remove_after_transfer set to true. Defaults to false. If true, skipped files will be removed as well.(A, C, N) ( --remove-skipped) |
resume_policy |
Enum[String] |
If a transfer is interrupted or fails to finish, this policy directs the transfer to resume without retransferring the files. Allowable values: - none : Always re-transfer the entire file.- attrs : Compare file attributes and resume if they match, and re-transfer if they do not.- sparse_csum : Compare file attributes and the sparse file checksums; resume if they match, and re-transfer if they do not.- full_csum : Compare file attributes and the full file checksums; resume if they match, and re-transfer if they do not.Note: transferd uses values: attributes, sparse_checksum, full_checksum.Allowed values: none, attrs, sparse_csum, full_csum.Default: faspmgr:none;other:sparse_csum.( -k (conversion){enum}) |
retry_duration |
Integer, String |
Specifies how long to wait before retrying transfer (e.g. 5min).(T) |
save_before_overwrite |
Bool |
If a transfer would result in an existing file File attributes are maintained in the renamed file. (A, N, T) ( --save-before-overwrite) |
skip_duplicate_check |
Bool |
Don't check for duplicate files at the destination. (A, T) ( --skip-dir-traversal-dupes) |
skip_special_files |
Bool |
All assets other than files, directories and symbolic links are considered special. A transfer will fail if the user attempts to transfer special assets. If true, ascp skips special assets and proceeds with the transfer of all other assets.(A, T) ( --skip-special-files) |
source_root |
String |
Path to be prepended to each source path. This is either a conventional path or it can be a URI but only if there is no root defined. ( --source-prefix64=(conversion){string}) |
source_root_id |
String |
The file ID of the source root directory. Required when using Bearer token auth for the source node. (N, T) |
src_base |
String |
Specify the prefix to be stripped off from each source object. The remaining portion of the source path is kept intact at the destination. Special care must be taken when used with cloud storage. (A, N, T) ( --src-base64=(conversion){string}) |
src_base64 |
String |
The folder name below which the directory structure is preserved (base64 encoded). (A, T) ( --src-base64={string}) |
ssh_args |
Array |
Add arguments to the command-line arguments passed to the external ssh program (implies -SSH). The arguments are inserted before any key file(s) supplied to ascp and before the user/host arguments.(A, T) (special: --ssh-arg={array}) |
ssh_port |
Integer |
Specifies SSH (TCP) port. Default: direct:22, other:33001.( -P {integer}) |
ssh_private_key |
String |
Private key used for SSH authentication. Shall look like: -----BEGIN RSA PRIV4TE KEY-----\nMII... Note the JSON encoding: \n for newlines. (A, T) (env: ASPERA_SCP_KEY) |
ssh_private_key_passphrase |
String |
The passphrase associated with the transfer user's SSH private key. Available as of 3.7.2. (A, T) (env: ASPERA_SCP_PASS) |
ssh_private_key_path |
String |
Path to private key for SSH. (A, T) ( -i {string}) |
sshfp |
String |
Check it against server SSH host key fingerprint. ( --check-sshfp={string}) |
symlink_policy |
Enum[String] |
Handle source side symbolic links. Refer to HSTS manual for more details. - follow: Follow a symbolic link and transfer the contents of the linked file or directory when the link target is in the user's docroot.- copy: Copy only the symbolic link. If a file with the same name exists at the destination, the symbolic link does not replace the file.- skip: Skip-symbolic links. The link or the file to which it points are not transferred.- copy+force: Copy only the symbolic link. If a file with the same name exists at the destination, the symbolic link replaces the file. If the file of the same name at the destination is a symbolic link to a directory, it is not replaced.Allowed values: follow, copy, copy+force, skip.Default: follow.( --symbolic-links={enum}) |
tags |
Hash |
Metadata for transfer as JSON. Key aspera is reserved. Key aspera.xfer_retry specifies a "retry timeout" but is not actually used for node API initiated transfers.( --tags64=(conversion){object}) |
tags64 |
String |
Base64 string for tags.(A, T) ( --tags64={string}) |
target_rate |
String, Integer |
Pseudo-parameter: desired transfer rate in bps, with an optional unit suffix. Converted to target_rate_kbps (rounded down) before the transfer is started, overriding it if both are present. Also applies to async (sync). Accepted formats: integer (bps), or integer followed by k/K (x1000), m/M (x1000000), g/G (x1000000000). Examples: 100000000, 100000k, 100m, 1g. |
target_rate_cap_kbps |
Integer |
Maximum target rate for incoming transfers, in kilobits per second. Returned by upload/download_setup node API. (C, T) |
target_rate_kbps |
Integer |
Specifies desired speed for the transfer. ( -l {integer}) |
title |
String |
Title of the transfer. (C, N, T) |
token |
String |
Authorization token. Type is Bearer, Basic or ATM. (env: ASPERA_SCP_TOKEN) |
use_ascp4 |
Bool |
Specify version of protocol. ascp4 will be deprecated.Default: false.(A, N, T) |
use_system_ssh |
Bool |
Use an external ssh program instead of the built-in libssh2 implementation to establish the connection to the remote host. The desired ssh program must be in the environment's PATH.To enable debugging of the ssh process, supply -DD and --ssh-arg=-vv arguments to ascp.(A, T) ( -SSH) |
wss_enabled |
Bool |
Server has Web Socket service enabled. (special: --ws-connect) |
wss_port |
Integer |
TCP port used for Web Socket service feed. |
xfer_max_retries |
Integer |
Maximum number of retries, for node API initiated transfers. Shall not exceed aspera.conf parameter transfer_manager_max_retries (default 3).(N) |
Destination folder for transfers🔗
The destination folder is set by ascli by default to:
.for downloads/for uploads
It is specified by the transfer-spec parameter destination_root.
As such, it can be modified with option: --ts=@json:'{"destination_root":"<PATH>"}'.
The option to_folder provides an equivalent and convenient way to change this parameter:
--to-folder=<PATH>.
List of files for transfers🔗
When uploading, downloading or sending files, the user must specify the list of files to transfer.
By default, the list of files to transfer is provided on the command line.
The list of (source) files to transfer is specified by (Extended Value) option sources (default: @args).
The list is either the list of source files, or a combined source/destination list (see below) depending on value of option src_type (default: list).
In ascli, all transfer parameters, including file list, are provided to the transfer agent in a transfer-spec so that execution of a transfer is independent of the transfer agent (direct, connect, node, transfer daemon...).
So, eventually, the list of files to transfer is provided to the transfer agent using the transfer-spec field: "paths" which is a list (array) of pairs of "source" (mandatory) and "destination" (optional).
The sources and src_type options provide convenient ways to populate the transfer spec with the source file list.
Possible values for option sources are:
@args: (default) the list of files (or file pair) is directly provided on the command line (after commands): unused arguments (not starting with-) are considered as source files.
Info
When using @: to build a command parameter and --sources=@args (default),
the END marker is required between the @: block and the file list.
See Positional Arguments with Dot-path.
Example:
ascli server upload ~/mysample.file secondfile
This is the same as (with default values):
ascli server upload --sources=@args --src-type=list ~/mysample.file secondfile
- An Extended Value with type
ArrayofString
Tip
An Extended Value can be tested with the command config echo
Examples:
-
Using Extended Value
Create the file list:
echo ~/mysample.file > myfilelist.txt echo secondfile >> myfilelist.txtUse the file list: one path per line:
--sources=@lines:@file:myfilelist.txt -
Using JSON array
--sources=@json:'["file1","file2"]' -
Using STDIN, one path per line
--sources=@lines:@stdin: -
Using Ruby code (one path per line in file)
--sources=@ruby:'File.read("myfilelist.txt").split("\n")' -
@ts: the user provides the list of files directly in thepathsfield of transfer spec (optionts). Examples: -
Using transfer spec
--sources=@ts --ts=@json:'{"paths":[{"source":"file1"},{"source":"file2"}]}'
The option src_type allows specifying if the list specified in option sources is a simple file list or if it is a file pair list.
Note
Option src_type is not used if option sources is set to @ts
Supported values for src_type are:
list: (default) the path of destination is the same as source and each entry is a source file pathpair: the first element is the first source, the second element is the first destination, and so on.
Example: Source file 200KB.1 is renamed sample1 on destination:
ascli server upload --src-type=pair ~/Documents/Samples/200KB.1 /Upload/sample1
Source directory structure on destination🔗
This section is not specific to ascli: it describes ascp behavior.
The transfer destination is normally expected to designate a destination folder.
But there is one exception: The destination specifies the new item name when the following are met:
- There is a single source item (file or folder)
- Transfer spec
create_diris not set totrue(ascpoption-dnot provided) - Destination is not an existing folder
- The
dirnameof destination is an existing folder
For this reason, it is recommended to set create_dir to true for consistent behavior between single and multiple item transfers; this is the default in ascli.
If a simple source file list is provided (no destination in paths, that is, no file_pair_list provided), the destination folder is used as destination folder for each source file, and source file folder names are not preserved.
The inner structure of source items that are folders is preserved on destination.
A leading / on destination is ignored (relative to docroot) unless docroot is not set (relative to home).
In the following table source folder d3 contains 2 files: f1 and d4/f2.
| Source files | Destination | Folders on Dest. | create_dir |
Destination Files |
|---|---|---|---|---|
f1 |
d/f |
- | false | Error: d does not exist. |
f1 |
d/f |
d |
false | d/f (renamed) |
f1 |
d/f/. |
d |
false | d/f (renamed) |
f1 |
d/f |
d/f |
false | d/f/f1 |
f1 f2 |
d |
d |
false | d/f1 d/f2 |
d3 |
d |
- | false | d/f1 d/f2 (renamed) |
f1 |
d |
- | true | d/f1 |
f1 f2 |
d |
- | true | d/f1 d/f2 |
d1/f1 d2/f2 |
d |
- | true | d/f1 d/f2 |
d3 |
d |
- | true | d/d3/f1 d/d3/d4/f2 |
If a file pair list is provided, then it is possible to rename or specify a different destination folder for each source (relative to the destination).
If transfer spec has a src_base, it has the side effect that the simple source file list is considered as a file pair list, and so the lower structure of source folders is preserved on destination.
| Source files | Destination | src_base |
Destination Files |
|---|---|---|---|
d1/d2/f2 d1/d3/f3 |
d |
d1 |
d/d2/f2 d/d3/f3 |
Advanced Example: Send files ./file1 and ./folder2/files2 to server (for example, /Upload) and keep the original file names and folders, that is, send file1 to /Upload/file1 and files2 to /Upload/folder2/files2.
- If files are specified as
./file1 ./folder2/files2,
then destination will be: /Upload/file1 /Upload/files2
- One possibility is to specify a file pair list:
--src-type=pair file1 file1 folder2/files2 folder2/files2 - Another possibility is to specify a source base (transfer spec parameter
src_base):--ts.src_base=$PWD $PWD/file1 $PWD/folder2/files2
The . path cannot be used as a source base.
- Similarly, create a temporary soft link (Linux):
ln -s . tmp_baseand use--ts.src_base=tmp_base tmp_base/file1 tmp_base/folder2/files2 - One can also similarly use
--sources=@tsand specify the list of files in thepathsfield of transfer spec with bothsourceanddestinationfor each file.
Multi-session transfer🔗
Multi session, that is, starting a transfer of a file set using multiple sessions (one ascp process per session) is supported on direct and node agents, not yet on connect.
--transfer=node
--ts=@json:'{"multi_session":10,"multi_session_threshold":1}'
Multi-session is directly supported by the node daemon.
--transfer=direct
--ts=@json:'{"multi_session":5,"multi_session_threshold":1,"resume_policy":"none"}'
Note
resume_policy set to attr may cause problems: none or sparse_csum shall be preferred.
ascli starts multiple ascp for Multi-session using direct agent.
When multi-session is used, one separate UDP port is used per session (refer to ascp manual page).
Content protection🔗
Content protection (Client-Side Encryption at Rest, CSEAR) ensures that files remain encrypted while stored on the server. With CSEAR, the client encrypts files during upload and decrypts files during download, using a passphrase known only to the users sharing the files.
- Upload: Files are encrypted on the client side before being sent to the server.
- Download: Files are decrypted on the client side as they are retrieved from the server.
At all times, files remain encrypted on the server; encryption and decryption occur exclusively on the client side.
Activating CSEAR consists of setting transfer spec parameters:
content_protection: activate encryption (encryptfor upload) or decryption (decryptfor download)content_protection_password: the passphrase to be used.
Example: parameter to download a Faspex package and decrypt on the fly
--ts=@json:'{"content_protection":"decrypt","content_protection_password":"<PASSWORD>"}'
Note
Faspex 5 requires package parameter ear_enabled set to true for CSEAR.
In that case the transfer spec parameter content_protection is automatically set.
content_protection_password is then required in all cases.
Transfer Spec Examples🔗
- Change target rate
--ts=@json:'{"target_rate_kbps":500000}'
- Override the FASP SSH port to a specific TCP port:
--ts=@json:'{"ssh_port":33002}'
- Force HTTP fallback mode:
--ts=@json:'{"http_fallback":"force"}'
- Activate progress when not activated by default on server
--ts=@json:'{"precalculate_job_size":true}'
Transfer progress bar🔗
File transfer operations are monitored, and a progress bar is displayed on the standard error if option progress_bar (Bool) is set to yes (default if the standard error is a terminal).
So, the progress bar is displayed even if the output of the command is redirected to a file.
The same progress bar is used for any type of transfer: using ascp, server to server, using HTTPS, and so on.
It shows the elapsed time, the percentage, the rate in megabits per second (Mbps, 1,000,000 bits per second, averaged over the last 5 seconds) and the estimated remaining time.
As long as the total size is not known (for example, when the job size is not pre-calculated), the transferred size is shown instead of the percentage.
For multi-session transfers, sessions are aggregated, and the number of running sessions is shown in brackets.
Files already at destination (resumed transfer) count in the progress, but not in the rate.
If the transfer fails, the progress bar stops at the reached progress, and shows failed.
Log lines are displayed above the progress bar.
Agent direct can display the native progress bar of ascp instead (see direct).
Scheduler🔗
ascli does not include a built-in scheduler.
Automated execution should therefore rely on operating system facilities.
Two common execution modes are supported:
-
Scheduled execution: run
asclicommands periodically. -
Daemon/service mode: run
asclicontinuously as a server.
Creating a wrapping script🔗
Before configuring scheduling or services, it is often useful to create a wrapper script that prepares the environment and optionally enforces execution limits.
This script may:
-
Configure environment variables (PATH, Ruby, and so on)
-
Enforce a maximum runtime
-
Centralize command options
Example (Linux)
#!/bin/bash
# Load the Ruby environment if required
# Forward arguments to ascli and ensure it does not exceed 30 min execution
exec timeout 30m ascli "${@}"
Save as:
/home/xfer/bin/ascli_tool
Then, make it executable.
Windows: Scheduler🔗
Windows provides the Task Scheduler.
Tasks can be configured using:
-
PowerShell module
ScheduledTasks -
taskschd.msc(UI)
By default, Windows Task Scheduler prevents overlapping executions.
Linux: systemd Timer🔗
Most modern Linux distributions use systemd which provides scheduling via timer units.
Define a name for the job, for example: ascli_job as <NAME> below.
-
Create the service
/etc/systemd/system/<NAME>.service[Unit] Description=Run ascli preview scan [Service] Type=oneshot ExecStart=/home/xfer/bin/ascli_tool preview scan --logger=syslog --out.level=error User=xfer Group=xfer -
Create the timer (same base name)
/etc/systemd/system/<NAME>.timer[Unit] Description=Hourly execution of <NAME> [Timer] OnCalendar=hourly AccuracySec=1min Persistent=true [Install] WantedBy=timers.target -
Enable the timer
sudo systemctl daemon-reload sudo systemctl enable --now <NAME>.timer
Check status:
systemctl status <NAME>.timer
systemctl list-timers | grep <NAME>
Run service once to test:
sudo systemctl start <NAME>.service
Linux: cron Scheduler🔗
Unix-like systems also provide cron.
Example of crontab for user xfer.
crontab<<EOF
0 * * * * /home/xfer/bin/ascli_tool preview scan --logger=syslog --out.level=error
2-59 * * * * /home/xfer/bin/ascli_tool preview trevents --logger=syslog --out.level=error
EOF
Linux also provides anacron for daily or hourly jobs that must run even if the system was previously offline.
Note
Logging options are specified directly in the cron entry so command output remains visible when running the command manually.
Running as system service (Daemon mode)🔗
Some commands run continuously (for example, listening on a network port).
In this case, it is recommended to run ascli as a system service.
On Linux this is typically done using systemd.
For example, below <NAME> is ascli_svc.
A wrapping script, again, is convenient: /usr/local/bin/start_<NAME>.sh:
Service definition at /etc/systemd/system/<NAME>.service:
[Unit]
Description=My ascli daemon
After=network.target
[Service]
ExecStart=/usr/local/bin/start_<NAME>.sh
Restart=on-failure
RestartSec=15
User=xfer
# Optional, specify a working directory
# WorkingDirectory=/path/to/working/dir
# Optional, redirect output
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
Enable and start:
sudo systemctl daemon-reload
sudo systemctl enable --now <NAME>.service
systemctl status <NAME>.service
journalctl -u <NAME>.service
Preventing Concurrent Execution🔗
When running scheduled jobs, it may be necessary to ensure that only one instance of ascli runs at a time.
This avoids situations such as:
-
Multiple instances accumulating and overloading the system
-
Duplicate transfers
-
Duplicate
previewoutputs
Option: lock_port🔗
ascli provides a built-in locking mechanism via the lock_port option.
-
lock_portprevents concurrent executions of the same command. -
If another instance is already running and holding the specified port, the new execution exits immediately.
Under the hood, this works by opening a local TCP server port. If the port is already in use, the process fails, effectively acting as a local lock. The lock is released automatically when the process exits.
Note
When using ascli in containers, lock_port only works within the same container.
Each container has its own network namespace, so the lock does not apply across containers.
Example: Test ascli locking by running this command in two separate terminals within 30 seconds:
ascli config echo @ruby:'sleep 30' --lock-port=12345
-
The first instance will sleep for 30 seconds.
-
The second instance will exit immediately with:
WARN -- : Another instance is already running (Address already in use - bind(2) for "127.0.0.1" port 12345).
OS-based concurrency protection🔗
Operating system schedulers also provide protection against overlapping runs.
- Windows Task Scheduler prevents concurrent runs by default.
- Linux
croncan leverageflockto achieve the same effect:
/usr/bin/flock -w 0 /var/cron.lock ascli ...
systemdavoids overlaps when scheduling aType=oneshotservice.
[Unit]
Description=My ascli daemon
...
[Service]
ExecStart=/usr/local/bin/start_<NAME>.sh
Type=oneshot
RemainAfterExit=yes
...
Note
Systemd timers can schedule this service and ensure no overlapping runs occur.
"Provençal"🔗
ascp, the underlying executable implementing Aspera file transfer using FASP, has a capability to not only access the local file system (using system's open,read,write,close primitives), but also to do the same operations on other data storage such as S3, Hadoop and others.
This mechanism is called PVCL (from Provençal, a restaurant located in Sophia Antipolis).
Several PVCL adapters are available, one is embedded in ascp, the others are provided in shared libraries and must be activated.
The list of supported PVCL adapters can be retrieved with command:
ascli config ascp info --fields=@re:'^pvcl'
process v1
shares v1
noded v1
faux v1
file v1
stdio v1
stdio-tar v1
The output shows the adapters: process, shares, noded, faux, file, stdio, stdio-tar.
Those adapters can be used wherever a file path is used in ascp including configuration.
They act as a pseudo drive.
The simplified format is:
<ADAPTER>:///<SUB_FILE_PATH>?<ARG1>=<VAL1>&...
One of the adapters, used in this manual, for testing, is faux.
It is a pseudo file system allowing generation of file data without actual storage (on source or destination).
Other PVCL modules exist and are implemented in shared libraries.
Those libraries are available only on HSTS.
To activate a PVCL library, place the corresponding shared library in the same folder as ascp.
Example:
cp /opt/aspera/lib/pvcl/libpvcl_cloud.so $(ascli config ascp info --fields=root)
Then check available modules as shown previously (ascp info).
faux: for testing🔗
This adapter can be used to simulate a file or a directory.
This is a feature of ascp, not ascli.
The following is an extract of the man page of ascp.
To discard data at the destination, the destination argument is set to faux://.
To send uninitialized data in place of an actual source file, the source file is replaced with an argument of the form:
faux:///filename?filesize
Where:
filenameis the name that will be assigned to the file on the destinationfilesizeis the number of bytes that will be sent (in decimal).
Note
Characters ? and & are shell special characters (wildcard and background), so faux file specification on command line should be protected (using quotes or \).
If not, the shell may give error: no matches found or equivalent.
For all sizes, a suffix can be added (case-insensitive) to the size: k, m, g, t, p, e (values are powers of 2, for example, 1M is 220, that is, 1 mebibyte, not megabyte).
The maximum allowed value is 8*260.
Extremely large faux file sizes (petabyte range and above) will likely fail due to lack of destination storage unless destination is faux://.
To send uninitialized data in place of a source directory, the source argument is replaced with an argument of the form:
faux:///dirname?<ARG1>=<VAL1>&...
Where:
dirnameis the folder name and can contain/to specify a subfolder.- Supported arguments are:
| Name | Type | Description |
|---|---|---|
| count | int | Number of files Mandatory |
| file | string | Basename for files Default: file |
| size | int | Size of first file. Default: 0 |
| inc | int | Increment applied to determine next file size Default: 0 |
| seq | enum | Sequence in determining next file size Values: random, sequentialDefault: sequential |
| buf_init | enum | How source data is initialized Option none is not allowed for downloads.Values: none, zero, randomDefault: zero |
The sequence parameter is applied as follows:
-
If
seqisrandomthen each file size is: -
size +/- (inc * rand()) - Where rand is a random number between 0 and 1
- File size must not be negative;
incis set tosizeif it is greater thansize. -
Similarly, overall file size must be less than 8*260. If size + inc is greater, inc will be reduced to limit size + inc to 7*260.
-
If
seqissequentialthen each file size is: -
size + ((file_index - 1) * inc) - Where first file is index 1
- So file1 is
sizebytes, file2 issize + incbytes, file3 issize + inc * 2bytes, and so on - As with
random,incwill be adjusted ifsize + (count * inc)is not less than 8*260.
Filenames generated are of the form: <FILE>_<00000 ... count>_<FILESIZE>
Examples:
- Upload 20 gibibytes of generated data to file
myfilein directory/Upload
ascli server upload faux:///myfile\?20g --to-folder=/Upload
- Upload a file /tmp/sample but do not save results to disk (no docroot on destination)
ascli server upload /tmp/sample --to-folder=faux://
- Upload a faux directory
mydircontaining 1 million files, sequentially with sizes ranging from 0 to 2 Mebibyte - 2 bytes, with the base name of each file beingtestfileto /Upload
ascli server upload "faux:///mydir?file=testfile&count=1m&size=0&inc=2&seq=sequential" --to-folder=/Upload
- Upload a faux directory
mydircontaining 1000 files, of size 1 byte, with the base name of each file beingtestfileto /Upload
ascli server upload "faux:///mydir?file=testfile&count=1000&size=1" --to-folder=/Upload
file: for growing files🔗
A growing file is a file that another program is still writing during the transfer, for example a live recording, a log file, or a stream capture.
The built-in file PVCL adapter of ascp transfers such a file: ascp sends the data as the file grows, and completes the transfer when the file stops changing.
This is a feature of ascp, not ascli.
For more information, see the HSTS ascp command reference.
Syntax of the file: URI🔗
To transfer a growing file, specify the source file as a file: URI that contains the grow parameter:
file:///<PATH>?grow=<WAIT_TIME>[&<PARAMETER>=<VALUE>...]
The path of the file follows the prefix file:///:
| Path | Example URI | File |
|---|---|---|
| Relative | file:///live.log?grow=60 |
live.log in the current folder |
| Absolute | file:////var/log/app.log?grow=60 |
/var/log/app.log |
An absolute path starts with /, so the URI of an absolute path contains four slashes.
The URI supports the following parameters:
| Parameter | Description |
|---|---|
grow |
Wait time, in seconds: the transfer completes when the file does not change during this time. Required. If the value is not a number, for example grow=default, the wait time is 10 seconds. |
wait_start |
Start of the wait time: - mtime: the last modification time of the file.- null_read: the time when ascp reaches the end of the file, that is, when a read of the file returns no data.Default: mtime |
confirm_stop |
If true, the program that writes the file confirms the end of the file. See Confirming the end of a growing file.Ignored if wait_start is null_read.Default: false |
max_grow_size |
Maximum size of the file, in bytes. Accepts the suffixes k, m, g, t, p, and e (powers of 1024).Default: 263 - 1 bytes, or the maximum file size of the file system if lower. |
Info
The growing file must be on a native file system, that is, a file system that the operating system accesses directly.
Tip
On the command line, enclose the URI in single quotes: the characters ? and & have a special meaning for the shell.
Completion of the transfer🔗
ascp sends the data as the program writes it.
When ascp has sent all the data that the file contains, it starts to count the wait time.
If the file grows during the wait time, ascp sends the new data and starts to count the wait time again.
When the wait time ends without a change to the file, the transfer completes.
As a result:
- The command completes after the wait time that follows the last write.
For example, with
grow=120, the command completes about 2 minutes after the program stops writing. - If the program pauses for longer than the wait time, the transfer completes before the end of the file, and the destination file is incomplete. Set a wait time that is longer than the longest pause of the program.
To complete the transfer without waiting, use confirm_stop.
Confirming the end of a growing file🔗
With confirm_stop=true, the program that writes the file signals the end of the file, so the transfer does not wait.
- Start the transfer with
confirm_stop=trueand a wait time that is longer than the longest pause of the program:
ascli server upload live.log --ts.source_root='file:///?grow=600&confirm_stop=true' --to-folder=/Upload
- When the program finishes writing the file, set the modification time of the file to a time earlier than the wait time. For example:
touch -t 200001010000 live.log
The transfer completes immediately.
Info
With confirm_stop=true, if the wait time ends before the program sets the modification time, ascp considers the file invalid and the transfer fails.
To report an error in the file, with or without confirm_stop, the program sets the modification time of the file to 0, that is, 1970-01-01 00:00:00 UTC.
ascp then fails the transfer of the file.
For example:
TZ=UTC touch -t 197001010000 live.log
Uploading a growing file🔗
ascp activates the growing file mode only if it finds the grow parameter in one of the following locations:
- The first source on its command line.
- The source prefix:
ascpoption--source-prefix, set by transfer spec parametersource_root. - The docroot of the transfer user, on the server.
ascp does not look for the grow parameter in a file list.
By default, the direct agent provides the sources to ascp in a file list.
As a result, if you specify a file: URI as a source with the default options, the transfer fails with the error Session initiation failed, fail to set pvcl.
Use one of the following methods.
The examples use the direct agent.
- Set the URI as source prefix (recommended)
Set transfer spec parameter source_root to a file: URI that contains the folder of the files and the parameters.
Specify the sources as paths relative to this folder.
ascp inserts each source path before the parameters: for example, file:////var/log?grow=120 and app.log give file:////var/log/app.log?grow=120.
All the files of the transfer use the same parameters.
To upload live.log from the current folder:
ascli server upload live.log --ts.source_root='file:///?grow=120' --to-folder=/Upload
To upload /var/log/app.log:
ascli server upload app.log --ts.source_root='file:////var/log?grow=120' --to-folder=/Upload
- Set the URI as source on the command line
Set transfer parameter file_list to false, so that ascli places the source on the ascp command line instead of in a file list.
Use this method for a single file.
ascli server upload 'file:///live.log?grow=120' --to-folder=/Upload --transfer.file_list=false
Tip
To display the native progress bar of ascp instead of the progress bar of ascli, add --transfer.quiet=false.
Downloading a growing file🔗
For a download, the growing file is on the server.
The transfer user on the server usually has a docroot, and the server does not resolve a file: URI from the client in this docroot: the transfer fails with the error No such file or directory.
Instead, the server administrator sets the parameters in the docroot of the transfer user.
For example, on the server:
asconfigurator -x 'set_user_data;user_name,xfer_live;absolute,file:////data/live?grow=120'
Then, the client downloads the file with a path relative to the docroot, as for any other file:
ascli server download live.log --to-folder=.
All the transfers of this user are then growing file transfers, so use a dedicated transfer user.
This method also applies to transfers that use a token, for example with the Node API or Aspera on Cloud: for a download with a token, ascp does not accept a file: URI as source prefix.
If the transfer user has no docroot and the transfer does not use a token, you can also use the upload methods, with absolute paths on the server.
Usage🔗
ascli -h
WARN No config file found. New configuration file: /home/runner/.aspera/ascli/config.yaml
NAME
ascli -- a command line tool for Aspera Applications (v4.29.0.pre)
SYNOPSIS
ascli [GLOBAL_OPTIONS] <command> [OPTIONS] [ARGS]
DESCRIPTION
Use Aspera application to perform operations on command line.
Documentation and examples: https://rubygems.org/gems/aspera-cli
execute: ascli conf doc
or visit: https://ibm.biz/ascli-doc
source repo: https://github.com/IBM/aspera-cli
ENVIRONMENT VARIABLES
Any option can be set as an environment variable, refer to the manual
COMMANDS
To list first level commands, execute: ascli
Note that commands can be written shortened (provided it is unique).
OPTIONS
Options begin with a '-' (minus), and value is provided on command line.
Special values are supported beginning with special prefix @pfx:, where pfx is one of:
val, base64, csvt, env, file, uri, json, lines, list, none, path, re, ruby, s, secret, stdin, yaml, zlib, extend, preset, vault
Dates format is 'YYYY-MM-DD HH:MM:SS', or 'now' or '-<num>h'
ARGS
Some commands require mandatory arguments, e.g. a path.
EXIT STATUS
0 Success
1 Other error
2 Command line error: unknown command or option, invalid or missing argument
3 Authentication or authorization failure: SSH, HTTP 401 or 403, OAuth grant
4 Resource not found, or several resources match the identifier, HTTP 404
5 Transfer failure
6 Network or TLS failure: name resolution, connection, timeout, certificate
7 Other error returned by a remote API
130 Interrupted by user (Ctrl-C)
OPTIONS: global
--interactive=yes|no Use interactive input of missing params
--ask-options=yes|no Ask even optional options
--out=HASH Output rendering options (dot-notation: format, level, file, fields, select, query, table[.pivot], flat, secrets, colors, utf8, img)
--display=info|data|error Output only some information (deprecated after 4.27.0: use --out.level)
--format=ENUM Output format (also: --out.format)
--output=VALUE Destination for results (deprecated after 4.27.0: use --out.file)
--fields=LIST Comma separated list of: fields, or ALL, or DEF (also: --out.fields)
--select=HASH Select only some items in lists: column, value (also: --out.select)
--table-style=HASH (Table) Display style (deprecated after 4.27.0: use --out.table)
--flat-hash=yes|no (Table) Display deep values as additional keys (deprecated after 4.27.0: use --out.flat)
--multi-single=no|yes|single (Table) Control how object list is displayed as single table, or multiple objects (deprecated after 4.27.0: use --out.table.pivot)
--show-secrets=yes|no Show secrets on command output (deprecated after 4.27.0: use --out.secrets)
--image=HASH Options for displaying images and thumbnails in the terminal (deprecated after 4.27.0: use --out.img)
-h, --help Show this message
--show-config Display parameters used for the provided action
-v, --version Display version
--ui=HASH User interface (dot-notation: mode, web_auth, command)
--invalid-characters=VALUE Replacement character and invalid filename characters
--log-level=ENUM Log level
--log-format=VALUE Log formatter
--logger=stderr|stdout|syslog Logging method
--log=HASH Logging options (dot-notation: level, type, format, secrets)
--security=HASH Security restrictions (dot-notation: level, allowed_paths)
--lock-port=INT Prevent dual execution of a command, e.g. in cron
--once-only=yes|no Process only new items (some commands)
--dry-run=yes|no Do not perform changes, only show what would be done
--log-secrets=yes|no Show passwords in logs
--temp=HASH Temporary files (dot-notation: folder, clean, expiry)
--clean-temp=yes|no Cleanup temporary files on exit (deprecated after 4.27.5: use --temp.clean)
--temp-folder=VALUE Temporary folder (deprecated after 4.27.5: use --temp.folder)
--pid-file=VALUE Write process identifier to file, delete on exit
--parser=none|json|ruby|yaml Default parser for structured parameters and options
--home=VALUE Home folder for tool
--config-file=VALUE Path to YAML file with preset configuration
--secret=VALUE Secret for access keys
--vault=HASH Secret vault configuration
--vault-password=VALUE Vault password
--progress-bar=yes|no Display progress bar
--fpac=VALUE Proxy auto configuration script
--proxy-credentials=LIST HTTP proxy credentials for fpac: user, password
--sql=VALUE SQL suffix appended to sqlite3 queries for admin subcommands (e.g. WHERE clause)
--ascp-logs=HASH Ascp: log analysis (dot-notation: period, zero_bytes, mtu, resolve, thresholds, timezone, health_minutes, remote_max_size)
-P, --preset=VALUE Load the named option preset from current config file
--version-check-days=INT Period in days to check new version (zero to disable)
--plugin-folder=VALUE Folder where to find additional plugins
--sdk-url=VALUE Ascp: URL to get Aspera Transfer Executables
--locations-url=LIST Ascp: URLs to get download locations of Aspera Transfer Daemon
--sdk-folder=VALUE Ascp: Path to folder with ascp (or product with "product:")
--smtp=HASH SMTP email server configuration
--notify-to=VALUE Email: Recipient for notification of transfers
--notify-template=VALUE Email: ERB template for notification of transfers
--cache-tokens=yes|no Save and reuse OAuth tokens
--expand-mounts=yes|no Commands: list commands of sub-trees provided by another plugin
-N, --no-default Do not load default configuration for plugin
--insecure=yes|no HTTP/S: Do not validate any certificate
--ignore-certificate=LIST HTTP/S: Do not validate certificate for these URLs
--warn-insecure=yes|no HTTP/S: Issue a warning if certificate is ignored
--cert-stores=LIST HTTP/S: List of folder with trusted certificates
--http-options=HASH HTTP/S connection parameters for REST calls (not `ascp` WSS)
--http-proxy=VALUE HTTP/S: URL for proxy with optional credentials
--query=HASH Additional filter for for some commands (list/delete)
--bulk=yes|no Bulk operation (only some)
--bfail=yes|no Bulk operation error handling
--ts=HASH Override transfer spec values
--to-folder=VALUE Destination folder for transferred files
--sources=VALUE How list of transferred files is provided (@args,@ts,Array)
--src-type=list|pair Type of file list
--transfer=HASH Transfer agent type, or agent parameters with optional agent key
--transfer-info=HASH Parameters for transfer agent (deprecated after 4.26.2: use --transfer instead)
PLUGINS
alee Aspera License Entitlement Engine
aoc Aspera on Cloud
ats Aspera Transfer Service
console Console
cos IBM Cloud Object Storage
faspex5 Faspex v5
faspio faspio Gateway
httpgw HTTP Gateway
mcp Model Context Protocol Server
node HSTS Node API
orchestrator Orchestrator
preview Preview
server HSTS Fasp/SSH
shares Shares
transferd Transfer Daemon
Bulk creation and deletion of resources🔗
Bulk creation and deletion of resources are possible using option bulk (yes,no(default)).
In that case, the operation expects an Array of Hash instead of a simple Hash using the Extended Value syntax.
This option is available only for some resources: if you need it, try and see if the entities you create or delete support this option.
Plugins🔗
ascli uses a plugin mechanism.
The first level command (just after ascli on the command line) is the name of the concerned plugin which will execute the command.
Each plugin usually represents commands sent to a specific application.
Available plugins can be found using command:
ascli config plugins list
╭────────┬────────┬────────┬─────────────────────────────────────────────────╮
│ plugin │ detect │ wizard │ path │
╞════════╪════════╪════════╪═════════════════════════════════════════════════╡
│ shares │ ✓ │ ✓ │ .../aspera-cli/lib/aspera/cli/plugins/shares.rb │
│ node │ ✓ │ ✓ │ .../aspera-cli/lib/aspera/cli/plugins/node.rb │
...
╰────────┴────────┴────────┴─────────────────────────────────────────────────╯
Most plugins will take the URL option: url to identify their location.
REST APIs of Aspera legacy applications (Aspera Node, Shares, Console, Orchestrator) use simple username/password authentication: HTTP Basic Authentication using options: username and password.
Aspera on Cloud and Faspex 5 rely on OAuth.
By default, plugins are looked up in folders specified by (multi-value) option plugin_folder:
ascli --show-config --fields=plugin_folder
You can create the skeleton of a new plugin like this:
ascli config plugins create foo .
Created ./foo.rb
ascli --plugin-folder=. foo
Plugins vs Transfer Agents🔗
Plugins typically represent a specific remote application on which ascli can operate, while transfer agents are the underlying components that handle the actual data transfer.
A given remote application can sometimes be both a plugin and a transfer agent.
For example: node and httpgw are both plugins and transfer agents.
A plugin is invoked as the first positional argument in a command line.
A Transfer Agent is used by setting the option transfer (for example, --transfer=node).
Command line parsing, Special Characters🔗
ascli is typically executed in a shell, either interactively or in a script.
ascli receives its arguments on the command line.
The way arguments are parsed and provided to ascli depends on the operating system and shell.
Shell parsing for Unix-like systems: Linux, macOS, AIX🔗
Linux command line parsing is well-defined: It is fully documented in the shell's documentation.
On Unix-like environments, this is typically a POSIX-like shell (bash, zsh, ksh, sh).
A c-shell (csh, tcsh) or other shell can also be used.
In this environment, the shell parses the command line, possibly replacing variables, and so on.
See bash shell operation.
The shell builds the list of arguments and then fork/exec Ruby with that list.
Ruby receives the list of command line arguments from the shell and gives it to ascli.
Special character handling (quotes, spaces, env vars, ...) is handled by the shell for any command executed.
Shell parsing for Windows🔗
On Windows, command line parsing first depends on the shell used (see next sections). MS Windows command line parsing differs from Unix-like systems because Windows does not provide a list of arguments to the executable (Ruby): it provides the whole command line as a single string, but the shell may interpret some special characters.
So command line parsing is not handled by the shell (cmd.exe), not handled by the operating system, but it is handled by the executable (Ruby).
Typically, Windows executables use the Microsoft library for this parsing.
As far as ascli is concerned: the executable is Ruby.
It has its own parsing algorithm, close to a Linux shell parsing.
Thankfully, ascli provides a command to check the value of an argument after parsing: config echo.
One can also run ascli with option --log-level=debug to display the command line after parsing.
It is also possible to display arguments received by Ruby using this command:
C:> ruby -e 'puts ARGV' "Hello World" 1 2
Hello World
1
2
Note
Use pp instead of puts to display as Ruby Array.
Once the shell has dealt with the command line "special" characters for it, the shell calls Windows' CreateProcess with the whole command line as a single string.
(Unlike Unix-like systems where the command line is split into arguments by the shell.)
It's up to the program to split arguments:
- Windows: How Command Line Parameters Are Parsed
- Understand Quoting and Escaping of Windows Command Line Arguments
ascli is a Ruby program, so Ruby parses the command line (received with GetCommandLineW) into arguments and provides them to the Ruby code ($0 and ARGV).
Ruby vaguely follows the Microsoft C/C++ parameter parsing rules.
(See w32_cmdvector in Ruby source win32.c):
- Space characters: split arguments (space, tab, newline)
- Backslash:
\escapes a single special character - Globbing characters:
*?[]{}for file globbing - Double quotes:
" - Single quotes:
'
Shell parsing for Windows: cmd.exe🔗
The following examples give the same result on Windows using cmd.exe:
- Single quote protects the double quote
ascli config echo @json:'{"url":"https://..."}'
- Triple double quotes are replaced with a single double quote
ascli config echo @json:{"""url""":"""https://..."""}
- Double quote is escaped with backslash within double quotes
ascli config echo @json:"{\"url\":\"https://...\"}"
cmd.exe handles some special characters: ^"<>|%&.
It handles I/O redirection (<>|), shell variables (%), and multiple commands (&).
Eventually, all those special characters are removed from the command line unless escaped with ^ or ".
" are kept and given to the program.
Shell parsing for Windows: PowerShell🔗
For PowerShell, the behavior depends on the version (5.1, 7.3+).
A difficulty is that PowerShell parses the command line for its own use and manages special characters, but then it passes the command line to the program (Ruby) as a single string, possibly without the special characters.
If not using PowerShell features (for example, variables), one can use the "stop-parsing" token --%.
Details can be found here:
PowerShell 5🔗
- Check your PowerShell version:
$psversiontable.psversion.Major
5
The following examples give the same result on Windows using PowerShell 5:
╭───────┬───────╮
│ field │ value │
╞═══════╪═══════╡
│ x │ true │
│ k │ v │
╰───────┴───────╯
- Use PowerShell argument
--%to place PowerShell in "stop-parsing" mode.
ascli config echo --% @json:'{"k":"v","x":true}'
- Triple double quotes are replaced with a single double quote in normal mode:
ascli config echo @json:'{"""k""":"""v""","""x""":true}'
- To insert PowerShell variables in the JSON string, one can do:
$var="v"
ascli config echo $('@json:{"""k""":"""' + $var + '""","""x""":true}')
PowerShell 7🔗
- Check your PowerShell version:
$psversiontable.psversion.Major
7
The following examples give the same result on Windows using PowerShell 7:
- Use PowerShell argument
--%to place PowerShell in "stop-parsing" mode.
ascli config echo --% @json:{"k":"v","x":true}
- Single quote protects double quote in normal mode:
ascli config echo @json:'{"k":"v","x":true}'
- To insert PowerShell variables in the JSON string, one can do:
$var="v"
ascli config echo $('@json:{"k":"' + $var + '","x":true}')
- Use PowerShell structure and then convert to JSON string:
$var="v"
ascli config echo "@json:$(@{ k = $var; x = $true } | ConvertTo-Json -Compress)"
Extended Value (JSON, Ruby, ...)🔗
Some values provided to ascli (options, Command Parameters) are expected to be Extended Values, that is, not a simple String, but a composite structure (Hash, Array).
For small structures with a few keys, the dot-path key.subkey=value notation (using @: to collect positional arguments, or --opt.key=value directly for options) is often the most convenient: it requires no quoting and reads naturally on the command line.
For larger or more complex structures (deep nesting, arrays of hashes, values copied from API documentation), the @json: modifier is typically a better fit.
@json: expects a JSON value; note that " is used to enclose a String in JSON, which may be difficult to specify in shells: enclose the whole argument in single quotes to avoid this.
Any option or Command Parameter expecting a Hash value accepts the special value help to display its schema.
See Schema Discovery with help.
Using a shell variable, parsed by shell, in an Extended Value🔗
To be evaluated by the shell, the shell variable must not be in single quotes.
Enclose the variable in double quotes so that it results in a single argument for ascli even if its value contains spaces: bash splits the result of an unquoted variable expansion into several words (zsh does not, by default).
Note
We use a simple shell variable in this example. It does not need to be exported as an environment variable.
MYVAR="Hello World"
ascli config echo @json:'{"title":"'"$MYVAR"'"}' --format=json
ascli config echo "@json:{\"title\":\"$MYVAR\"}" --format=json
{"title":"Hello World"}
Double quote in strings in command line🔗
Double quote is a shell special character. Like any shell special character, it can be protected either by preceding with a backslash or by enclosing in a single quote.
ascli config echo \"
ascli config echo '"'
"
Double quote in JSON is a little tricky because " is special both for the shell and JSON.
Both shell and JSON syntax allow protecting ", but only the shell allows protection using single quote.
ascli config echo @json:'"\""' --format=text
ascli config echo @json:\"\\\"\" --format=text
ascli config echo @ruby:\'\"\' --format=text
"
Here a single quote or a backslash protects the double quote to avoid shell processing, and then an additional \ is added to protect the " for JSON.
But as \ is also shell special, then it is protected by another \.
Shell and JSON or Ruby special characters in Extended Value🔗
Construction of values with special characters is done like this:
-
First select a syntax to represent the Extended Value, for example, JSON or Ruby
-
Write the expression using this syntax, for example, using JSON:
{"title":"Test \" ' & \\"}
Or, using Ruby:
{"title"=>"Test \" ' & \\"}
{'title'=>%q{Test " ' & \\}}
Both " and \ are special characters for JSON and Ruby and can be protected with \ (unless Ruby's extended single quote notation %q is used).
- Then, since the value will be evaluated by shell, any shell special characters must be protected, either using preceding
\for each character to protect, or by enclosing in single quote:
ascli config echo @json:{\"title\":\"Test\ \\\"\ \'\ \&\ \\\\\"} --format=json
ascli config echo @json:'{"title":"Test \" '\'' & \\"}' --format=json
ascli config echo @ruby:"{'title'=>%q{Test \" ' & \\\\}}" --format=json
{"title":"Test \" ' & \\"}
Reading special characters interactively🔗
If ascli is used interactively (a user typing on terminal), values can be entered by the user:
ascli config echo @ruby:"{'title'=>gets.chomp}" --format=json
gets is Ruby's method of terminal input (terminated by \n), and chomp removes the trailing \n.
Command line arguments from a file🔗
If you need to provide a list of command line arguments from lines that are in a file, on Linux you can use the xargs command:
xargs -a lines.txt -d \\n ascli config echo
This is equivalent to execution of:
ascli config echo [line1] [line2] [line3] ...
If there are spaces in the lines, those are not taken as separator, as we provide option -d \\n to xargs.
Extended value using special characters read from environment variables or files🔗
Using a text editor or shell: create a file title.txt (and env var) that contains exactly the text required: Test " ' & \ :
export MYTITLE='Test " '\'' & \'
echo -n $MYTITLE > title.txt
Using those values will not require any escaping of characters since values do not go through shell or JSON parsing.
If the value is to be assigned directly to an option of ascli, then you can directly use the content of the file or env var using the @file: or @env: readers:
ascli config echo @file:title.txt --format=text
ascli config echo @env:MYTITLE --format=text
Test " ' & \
If the value to be used is in a more complex structure, then the @ruby: modifier can be used: it allows any Ruby code in expression, including reading from file or env var.
In those cases, there is no character to protect because values are not parsed by the shell, or JSON or even Ruby.
ascli config echo @ruby:"{'title'=>File.read('title.txt')}" --format=json
ascli config echo @ruby:"{'title'=>ENV['MYTITLE']}" --format=json
{"title":"Test \" ' & \\"}