ONNX Support#

ONNX supported operators#

Currently, HElayers partially support ONNX files. It is recommended to review the supported operations below before loading an ONNX file into HElayers. When an unsupported operator is loaded to HElayers, it will try to automatically identify it and alert the user about it.

The supported ONNX operators are:

  • AveragePool

  • BatchNormalization

  • Cast

  • Concat

  • Conv

  • Flatten

  • Gemm

  • Identity

  • MatMul

  • ReduceSum

  • ReduceMean

  • Squeeze

  • Transpose

  • Unsqueeze

In addition, the following operators are interpreted by HElayers as polynomial activation layers:

  • Constant

  • Div

  • Pow

  • Sub

  • The “Add” and “Mul” operators are also supported and may be used either as a separate layer that performs element-wise addition or multiplication on the outputs of two previous layers together, or as part of an activation layer.

HElayers support non-sequential graphs of operators: operator’s output can be the input of multiple operators, and the outputs of two operators can be added, multiplied or concatenated to one another as long as the tensor shapes are compatible. The operators must form a DAG, where all paths start and end in the same nodes. For example, an NN must have a single input, which can then be fed into multiple layers.

Note: dimension 0 is assumed to be the batch dimension. This dimension must not be changed, manipulated or involved in operations in any way.

Add#

  • Must have two inputs: the input data and weights, or two inputs.

  • Must have one output.

  • No attributes are supported (any attribute will be ignored).

AveragePool#

  • Must have one input: the input data.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”kernel_shape” - required, must contain two integers

    • ”strides” - required, must contain two integers

    • ”pads” - required, must contain 4 integers. Alternatively, pads will be taken from a previous “Pad” operator, if such operator appears before this layer.

  • NOTICE: when an average pooling layer includes padding, its evaluation by Helayers will include the pads when averaging, regardless of any setting or behaviour of an external library.

BatchNormalization#

  • Must have five inputs: the input data, the scales, the biases, the means and the variances.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”epsilon” - required, the value to use for the normalization, to avoid deviding by zero.

  • The batch normalization is assumed to be over dimension #1.

Cast#

  • Must have one input.

  • Must have one output.

  • No attributes are supported (any attribute will be ignored).

  • Has no effect.

Concat#

  • Must have two inputs.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”axis” - required, the axis to concatenate along.

Constant#

  • Must have no inputs.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”value” - required, must contain a valid tensor with a single value.

Conv#

  • Must have two or three inputs: the input data, filters and optional biases.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”kernel_shape” - required, must contain two integers

    • ”strides” - required, must contain two integers

    • ”pads” - required, must contain 4 integers

    • ”group” - required. Currently, must be either 1, or the number of input channels in case they also equal the number of output channels.

  • The number of filters will be deduced from the associated filters specified as input for the operator.

  • Filters are assumed to be of shape [filters][channels][rows][cols]

Div#

  • Must have two inputs.

  • Must have one output.

  • No attributes are supported (any attribute will be ignored).

Flatten#

  • Must have one input: the input data.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”axis” - required, must contain 1 integer with a value of 1

Gemm#

  • Must have two or three inputs: the input data, weights and optional bias.

  • Must have one output.

  • No attributes are supported (any attribute will be ignored).

  • The output size will be deduced from the associated weights specified as input for the operator.

  • Weights are assumed to be of shape [output_size][input_size]

Identity#

  • Must have one input.

  • Must have one output.

  • No attributes are supported (any attribute will be ignored).

MatMul#

  • Must have two inputs: the input data and weights, or two inputs.

  • Must have one output.

  • No attributes are supported (any attribute will be ignored).

Mul#

  • Must have two inputs: the input data and weights, or two inputs.

  • Must have one output.

  • No attributes are supported (any attribute will be ignored).

Pow#

  • Must have two inputs: the input data and the exponent.

  • Must have one output.

  • No attributes are supported (any attribute will be ignored).

  • The input associated with the exponent must contain a valid tensor with a single value, or be a “Constant” operator representing a single scalar. The exponent value must be a non-negative integer.

Pad#

  • Must have one or two inputs: the input data and optional pad values.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”pads” - required, must contain 8 integers: 0,0,T,L,0,0,B,R where T,L,B,R are the paddings from top, left, bottom and right, and can be zero or non-zero values. If provided, will be applied over the next layer which must be AveragePool. In case the operator has two inputs, the pad values will be taken from the second input and not from the “pads” attribute. In such case, the second input must contain a tensor of 8 integers that follow the above limitations.

ReduceMean#

  • Must have one or two inputs: the input data and optional axes value.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”axes” - required, the axis to apply the reduce along. In case the operator has two inputs, the are value will be taken from the second input and not from the “axes” attribute.

    • ”keepdims” - required, whether to keep the reduced dim as a degenerate dim or to remove it.

ReduceSum#

  • Must have one or two inputs: the input data and optional axes value.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”axes” - required, the axis to apply the reduce along. In case the operator has two inputs, the are value will be taken from the second input and not from the “axes” attribute.

    • ”keepdims” - required, whether to keep the reduced dim as a degenerate dim or to remove it.

Squeeze#

  • Must have one input.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”axes” - optional, the dimensions to squeeze. If empty, all the dimensions of size 1 will be squeezed.

Sub#

  • Must have two inputs.

  • Must have one output.

  • No attributes are supported (any attribute will be ignored).

Transpose#

  • Must have one input.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”perm” - required, the permutation to apply.

Unsqueeze#

  • Must have one input.

  • Must have one output.

  • The supported attributes are: (any other attribute will be ignored)

    • ”axes” - required, the dimensions to unsqueeze.