Repository navigation
[RELAX][Frontend][ONNX] Support for SimplifiedLayerNormalization and SkipSimplifiedLayerNormalization contrib ops - #20607
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The normalization converters fail on valid output and shape configurations and produce incorrect results for optional statistics, mixed dtypes, and float16 accumulation.
8 open issues
Promote float16 operands before normalization · New Compute requested normalization statistics · New Return the optional fourth residual-sum output · New Support broadcastable scale shapes · New Handle all-axes normalization · New Support the optional inverse-standard-deviation output · New Preserve the scale dtype for mixed-dtype inputs · New Use the ONNX default epsilon of 1e-12 · New
What changed in this PR
Adds support for two ONNX Runtime normalization operators in Relax’s ONNX frontend, targeting transformer model import failures reported in #20625.
Changes:
- Maps
SimplifiedLayerNormalizationto the existing RMS normalization converter. - Adds
SkipSimplifiedLayerNormalizationconversion with skip addition and optional bias. - Adds structural IR checks for static and symbolic shapes, including float16 simplified-normalization cases.
| File | Description |
|---|---|
tests/python/relax/test_frontend_onnx.py |
Adds expected IR helpers and tests for both operators. |
python/tvm/relax/frontend/onnx/onnx_frontend.py |
Registers both operators and implements skip/bias RMS normalization conversion. |
Review effort: Max
6cc048a to
953f424
Compare
|
Hi @mshr-h, @yzhliu, @tqchen, @tlopex, I have updated the PR to resolve merge conflicts and address all the review changes: SkipSimplifiedLayerNormalization
SimplifiedLayerNormalization (operator-specific adapter instead of an RMSNormalization alias)
Tests: execution and structural IR tests cover single and multi-output variants, all-axes reduction, rank-1 scales, and the fp16/fp32 dtype grid. Could you please take a look into the changes when you get a chance? Thanks! |
There was a problem hiding this comment.
🟡 Changes recommended
The converters return zero placeholders or incorrectly shaped normalization statistics, causing graphs that consume these optional outputs to produce incorrect results.
6 open issues
Preserve singleton dimensions in inv_std_var · New Support broadcastable RMSNorm scales of higher rank · New Honor or reject unsupported stash_type values · New Add numerical float16 and residual-output tests · New Return a tensor for single-output nodes · New Use the default ONNX domain for SimplifiedLayerNormalization · New
8 resolved since last review
Preserve the scale dtype for mixed-dtype inputs Support the optional inverse-standard-deviation output Handle all-axes normalization Support broadcastable scale shapes Return the optional fourth residual-sum output Compute requested normalization statistics Promote float16 operands before normalization Use the ONNX default epsilon of 1e-12
Review effort: Max
|
|
||
| if full_reduce or num_outputs > 1: | ||
| mean_square = relax.op.mean( | ||
| relax.op.multiply(data_compute, data_compute), axis=axes, keepdims=full_reduce |
| if isinstance(data.ty.shape, relax.ShapeExpr): | ||
| normalized_shape = relax.ShapeExpr(data.ty.shape.values[axis:]) | ||
| if not tvm_ffi.structural_equal(scale.ty.shape, normalized_shape): | ||
| scale = relax.op.broadcast_to(scale, normalized_shape) |
| # stash_type == 1 -> compute in float32, otherwise in the input dtype | ||
| compute_dtype = "float32" if stash_type == 1 else input_dtype |
| input_dtype = data.ty.dtype.dtype | ||
| low_precision = input_dtype in ("float16", "bfloat16") |
| if num_outputs > 3: | ||
| # Output 4 - input_skip_bias_sum | ||
| return relax.Tuple([output, placeholder, placeholder, data]) | ||
| return relax.Tuple([output, placeholder, placeholder]) |
| "SimplifiedLayerNormalization", | ||
| inputs=["input", "scale"], | ||
| outputs=["y"], | ||
| domain="com.microsoft", |


Summary
This PR adds frontend support for ONNX Runtime contrib operators SimplifiedLayerNormalization and SkipSimplifiedLayerNormalization in TVM Relax. These operators appear in transformer models (such as Xenova/sweep-next-edit-1.5B and SwinliQ-AIs/SmolLM2-1.7B-Instruct) and the absence of these ops raises tvm.error.OpNotImplemented error.
Scripts Involved
Testing & Validation
Test modules test_simplified_layer_norm and test_skipsimplifiedlayernormalization validate structural IR equality against expected TVMScript Relax IR for static and dynamic dimensions.
Closes #20625