Skip to content

[RELAX][Frontend][ONNX] Support for SimplifiedLayerNormalization and SkipSimplifiedLayerNormalization contrib ops - #20607

Open
Netra-Srinivasan-McW wants to merge 2 commits into
apache:mainfrom
Netra-Srinivasan-McW:Netra-Srinivasan-McW/onnx-simplified-layer-norm
Open

Netra-Srinivasan-McW wants to merge 2 commits into
apache:mainfrom
Netra-Srinivasan-McW:Netra-Srinivasan-McW/onnx-simplified-layer-norm

Conversation

@Netra-Srinivasan-McW

@Netra-Srinivasan-McW Netra-Srinivasan-McW commented Oct 8, 2026 •

Copy link
Copy Markdown

Summary
This PR adds frontend support for ONNX Runtime contrib operators SimplifiedLayerNormalization and SkipSimplifiedLayerNormalization in TVM Relax. These operators appear in transformer models (such as Xenova/sweep-next-edit-1.5B and SwinliQ-AIs/SmolLM2-1.7B-Instruct) and the absence of these ops raises tvm.error.OpNotImplemented error.

Scripts Involved

  1. python/tvm/relax/frontend/onnx/onnx_frontend.py
  2. tests/python/relax/test_frontend_onnx.py

Testing & Validation
Test modules test_simplified_layer_norm and test_skipsimplifiedlayernormalization validate structural IR equality against expected TVMScript Relax IR for static and dynamic dimensions.

Closes #20625

@Netra-Srinivasan-McW

Netra-Srinivasan-McW commented Oct 9, 2026 •

Copy link
Copy Markdown
Author

@tqchen @tlopex Could you review this PR when you find time. Thanks.

@Netra-Srinivasan-McW

Copy link
Copy Markdown
Author

Hi @mshr-h, @yzhliu, @tqchen, @tlopex,
Could you please take a look at this PR when you get a chance. Thanks.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The normalization converters fail on valid output and shape configurations and produce incorrect results for optional statistics, mixed dtypes, and float16 accumulation.

8 open issues
What changed in this PR

Adds support for two ONNX Runtime normalization operators in Relax’s ONNX frontend, targeting transformer model import failures reported in #20625.

Changes:

  • Maps SimplifiedLayerNormalization to the existing RMS normalization converter.
  • Adds SkipSimplifiedLayerNormalization conversion with skip addition and optional bias.
  • Adds structural IR checks for static and symbolic shapes, including float16 simplified-normalization cases.
File Description
tests/​python/​relax/​test_frontend_onnx.py Adds expected IR helpers and tests for both operators.
python/​tvm/​relax/​frontend/​onnx/​onnx_frontend.py Registers both operators and implements skip/bias RMS normalization conversion.

Review effort: Max

@Netra-Srinivasan-McW
Netra-Srinivasan-McW force-pushed the Netra-Srinivasan-McW/onnx-simplified-layer-norm branch from 6cc048a to 953f424 Compare November 20, 2026 10:28
@Netra-Srinivasan-McW

Copy link
Copy Markdown
Author

Hi @mshr-h, @yzhliu, @tqchen, @tlopex,

I have updated the PR to resolve merge conflicts and address all the review changes:

SkipSimplifiedLayerNormalization

  • Promote fp16/bf16 operands to fp32 before the residual add and normalization, then cast the outputs back.
  • Default epsilon is now 1e-12, matching ORT.
  • Return the optional 4th output (input_skip_bias_sum).

SimplifiedLayerNormalization (operator-specific adapter instead of an RMSNormalization alias)

  • Support the optional 2nd output (inv_std_var).
  • Broadcast scale to the normalized shape with relax.op.broadcast_to when needed.
  • Decompose all-axes normalization into explicit Relax ops to avoid the TOPI 0-dim indexing bug.
  • Compute in the stash_type precision and cast the output to the scale dtype.

Tests: execution and structural IR tests cover single and multi-output variants, all-axes reduction, rank-1 scales, and the fp16/fp32 dtype grid.

Could you please take a look into the changes when you get a chance? Thanks!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


if full_reduce or num_outputs > 1:
mean_square = relax.op.mean(
relax.op.multiply(data_compute, data_compute), axis=axes, keepdims=full_reduce
if isinstance(data.ty.shape, relax.ShapeExpr):
normalized_shape = relax.ShapeExpr(data.ty.shape.values[axis:])
if not tvm_ffi.structural_equal(scale.ty.shape, normalized_shape):
scale = relax.op.broadcast_to(scale, normalized_shape)
Comment on lines +5644 to +5645
# stash_type == 1 -> compute in float32, otherwise in the input dtype
compute_dtype = "float32" if stash_type == 1 else input_dtype
Comment on lines +5699 to +5700
input_dtype = data.ty.dtype.dtype
low_precision = input_dtype in ("float16", "bfloat16")
if num_outputs > 3:
# Output 4 - input_skip_bias_sum
return relax.Tuple([output, placeholder, placeholder, data])
return relax.Tuple([output, placeholder, placeholder])
"SimplifiedLayerNormalization",
inputs=["input", "scale"],
outputs=["y"],
domain="com.microsoft",

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][ONNX] SimplifiedLayerNormalization and SkipSimplifiedLayerNormalization ops unsupported

2 participants