Skip to content

Add a16w8 per-op test for bmm#19599

Merged
christine-long-meta merged 1 commit into
pytorch:mainfrom
christine-long-meta:export-D104532363
May 19, 2026
Merged

Add a16w8 per-op test for bmm#19599
christine-long-meta merged 1 commit into
pytorch:mainfrom
christine-long-meta:export-D104532363

Conversation

@christine-long-meta
Copy link
Copy Markdown
Contributor

@christine-long-meta christine-long-meta commented May 14, 2026

Summary:
Add int16 activation / int8 weight (a16w8) quantization tests for aten.bmm on Ethos-U55 and Ethos-U85.

Context

Batch matrix multiply (bmm) implements the core Q @ K^T and attn_weights @ V operations in the multi-head attention of the EMG2Pose Conformer. At int16 IO precision the accumulator width and rescale path differ between U55 and U85, so dedicated per-op coverage is needed to catch numerics divergence before it surfaces as an end-to-end SNR regression. The test matrix includes square, rectangular, and large-batch configurations to exercise different tiling strategies in the Vela backend.

Also removes unused aten_op_mm / exir_op_mm variables that were dead code in test_bmm.py.

Changes

  • Add a16w8_bmm_test_parameters dict with 5 test configurations covering same-shape, different-shape, rectangular, batch-10, and negative-value tensors
  • Add test_bmm_a16w8_u55_INT using EthosU55PipelineINT with a16w8_quantization=True, symmetric_io_quantization=True, qtol=128, epsilon=2**-16
  • Add test_bmm_a16w8_u85_INT using EthosU85PipelineINT with same kwargs
  • Remove unused aten_op_mm and exir_op_mm variables
  • Register ops/test_bmm.py in fbcode/ and xplat/ targets.bzl

bypass-pytorch-oss-checks

Reviewed By: Ninja91

Differential Revision: D104532363

@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented May 14, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19599

Note: Links to docs will display an error until the docs builds have been completed.

❗ 2 Active SEVs

There are 2 currently active SEVs. If your PR is affected, please view them below:

⏳ No Failures, 269 Pending

As of commit 1c06ec5 with merge base 41a38d8 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@github-actions github-actions Bot added ciflow/trunk module: arm Issues related to arm backend labels May 14, 2026
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 14, 2026
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented May 14, 2026

Workflows were awaiting approval. CI has now been triggered for the ciflow labels on this PR.

@github-actions
Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

christine-long-meta added a commit to christine-long-meta/executorch that referenced this pull request May 14, 2026
Summary:

Add int16 activation / int8 weight (a16w8) quantization tests for `aten.bmm` on Ethos-U55 and Ethos-U85.


## Changes
- Add `a16w8_bmm_test_parameters` dict with 5 test configurations covering same-shape, different-shape, rectangular, batch-10, and negative-value tensors
- Add `test_bmm_a16w8_u55_INT` using `EthosU55PipelineINT` with `a16w8_quantization=True, symmetric_io_quantization=True, qtol=128, epsilon=2**-16`
- Add `test_bmm_a16w8_u85_INT` using `EthosU85PipelineINT` with same kwargs
- Remove unused `aten_op_mm` and `exir_op_mm` variables
- Register `ops/test_bmm.py` in `fbcode/` and `xplat/` `targets.bzl`

bypass-pytorch-oss-checks

Differential Revision: D104532363
@meta-codesync meta-codesync Bot changed the title Add a16w8 per-op test for bmm Add a16w8 per-op test for bmm (#19599) May 14, 2026
@meta-codesync
Copy link
Copy Markdown
Contributor

meta-codesync Bot commented May 14, 2026

@christine-long-meta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D104532363.

1 similar comment
@meta-codesync
Copy link
Copy Markdown
Contributor

meta-codesync Bot commented May 14, 2026

@christine-long-meta has exported this pull request. If you are a Meta employee, you can view the originating Diff in D104532363.

christine-long-meta added a commit to christine-long-meta/executorch that referenced this pull request May 14, 2026
Summary:
Pull Request resolved: pytorch#19599

Add int16 activation / int8 weight (a16w8) quantization tests for `aten.bmm` on Ethos-U55 and Ethos-U85.

## Changes
- Add `a16w8_bmm_test_parameters` dict with 5 test configurations covering same-shape, different-shape, rectangular, batch-10, and negative-value tensors
- Add `test_bmm_a16w8_u55_INT` using `EthosU55PipelineINT` with `a16w8_quantization=True, symmetric_io_quantization=True, qtol=128, epsilon=2**-16`
- Add `test_bmm_a16w8_u85_INT` using `EthosU85PipelineINT` with same kwargs
- Remove unused `aten_op_mm` and `exir_op_mm` variables
- Register `ops/test_bmm.py` in `fbcode/` and `xplat/` `targets.bzl`

bypass-pytorch-oss-checks

Differential Revision: D104532363
@christine-long-meta christine-long-meta force-pushed the export-D104532363 branch 2 times, most recently from 6a8da35 to 71103e7 Compare May 16, 2026 02:07
@meta-codesync meta-codesync Bot changed the title Add a16w8 per-op test for bmm (#19599) Add a16w8 per-op test for bmm May 16, 2026
@meta-codesync meta-codesync Bot changed the title Add a16w8 per-op test for bmm Add a16w8 per-op test for bmm (#19599) May 16, 2026
@meta-codesync meta-codesync Bot changed the title Add a16w8 per-op test for bmm (#19599) Add a16w8 per-op test for bmm May 17, 2026
@zingo
Copy link
Copy Markdown
Collaborator

zingo commented May 18, 2026

You seem to got a strange error in Yest ARM Backend, It seems very unrelated, I rerun the tests maybe it was a network glitch and will go away.

Copy link
Copy Markdown
Collaborator

@zingo zingo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test re-run passed, approving :) Thanks!

@meta-codesync meta-codesync Bot changed the title Add a16w8 per-op test for bmm Add a16w8 per-op test for bmm (#19599) May 19, 2026
christine-long-meta added a commit to christine-long-meta/executorch that referenced this pull request May 19, 2026
Summary:

Add int16 activation / int8 weight (a16w8) quantization tests for `aten.bmm` on Ethos-U55 and Ethos-U85.

## Changes
- Add `a16w8_bmm_test_parameters` dict with 5 test configurations covering same-shape, different-shape, rectangular, batch-10, and negative-value tensors
- Add `test_bmm_a16w8_u55_INT` using `OpNotSupportedPipeline` to verify that bmm with INT16 inputs is correctly rejected on U55 (which does not support bmm with int16)
- Add `test_bmm_a16w8_u85_INT` using `EthosU85PipelineINT` with `a16w8_quantization=True, symmetric_io_quantization=True`
- Remove unused `aten_op_mm` and `exir_op_mm` variables
- Register `ops/test_bmm.py` in `fbcode/` and `xplat/` `targets.bzl`

Reviewed By: Ninja91

Differential Revision: D104532363
Summary:
Add int16 activation / int8 weight (a16w8) quantization tests for `aten.bmm` on Ethos-U55 and Ethos-U85.

## Context
Batch matrix multiply (`bmm`) implements the core `Q @ K^T` and `attn_weights @ V` operations in the multi-head attention of the EMG2Pose Conformer. At int16 IO precision the accumulator width and rescale path differ between U55 and U85, so dedicated per-op coverage is needed to catch numerics divergence before it surfaces as an end-to-end SNR regression. The test matrix includes square, rectangular, and large-batch configurations to exercise different tiling strategies in the Vela backend.

Also removes unused `aten_op_mm` / `exir_op_mm` variables that were dead code in `test_bmm.py`.

## Changes
- Add `a16w8_bmm_test_parameters` dict with 5 test configurations covering same-shape, different-shape, rectangular, batch-10, and negative-value tensors
- Add `test_bmm_a16w8_u55_INT` using `EthosU55PipelineINT` with `a16w8_quantization=True, symmetric_io_quantization=True, qtol=128, epsilon=2**-16`
- Add `test_bmm_a16w8_u85_INT` using `EthosU85PipelineINT` with same kwargs
- Remove unused `aten_op_mm` and `exir_op_mm` variables
- Register `ops/test_bmm.py` in `fbcode/` and `xplat/` `targets.bzl`

bypass-pytorch-oss-checks

Reviewed By: Ninja91

Differential Revision: D104532363
@meta-codesync meta-codesync Bot changed the title Add a16w8 per-op test for bmm (#19599) Add a16w8 per-op test for bmm May 19, 2026
@christine-long-meta christine-long-meta merged commit afd32cc into pytorch:main May 19, 2026
433 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported meta-exported module: arm Issues related to arm backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants