Skip to content

Add --test HF CLI path for 2-layer random model configs, olive run and ModelBuilder support, Qwen how-to/layer-types fix, and merge conflict resolution #2459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Copilot wants to merge 37 commits into
base: main
Choose a base branch
from

Merge branch 'main' into copilot/fr-add-model-to-config-json

be35ef4
Select commit
Loading
Failed to load commit list.
Open

Add --test HF CLI path for 2-layer random model configs, olive run and ModelBuilder support, Qwen how-to/layer-types fix, and merge conflict resolution #2459

Merge branch 'main' into copilot/fr-add-model-to-config-json
be35ef4
Select commit
Loading
Failed to load commit list.
Azure Pipelines / Olive CI failed May 20, 2026 in 39m 8s

Build #20260520.4 had test failures

4 errors / 1 warnings

Details

Tests

  • Failed: 6 (0.12%)
  • Passed: 4,792 (93.83%)
  • Other: 309 (6.05%)
  • Total: 5,107

Annotations

Check failure on line 8487 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

Build log #L8487 

Bash exited with code '1'.

Check failure on line 18 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

Build log #L18 

There are one or more test failures detected in result files. Detailed summary of published test results can be viewed in the Tests tab.

Check failure on line 21 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

Build log #L21 

There are one or more test failures detected in result files. Detailed summary of published test results can be viewed in the Tests tab.

Check failure on line 20 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

Build log #L20 

There are one or more test failures detected in result files. Detailed summary of published test results can be viewed in the Tests tab.

Check failure on line 1 in test_hf_config_io_config

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

test_hf_config_io_config 

AssertionError: expected call not found.
Expected: get_model_io_config('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', BertForSequenceClassification(
  (bert): BertModel(
    (embeddings): BertEmbeddings(
      (word_embeddings): Embedding(1124, 32, padding_idx=0)
      (position_embeddings): Embedding(512, 32)
      (token_type_embeddings): Embedding(16, 32)
      (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
      (dropout): Dropout(p=0.1, inplace=False)
    )
    (encoder): BertEncoder(
      (layer): ModuleList(
        (0-4): 5 x BertLayer(
          (attention): BertAttention(
            (self): BertSelfAttention(
              (query): Linear(in_features=32, out_features=32, bias=True)
              (key): Linear(in_features=32, out_features=32, bias=True)
              (value): Linear(in_features=32, out_features=32, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
            (output): BertSelfOutput(
              (dense): Linear(in_features=32, out_features=32, bias=True)
              (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (intermediate): BertIntermediate(
            (dense): Linear(in_features=32, out_features=37, bias=True)
            (intermediate_act_fn): GELUActivation()
          )
          (output): BertOutput(
            (dense): Linear(in_features=37, out_features=32, bias=True)
            (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
            (dropout): Dropout(p=0.1, inplace=False)
          )
        )
      )
    )
    (pooler): BertPooler(
      (dense): Linear(in_features=32, out_features=32, bias=True)
      (activation): Tanh()
    )
  )
  (dropout): Dropout(p=0.1, inplace=False)
  (classifier): Linear(in_features=32, out_features=2, bias=True)
))
  Actual: get_model_io_config('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', BertForSequenceClassification(
  (bert): BertModel(
    (embeddings): BertEmbeddings(
      (word_embeddings): Embedding(1124, 32, padding_idx=0)
      (position_embeddings): Embedding(512, 32)
      (token_type_embeddings): Embedding(16, 32)
      (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
      (dropout): Dropout(p=0.1, inplace=False)
    )
    (encoder): BertEncoder(
      (layer): ModuleList(
        (0-4): 5 x BertLayer(
          (attention): BertAttention(
            (self): BertSelfAttention(
              (query): Linear(in_features=32, out_features=32, bias=True)
              (key): Linear(in_features=32, out_features=32, bias=True)
              (value): Linear(in_features=32, out_features=32, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
            (output): BertSelfOutput(
              (dense): Linear(in_features=32, out_features=32, bias=True)
              (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (intermediate): BertIntermediate(
            (dense): Linear(in_features=32, out_features=37, bias=True)
            (intermediate_act_fn): GELUActivation()
          )
          (output): BertOutput(
            (dense): Linear(in_features=37, out_features=32, bias=True)
            (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
            (dropout): Dropout(p=0.1, inplace=False)
          )
        )
      )
    )
    (pooler): BertPooler(
      (dense): Linear(in_features=32, out_features=32, bias=True)
      (activation): Tanh()
    )
  )
  (dropout): Dropout(p=0.1, inplace=False)
  (classifier): Linear(in_features=32, out_features=2, bias=True)
), test_model_config=None)
Raw output 
test/model/test_hf_model.py:190: in test_hf_config_io_config
    get_model_io_config.assert_called_once_with(self.model_name, self.task, olive_model.load_model())
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/unittest/mock.py:961: in assert_called_once_with
    return self.assert_called_with(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/unittest/mock.py:949: in assert_called_with
    raise AssertionError(_error_message()) from cause
E   AssertionError: expected call not found.
E   Expected: get_model_io_config('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', BertForSequenceClassification(
E     (bert): BertModel(
E       (embeddings): BertEmbeddings(
E         (word_embeddings): Embedding(1124, 32, padding_idx=0)
E         (position_embeddings): Embedding(512, 32)
E         (token_type_embeddings): Embedding(16, 32)
E         (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E         (dropout): Dropout(p=0.1, inplace=False)
E       )
E       (encoder): BertEncoder(
E         (layer): ModuleList(
E           (0-4): 5 x BertLayer(
E             (attention): BertAttention(
E               (self): BertSelfAttention(
E                 (query): Linear(in_features=32, out_features=32, bias=True)
E                 (key): Linear(in_features=32, out_features=32, bias=True)
E                 (value): Linear(in_features=32, out_features=32, bias=True)
E                 (dropout): Dropout(p=0.1, inplace=False)
E               )
E               (output): BertSelfOutput(
E                 (dense): Linear(in_features=32, out_features=32, bias=True)
E                 (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E                 (dropout): Dropout(p=0.1, inplace=False)
E               )
E             )
E             (intermediate): BertIntermediate(
E               (dense): Linear(in_features=32, out_features=37, bias=True)
E               (intermediate_act_fn): GELUActivation()
E             )
E             (output): BertOutput(
E               (dense): Linear(in_features=37, out_features=32, bias=True)
E               (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E               (dropout): Dropout(p=0.1, inplace=False)
E             )
E           )
E         )
E       )
E       (pooler): BertPooler(
E         (dense): Linear(in_features=32, out_features=32, bias=True)
E         (activation): Tanh()
E       )
E     )
E     (dropout): Dropout(p=0.1, inplace=False)
E     (classifier): Linear(in_features=32, out_features=2, bias=True)
E   ))
E     Actual: get_model_io_config('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', BertForSequenceClassification(
E     (bert): BertModel(
E       (embeddings): BertEmbeddings(
E         (word_embeddings): Embedding(1124, 32, padding_idx=0)
E         (position_embeddings): Embedding(512, 32)
E         (token_type_embeddings): Embedding(16, 32)
E         (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E         (dropout): Dropout(p=0.1, inplace=False)
E       )
E       (encoder): BertEncoder(
E         (layer): ModuleList(
E           (0-4): 5 x BertLayer(
E             (attention): BertAttention(
E               (self): BertSelfAttention(
E                 (query): Linear(in_features=32, out_features=32, bias=True)
E                 (key): Linear(in_features=32, out_features=32, bias=True)
E                 (value): Linear(in_features=32, out_features=32, bias=True)
E                 (dropout): Dropout(p=0.1, inplace=False)
E               )
E               (output): BertSelfOutput(
E                 (dense): Linear(in_features=32, out_features=32, bias=True)
E
... [The stack trace has been truncated as it exceeded the maximum allowed size. Please refer to the complete log available in the Test Run attachments for full details.]

Check failure on line 1 in test_hf_onnx_config_dummy_inputs

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

test_hf_onnx_config_dummy_inputs 

AssertionError: expected call not found.
Expected: get_model_dummy_input('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', model=<ANY>)
  Actual: get_model_dummy_input('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', model=BertForSequenceClassification(
  (bert): BertModel(
    (embeddings): BertEmbeddings(
      (word_embeddings): Embedding(1124, 32, padding_idx=0)
      (position_embeddings): Embedding(512, 32)
      (token_type_embeddings): Embedding(16, 32)
      (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
      (dropout): Dropout(p=0.1, inplace=False)
    )
    (encoder): BertEncoder(
      (layer): ModuleList(
        (0-4): 5 x BertLayer(
          (attention): BertAttention(
            (self): BertSelfAttention(
              (query): Linear(in_features=32, out_features=32, bias=True)
              (key): Linear(in_features=32, out_features=32, bias=True)
              (value): Linear(in_features=32, out_features=32, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
            (output): BertSelfOutput(
              (dense): Linear(in_features=32, out_features=32, bias=True)
              (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (intermediate): BertIntermediate(
            (dense): Linear(in_features=32, out_features=37, bias=True)
            (intermediate_act_fn): GELUActivation()
          )
          (output): BertOutput(
            (dense): Linear(in_features=37, out_features=32, bias=True)
            (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
            (dropout): Dropout(p=0.1, inplace=False)
          )
        )
      )
    )
    (pooler): BertPooler(
      (dense): Linear(in_features=32, out_features=32, bias=True)
      (activation): Tanh()
    )
  )
  (dropout): Dropout(p=0.1, inplace=False)
  (classifier): Linear(in_features=32, out_features=2, bias=True)
), test_model_config=None)
Raw output 
test/model/test_hf_model.py:217: in test_hf_onnx_config_dummy_inputs
    get_model_dummy_input.assert_called_once_with(self.model_name, self.task, model=ANY)
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/unittest/mock.py:961: in assert_called_once_with
    return self.assert_called_with(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/unittest/mock.py:949: in assert_called_with
    raise AssertionError(_error_message()) from cause
E   AssertionError: expected call not found.
E   Expected: get_model_dummy_input('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', model=<ANY>)
E     Actual: get_model_dummy_input('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', model=BertForSequenceClassification(
E     (bert): BertModel(
E       (embeddings): BertEmbeddings(
E         (word_embeddings): Embedding(1124, 32, padding_idx=0)
E         (position_embeddings): Embedding(512, 32)
E         (token_type_embeddings): Embedding(16, 32)
E         (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E         (dropout): Dropout(p=0.1, inplace=False)
E       )
E       (encoder): BertEncoder(
E         (layer): ModuleList(
E           (0-4): 5 x BertLayer(
E             (attention): BertAttention(
E               (self): BertSelfAttention(
E                 (query): Linear(in_features=32, out_features=32, bias=True)
E                 (key): Linear(in_features=32, out_features=32, bias=True)
E                 (value): Linear(in_features=32, out_features=32, bias=True)
E                 (dropout): Dropout(p=0.1, inplace=False)
E               )
E               (output): BertSelfOutput(
E                 (dense): Linear(in_features=32, out_features=32, bias=True)
E                 (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E                 (dropout): Dropout(p=0.1, inplace=False)
E               )
E             )
E             (intermediate): BertIntermediate(
E               (dense): Linear(in_features=32, out_features=37, bias=True)
E               (intermediate_act_fn): GELUActivation()
E             )
E             (output): BertOutput(
E               (dense): Linear(in_features=37, out_features=32, bias=True)
E               (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E               (dropout): Dropout(p=0.1, inplace=False)
E             )
E           )
E         )
E       )
E       (pooler): BertPooler(
E         (dense): Linear(in_features=32, out_features=32, bias=True)
E         (activation): Tanh()
E       )
E     )
E     (dropout): Dropout(p=0.1, inplace=False)
E     (classifier): Linear(in_features=32, out_features=2, bias=True)
E   ), test_model_config=None)

Check failure on line 1 in test_hf_config_io_config

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

test_hf_config_io_config 

AssertionError: expected call not found.
Expected: get_model_io_config('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', BertForSequenceClassification(
  (bert): BertModel(
    (embeddings): BertEmbeddings(
      (word_embeddings): Embedding(1124, 32, padding_idx=0)
      (position_embeddings): Embedding(512, 32)
      (token_type_embeddings): Embedding(16, 32)
      (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
      (dropout): Dropout(p=0.1, inplace=False)
    )
    (encoder): BertEncoder(
      (layer): ModuleList(
        (0-4): 5 x BertLayer(
          (attention): BertAttention(
            (self): BertSelfAttention(
              (query): Linear(in_features=32, out_features=32, bias=True)
              (key): Linear(in_features=32, out_features=32, bias=True)
              (value): Linear(in_features=32, out_features=32, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
            (output): BertSelfOutput(
              (dense): Linear(in_features=32, out_features=32, bias=True)
              (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (intermediate): BertIntermediate(
            (dense): Linear(in_features=32, out_features=37, bias=True)
            (intermediate_act_fn): GELUActivation()
          )
          (output): BertOutput(
            (dense): Linear(in_features=37, out_features=32, bias=True)
            (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
            (dropout): Dropout(p=0.1, inplace=False)
          )
        )
      )
    )
    (pooler): BertPooler(
      (dense): Linear(in_features=32, out_features=32, bias=True)
      (activation): Tanh()
    )
  )
  (dropout): Dropout(p=0.1, inplace=False)
  (classifier): Linear(in_features=32, out_features=2, bias=True)
))
  Actual: get_model_io_config('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', BertForSequenceClassification(
  (bert): BertModel(
    (embeddings): BertEmbeddings(
      (word_embeddings): Embedding(1124, 32, padding_idx=0)
      (position_embeddings): Embedding(512, 32)
      (token_type_embeddings): Embedding(16, 32)
      (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
      (dropout): Dropout(p=0.1, inplace=False)
    )
    (encoder): BertEncoder(
      (layer): ModuleList(
        (0-4): 5 x BertLayer(
          (attention): BertAttention(
            (self): BertSelfAttention(
              (query): Linear(in_features=32, out_features=32, bias=True)
              (key): Linear(in_features=32, out_features=32, bias=True)
              (value): Linear(in_features=32, out_features=32, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
            (output): BertSelfOutput(
              (dense): Linear(in_features=32, out_features=32, bias=True)
              (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (intermediate): BertIntermediate(
            (dense): Linear(in_features=32, out_features=37, bias=True)
            (intermediate_act_fn): GELUActivation()
          )
          (output): BertOutput(
            (dense): Linear(in_features=37, out_features=32, bias=True)
            (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
            (dropout): Dropout(p=0.1, inplace=False)
          )
        )
      )
    )
    (pooler): BertPooler(
      (dense): Linear(in_features=32, out_features=32, bias=True)
      (activation): Tanh()
    )
  )
  (dropout): Dropout(p=0.1, inplace=False)
  (classifier): Linear(in_features=32, out_features=2, bias=True)
), test_model_config=None)
Raw output 
test\model\test_hf_model.py:190: in test_hf_config_io_config
    get_model_io_config.assert_called_once_with(self.model_name, self.task, olive_model.load_model())
C:\ToolCache\Python\3.12.10\x64\Lib\unittest\mock.py:961: in assert_called_once_with
    return self.assert_called_with(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
C:\ToolCache\Python\3.12.10\x64\Lib\unittest\mock.py:949: in assert_called_with
    raise AssertionError(_error_message()) from cause
E   AssertionError: expected call not found.
E   Expected: get_model_io_config('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', BertForSequenceClassification(
E     (bert): BertModel(
E       (embeddings): BertEmbeddings(
E         (word_embeddings): Embedding(1124, 32, padding_idx=0)
E         (position_embeddings): Embedding(512, 32)
E         (token_type_embeddings): Embedding(16, 32)
E         (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E         (dropout): Dropout(p=0.1, inplace=False)
E       )
E       (encoder): BertEncoder(
E         (layer): ModuleList(
E           (0-4): 5 x BertLayer(
E             (attention): BertAttention(
E               (self): BertSelfAttention(
E                 (query): Linear(in_features=32, out_features=32, bias=True)
E                 (key): Linear(in_features=32, out_features=32, bias=True)
E                 (value): Linear(in_features=32, out_features=32, bias=True)
E                 (dropout): Dropout(p=0.1, inplace=False)
E               )
E               (output): BertSelfOutput(
E                 (dense): Linear(in_features=32, out_features=32, bias=True)
E                 (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E                 (dropout): Dropout(p=0.1, inplace=False)
E               )
E             )
E             (intermediate): BertIntermediate(
E               (dense): Linear(in_features=32, out_features=37, bias=True)
E               (intermediate_act_fn): GELUActivation()
E             )
E             (output): BertOutput(
E               (dense): Linear(in_features=37, out_features=32, bias=True)
E               (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E               (dropout): Dropout(p=0.1, inplace=False)
E             )
E           )
E         )
E       )
E       (pooler): BertPooler(
E         (dense): Linear(in_features=32, out_features=32, bias=True)
E         (activation): Tanh()
E       )
E     )
E     (dropout): Dropout(p=0.1, inplace=False)
E     (classifier): Linear(in_features=32, out_features=2, bias=True)
E   ))
E     Actual: get_model_io_config('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', BertForSequenceClassification(
E     (bert): BertModel(
E       (embeddings): BertEmbeddings(
E         (word_embeddings): Embedding(1124, 32, padding_idx=0)
E         (position_embeddings): Embedding(512, 32)
E         (token_type_embeddings): Embedding(16, 32)
E         (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E         (dropout): Dropout(p=0.1, inplace=False)
E       )
E       (encoder): BertEncoder(
E         (layer): ModuleList(
E           (0-4): 5 x BertLayer(
E             (attention): BertAttention(
E               (self): BertSelfAttention(
E                 (query): Linear(in_features=32, out_features=32, bias=True)
E                 (key): Linear(in_features=32, out_features=32, bias=True)
E                 (value): Linear(in_features=32, out_features=32, bias=True)
E                 (dropout): Dropout(p=0.1, inplace=False)
E               )
E               (output): BertSelfOutput(
E                 (dense): Linear(in_features=32, out_features=32, bias=True)
E                 (LayerNorm): LayerNor
... [The stack trace has been truncated as it exceeded the maximum allowed size. Please refer to the complete log available in the Test Run attachments for full details.]

Check failure on line 1 in test_hf_onnx_config_dummy_inputs

See this annotation in the file changed.

@azure-pipelines azure-pipelines / Olive CI

test_hf_onnx_config_dummy_inputs 

AssertionError: expected call not found.
Expected: get_model_dummy_input('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', model=<ANY>)
  Actual: get_model_dummy_input('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', model=BertForSequenceClassification(
  (bert): BertModel(
    (embeddings): BertEmbeddings(
      (word_embeddings): Embedding(1124, 32, padding_idx=0)
      (position_embeddings): Embedding(512, 32)
      (token_type_embeddings): Embedding(16, 32)
      (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
      (dropout): Dropout(p=0.1, inplace=False)
    )
    (encoder): BertEncoder(
      (layer): ModuleList(
        (0-4): 5 x BertLayer(
          (attention): BertAttention(
            (self): BertSelfAttention(
              (query): Linear(in_features=32, out_features=32, bias=True)
              (key): Linear(in_features=32, out_features=32, bias=True)
              (value): Linear(in_features=32, out_features=32, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
            (output): BertSelfOutput(
              (dense): Linear(in_features=32, out_features=32, bias=True)
              (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
          (intermediate): BertIntermediate(
            (dense): Linear(in_features=32, out_features=37, bias=True)
            (intermediate_act_fn): GELUActivation()
          )
          (output): BertOutput(
            (dense): Linear(in_features=37, out_features=32, bias=True)
            (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
            (dropout): Dropout(p=0.1, inplace=False)
          )
        )
      )
    )
    (pooler): BertPooler(
      (dense): Linear(in_features=32, out_features=32, bias=True)
      (activation): Tanh()
    )
  )
  (dropout): Dropout(p=0.1, inplace=False)
  (classifier): Linear(in_features=32, out_features=2, bias=True)
), test_model_config=None)
Raw output 
test\model\test_hf_model.py:217: in test_hf_onnx_config_dummy_inputs
    get_model_dummy_input.assert_called_once_with(self.model_name, self.task, model=ANY)
C:\ToolCache\Python\3.12.10\x64\Lib\unittest\mock.py:961: in assert_called_once_with
    return self.assert_called_with(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
C:\ToolCache\Python\3.12.10\x64\Lib\unittest\mock.py:949: in assert_called_with
    raise AssertionError(_error_message()) from cause
E   AssertionError: expected call not found.
E   Expected: get_model_dummy_input('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', model=<ANY>)
E     Actual: get_model_dummy_input('hf-internal-testing/tiny-random-BertForSequenceClassification', 'text-classification', model=BertForSequenceClassification(
E     (bert): BertModel(
E       (embeddings): BertEmbeddings(
E         (word_embeddings): Embedding(1124, 32, padding_idx=0)
E         (position_embeddings): Embedding(512, 32)
E         (token_type_embeddings): Embedding(16, 32)
E         (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E         (dropout): Dropout(p=0.1, inplace=False)
E       )
E       (encoder): BertEncoder(
E         (layer): ModuleList(
E           (0-4): 5 x BertLayer(
E             (attention): BertAttention(
E               (self): BertSelfAttention(
E                 (query): Linear(in_features=32, out_features=32, bias=True)
E                 (key): Linear(in_features=32, out_features=32, bias=True)
E                 (value): Linear(in_features=32, out_features=32, bias=True)
E                 (dropout): Dropout(p=0.1, inplace=False)
E               )
E               (output): BertSelfOutput(
E                 (dense): Linear(in_features=32, out_features=32, bias=True)
E                 (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E                 (dropout): Dropout(p=0.1, inplace=False)
E               )
E             )
E             (intermediate): BertIntermediate(
E               (dense): Linear(in_features=32, out_features=37, bias=True)
E               (intermediate_act_fn): GELUActivation()
E             )
E             (output): BertOutput(
E               (dense): Linear(in_features=37, out_features=32, bias=True)
E               (LayerNorm): LayerNorm((32,), eps=1e-12, elementwise_affine=True, bias=True)
E               (dropout): Dropout(p=0.1, inplace=False)
E             )
E           )
E         )
E       )
E       (pooler): BertPooler(
E         (dense): Linear(in_features=32, out_features=32, bias=True)
E         (activation): Tanh()
E       )
E     )
E     (dropout): Dropout(p=0.1, inplace=False)
E     (classifier): Linear(in_features=32, out_features=2, bias=True)
E   ), test_model_config=None)
View more details on Azure Pipelines