Skip to content

Create new test type aamtest for accessibility API testing #57696

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
spectranaut wants to merge 38 commits into
base: master
Choose a base branch
from

Fix tables and links

d4fca40
Select commit
Loading
Failed to load commit list.
Open

Create new test type aamtest for accessibility API testing #57696

Fix tables and links
d4fca40
Select commit
Loading
Failed to load commit list.
Community-TC Integration / wpt-firefox-nightly-results succeeded Apr 28, 2026 in 18m 5s

Community-TC (pull_request)

Collect results for all tests affected by a pull request in firefox.

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2026-04-28T20:22:46.119Z
Resolved: 2026-04-28T20:26:22.046Z
Task Execution Time: 3 minutes, 35 seconds, 927 milliseconds
Task Status: completed
Reason Resolved: completed
TaskId: Gh3KDFMZRtONEqJqBkZ6QQ
RunId: 0

Artifacts

- public/logs/live_backing.log
- public/logs/live.log
- public/results/checkrun.md
- public/results/wpt_report.json.gz
- public/results/wpt_screenshot.txt.gz

WPT Command: python3 ./wpt run --channel=nightly --no-fail-on-unexpected --log-wptreport=../artifacts/wpt_report.json --log-wptscreenshot=../artifacts/wpt_screenshot.txt --affected base_head --log-mach-level=info --log-mach=- -y --no-pause --no-restart-on-unexpected --install-fonts --verify-log-full --no-headless --binary=/home/test/build/firefox/firefox firefox

[taskcluster 2026-04-28 20:22:46.207Z] Task ID: Gh3KDFMZRtONEqJqBkZ6QQ
[taskcluster 2026-04-28 20:22:46.207Z] Worker ID: 7309133531607512496
[taskcluster 2026-04-28 20:22:46.207Z] Worker Group: us-east1-b
[taskcluster 2026-04-28 20:22:46.207Z] Worker Node Type: projects/757942385826/machineTypes/n2-standard-4
[taskcluster 2026-04-28 20:22:46.207Z] Worker Pool: proj-wpt/ci
[taskcluster 2026-04-28 20:22:46.207Z] Worker Version: 46.1.0
[taskcluster 2026-04-28 20:22:46.207Z] Public IP: 34.73.20.8
[taskcluster 2026-04-28 20:22:46.207Z] Hostname: proj-wpt-ci-fio7qil3s3-6fudreqr5aw

[taskcluster 2026-04-28 20:22:47.297Z] === Task Starting ===
+ '[' '' ']'
+ '[' -d /etc/profile.d ']'
+ for i in /etc/profile.d/*.sh
+ '[' -r /etc/profile.d/01-locale-fix.sh ']'
+ . /etc/profile.d/01-locale-fix.sh
+++ /usr/bin/locale-check C.UTF-8
++ eval
+ for i in /etc/profile.d/*.sh
+ '[' -r /etc/profile.d/apps-bin-path.sh ']'
+ . /etc/profile.d/apps-bin-path.sh

...(410 lines hidden)...

 1:40.31 SUITE_START: web-platform-test - running 7 tests
mem avail: 14168 of 15806 MiB (89.64%), swap free:    0 of    0 MiB ( 0.00%)
 2:01.49 INFO Using 1 child processes
 2:01.51 INFO Setting up ssl
 2:01.53 certutil Full command: /usr/bin/certutil -N -d /tmp/tmp8oyn1e5v.mozrunner -f /tmp/tmp8oyn1e5v.mozrunner/.crtdbpw
certutil b''
 2:01.56 certutil b''
 2:01.56 certutil b'\nCertificate Nickname                                         Trust Attributes\n                                                             SSL,S/MIME,JAR/XPI\n\nweb-platform-tests                                           CT,, \n'
 2:01.56 INFO Starting WebDriver: /home/test/web-platform-tests/_venv3/bin/geckodriver --host 127.0.0.1 --port 0 --binary /home/test/build/firefox/firefox
 2:01.68 pid:2059 Full command: /home/test/web-platform-tests/_venv3/bin/geckodriver --host 127.0.0.1 --port 0 --binary /home/test/build/firefox/firefox
pid:2059 1777407958077	geckodriver	INFO	Listening on 127.0.0.1:38749
 2:01.68 INFO Webdriver started successfully.
 2:01.68 INFO Starting runner
 2:02.04 TEST_START: /core-aam/aamtests/attribute/aria_autocomplete_inline_list_both.py
 2:02.42 INFO STDOUT: ============================= test session starts ==============================
 2:02.42 INFO STDOUT: platform linux -- Python 3.10.12, pytest-8.2.1, pluggy-unknown -- /usr/bin/python3
 2:02.42 INFO STDOUT: rootdir: /home/test/web-platform-tests
 2:02.42 INFO STDOUT: plugins: asyncio-0.19.0
 2:02.42 INFO STDOUT: asyncio: mode=strict
 2:02.42 INFO STDOUT: collecting ... 
 2:02.48 INFO STDOUT: collected 3 items
 2:02.48 INFO STDOUT: core-aam/aamtests/attribute/aria_autocomplete_inline_list_both.py::test_atspi[both] 
 2:02.48 pid:2059 1777407958983	mozrunner::runner	INFO	Running command: MOZ_CRASHREPORTER="1" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_CRASHREPORTER_SHUTDOWN="1" "/home/test/build/firefox/firefox" "--marionette" "--profile" "/tmp/tmp8oyn1e5v.mozrunner" "-no-remote"
 2:02.49 pid:2059 [2077] Sandbox: CanCreateUserNamespace() clone() failure: EPERM
 2:03.01 pid:2059 1777407959504	Marionette	INFO	Marionette enabled
 2:03.12 pid:2059 1777407959615	Marionette	INFO	Listening on port 36473
 2:03.19 pid:2059 Read port: 36473
Failed to read: session.screen0.titlebar.left
Setting default value
Failed to read: session.screen0.titlebar.right
Setting default value
 2:07.02 INFO STDOUT: PASSED
 2:07.05 INFO STDOUT: core-aam/aamtests/attribute/aria_autocomplete_inline_list_both.py::test_atspi[inline] 
 2:07.09 INFO STDOUT: PASSED
 2:07.12 INFO STDOUT: core-aam/aamtests/attribute/aria_autocomplete_inline_list_both.py::test_atspi[list] 
 2:07.15 INFO STDOUT: PASSED
 2:07.17 pid:2059 1777407963670	Marionette	INFO	Stopped listening on port 36473
 2:07.63 pid:2059 JavaScript error: resource://gre/modules/PrivateBrowsingUtils.sys.mjs, line 50: TypeError: can't access property "QueryInterface", aWindow.docShell is null
 2:07.70 INFO STDOUT: ============================== 3 passed in 5.28s ===============================
 2:07.70 TEST_END: Test OK. Subtests passed 3/3. Unexpected 0
 2:07.70 TEST_START: /core-aam/aamtests/attribute/aria_braillelabel.py
 2:07.74 INFO STDOUT: ============================= test session starts ==============================
 2:07.74 INFO STDOUT: platform linux -- Python 3.10.12, pytest-8.2.1, pluggy-unknown -- /usr/bin/python3
 2:07.74 INFO STDOUT: rootdir: /home/test/web-platform-tests
 2:07.74 INFO STDOUT: plugins: asyncio-0.19.0
 2:07.74 INFO STDOUT: asyncio: mode=strict
 2:07.74 INFO STDOUT: collecting ... 
 2:07.79 INFO STDOUT: collected 1 item
 2:07.79 INFO STDOUT: core-aam/aamtests/attribute/aria_braillelabel.py::test_atspi 
 2:07.80 pid:2059 1777407964301	mozrunner::runner	INFO	Running command: MOZ_CRASHREPORTER="1" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_CRASHREPORTER_SHUTDOWN="1" "/home/test/build/firefox/firefox" "--marionette" "--profile" "/tmp/tmp8oyn1e5v.mozrunner" "-no-remote"
 2:07.81 pid:2059 [2391] Sandbox: CanCreateUserNamespace() clone() failure: EPERM
 2:08.18 pid:2059 1777407964679	Marionette	INFO	Marionette enabled
 2:08.22 pid:2059 1777407964715	Marionette	INFO	Listening on port 33461
 2:08.30 pid:2059 Read port: 33461
 2:09.43 INFO STDOUT: PASSED
 2:09.51 pid:2059 1777407966006	Marionette	INFO	Stopped listening on port 33461
 2:09.95 INFO STDOUT: ============================== 1 passed in 2.21s ===============================
 2:09.95 TEST_END: Test OK. Subtests passed 1/1. Unexpected 0
 2:09.95 TEST_START: /core-aam/aamtests/attribute/aria_error_message.py
 2:09.99 INFO STDOUT: ============================= test session starts ==============================
 2:09.99 INFO STDOUT: platform linux -- Python 3.10.12, pytest-8.2.1, pluggy-unknown -- /usr/bin/python3
 2:09.99 INFO STDOUT: rootdir: /home/test/web-platform-tests
 2:09.99 INFO STDOUT: plugins: asyncio-0.19.0
 2:09.99 INFO STDOUT: asyncio: mode=strict
 2:09.99 INFO STDOUT: collecting ... 
 2:10.04 INFO STDOUT: collected 1 item
 2:10.04 INFO STDOUT: core-aam/aamtests/attribute/aria_error_message.py::test_atspi 
 2:10.05 pid:2059 1777407966548	mozrunner::runner	INFO	Running command: MOZ_CRASHREPORTER="1" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_CRASHREPORTER_SHUTDOWN="1" "/home/test/build/firefox/firefox" "--marionette" "--profile" "/tmp/tmp8oyn1e5v.mozrunner" "-no-remote"
 2:10.05 pid:2059 [2704] Sandbox: CanCreateUserNamespace() clone() failure: EPERM
 2:10.44 pid:2059 1777407966941	Marionette	INFO	Marionette enabled
 2:10.48 pid:2059 1777407966979	Marionette	INFO	Listening on port 40897
 2:10.55 pid:2059 Read port: 40897
 2:11.53 INFO STDOUT: PASSED
 2:11.56 pid:2059 1777407968054	Marionette	INFO	Stopped listening on port 40897
 2:11.83 INFO STDOUT: ============================== 1 passed in 1.84s ===============================
 2:11.83 TEST_END: Test OK. Subtests passed 1/1. Unexpected 0
 2:11.83 TEST_START: /core-aam/aamtests/role/blockquote.py
 2:11.87 INFO STDOUT: ============================= test session starts ==============================
 2:11.87 INFO STDOUT: platform linux -- Python 3.10.12, pytest-8.2.1, pluggy-unknown -- /usr/bin/python3
 2:11.87 INFO STDOUT: rootdir: /home/test/web-platform-tests
 2:11.87 INFO STDOUT: plugins: asyncio-0.19.0
 2:11.87 INFO STDOUT: asyncio: mode=strict
 2:11.87 INFO STDOUT: collecting ... 
 2:11.93 INFO STDOUT: collected 3 items
 2:11.93 INFO STDOUT: core-aam/aamtests/role/blockquote.py::test_atspi 
 2:11.94 pid:2059 1777407968434	mozrunner::runner	INFO	Running command: MOZ_CRASHREPORTER="1" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_CRASHREPORTER_SHUTDOWN="1" "/home/test/build/firefox/firefox" "--marionette" "--profile" "/tmp/tmp8oyn1e5v.mozrunner" "-no-remote"
 2:11.94 pid:2059 [2943] Sandbox: CanCreateUserNamespace() clone() failure: EPERM
 2:12.32 pid:2059 1777407968818	Marionette	INFO	Marionette enabled
 2:12.38 pid:2059 1777407968880	Marionette	INFO	Listening on port 45101
 2:12.44 pid:2059 Read port: 45101
 2:13.44 INFO STDOUT: PASSED
 2:13.48 INFO STDOUT: core-aam/aamtests/role/blockquote.py::test_axapi 
 2:13.49 INFO STDOUT: SKIPPED
 2:13.52 INFO STDOUT: core-aam/aamtests/role/blockquote.py::test_ia2 
 2:13.53 INFO STDOUT: SKIPPED
 2:13.53 INFO STDOUT:  (NOT_APPLICABLE)
 2:13.56 pid:2059 1777407970055	Marionette	INFO	Stopped listening on port 45101
 2:13.82 INFO STDOUT: ========================= 1 passed, 2 skipped in 1.95s =========================
 2:13.82 TEST_END: Test OK. Subtests passed 1/1. Unexpected 0
 2:13.82 TEST_START: /core-aam/aamtests/role/button.py
 2:13.86 INFO STDOUT: ============================= test session starts ==============================
 2:13.86 INFO STDOUT: platform linux -- Python 3.10.12, pytest-8.2.1, pluggy-unknown -- /usr/bin/python3
 2:13.86 INFO STDOUT: rootdir: /home/test/web-platform-tests
 2:13.86 INFO STDOUT: plugins: asyncio-0.19.0
 2:13.86 INFO STDOUT: asyncio: mode=strict
 2:13.86 INFO STDOUT: collecting ... 
 2:13.91 INFO STDOUT: collected 4 items
 2:13.91 INFO STDOUT: core-aam/aamtests/role/button.py::test_atspi[no-attributes] 
 2:13.92 pid:2059 1777407970419	mozrunner::runner	INFO	Running command: MOZ_CRASHREPORTER="1" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_CRASHREPORTER_SHUTDOWN="1" "/home/test/build/firefox/firefox" "--marionette" "--profile" "/tmp/tmp8oyn1e5v.mozrunner" "-no-remote"
 2:13.92 pid:2059 [3184] Sandbox: CanCreateUserNamespace() clone() failure: EPERM
 2:14.30 pid:2059 1777407970799	Marionette	INFO	Marionette enabled
 2:14.34 pid:2059 1777407970837	Marionette	INFO	Listening on port 40675
 2:14.42 pid:2059 Read port: 40675
 2:15.33 INFO STDOUT: PASSED
 2:15.36 INFO STDOUT: core-aam/aamtests/role/button.py::test_atspi[aria-pressed-undefined] 
 2:15.41 INFO STDOUT: PASSED
 2:15.44 INFO STDOUT: core-aam/aamtests/role/button.py::test_atspi[aria-haspopup-undefined] 
 2:15.48 INFO STDOUT: PASSED
 2:15.50 INFO STDOUT: core-aam/aamtests/role/button.py::test_atspi[aria-haspopup-false] 
 2:15.55 INFO STDOUT: PASSED
 2:15.57 pid:2059 1777407972069	Marionette	INFO	Stopped listening on port 40675
 2:15.83 INFO STDOUT: ============================== 4 passed in 1.97s ===============================
 2:15.83 TEST_END: Test OK. Subtests passed 4/4. Unexpected 0
 2:15.83 TEST_START: /core-aam/aamtests/role/button_haspopup.py
 2:15.88 INFO STDOUT: ============================= test session starts ==============================
 2:15.88 INFO STDOUT: platform linux -- Python 3.10.12, pytest-8.2.1, pluggy-unknown -- /usr/bin/python3
 2:15.88 INFO STDOUT: rootdir: /home/test/web-platform-tests
 2:15.88 INFO STDOUT: plugins: asyncio-0.19.0
 2:15.88 INFO STDOUT: asyncio: mode=strict
 2:15.88 INFO STDOUT: collecting ... 
 2:15.93 INFO STDOUT: collected 6 items
 2:15.93 INFO STDOUT: core-aam/aamtests/role/button_haspopup.py::test_atspi[true] 
 2:15.94 pid:2059 1777407972435	mozrunner::runner	INFO	Running command: MOZ_CRASHREPORTER="1" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_CRASHREPORTER_SHUTDOWN="1" "/home/test/build/firefox/firefox" "--marionette" "--profile" "/tmp/tmp8oyn1e5v.mozrunner" "-no-remote"
 2:15.94 pid:2059 [3424] Sandbox: CanCreateUserNamespace() clone() failure: EPERM
 2:16.32 pid:2059 1777407972817	Marionette	INFO	Marionette enabled
 2:16.36 pid:2059 1777407972856	Marionette	INFO	Listening on port 41077
 2:16.44 pid:2059 Read port: 41077
 2:17.37 INFO STDOUT: PASSED
 2:17.40 INFO STDOUT: core-aam/aamtests/role/button_haspopup.py::test_atspi[menu] 
 2:17.47 INFO STDOUT: PASSED
 2:17.49 INFO STDOUT: core-aam/aamtests/role/button_haspopup.py::test_atspi[listbox] 
 2:17.54 INFO STDOUT: PASSED
 2:17.56 INFO STDOUT: core-aam/aamtests/role/button_haspopup.py::test_atspi[tree] 
 2:17.64 INFO STDOUT: PASSED
 2:17.67 INFO STDOUT: core-aam/aamtests/role/button_haspopup.py::test_atspi[grid] 
 2:17.73 INFO STDOUT: PASSED
 2:17.75 INFO STDOUT: core-aam/aamtests/role/button_haspopup.py::test_atspi[dialog] 
 2:17.80 INFO STDOUT: PASSED
 2:17.82 pid:2059 1777407974320	Marionette	INFO	Stopped listening on port 41077
 2:18.09 INFO STDOUT: ============================== 6 passed in 2.22s ===============================
 2:18.09 TEST_END: Test OK. Subtests passed 6/6. Unexpected 0
 2:18.09 TEST_START: /core-aam/aamtests/role/button_pressed.py
 2:18.13 INFO STDOUT: ============================= test session starts ==============================
 2:18.13 INFO STDOUT: platform linux -- Python 3.10.12, pytest-8.2.1, pluggy-unknown -- /usr/bin/python3
 2:18.13 INFO STDOUT: rootdir: /home/test/web-platform-tests
 2:18.13 INFO STDOUT: plugins: asyncio-0.19.0
 2:18.13 INFO STDOUT: asyncio: mode=strict
 2:18.13 INFO STDOUT: collecting ... 
 2:18.18 INFO STDOUT: collected 2 items
 2:18.18 INFO STDOUT: core-aam/aamtests/role/button_pressed.py::test_atspi[true] 
 2:18.19 pid:2059 1777407974690	mozrunner::runner	INFO	Running command: MOZ_CRASHREPORTER="1" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_CRASHREPORTER_SHUTDOWN="1" "/home/test/build/firefox/firefox" "--marionette" "--profile" "/tmp/tmp8oyn1e5v.mozrunner" "-no-remote"
 2:18.19 pid:2059 [3744] Sandbox: CanCreateUserNamespace() clone() failure: EPERM
 2:18.58 pid:2059 1777407975080	Marionette	INFO	Marionette enabled
 2:18.62 pid:2059 1777407975118	Marionette	INFO	Listening on port 33297
 2:18.69 pid:2059 Read port: 33297
 2:19.62 INFO STDOUT: PASSED
 2:19.65 INFO STDOUT: core-aam/aamtests/role/button_pressed.py::test_atspi[false] 
 2:19.69 INFO STDOUT: PASSED
 2:19.71 pid:2059 1777407976210	Marionette	INFO	Stopped listening on port 33297
 2:19.97 INFO STDOUT: ============================== 2 passed in 1.84s ===============================
 2:19.98 TEST_END: Test OK. Subtests passed 2/2. Unexpected 0
 2:19.98 INFO No more tests
 2:19.98 INFO Closing logging queue
 2:19.98 INFO queue closed
 2:20.11 SUITE_END

web-platform-test
~~~~~~~~~~~~~~~~~
Ran 25 checks (18 subtests, 7 tests)
Expected results: 25
Unexpected results: 0
OK
 2:20.11 INFO Got 0 unexpected results, with 0 unexpected passes
 2:20.11 wptserve INFO Stopped http server on 127.0.0.1:8000
 2:20.11 wptserve INFO Stopped http server on 127.0.0.1:8001
 2:20.12 wptserve INFO Stopped http server on 127.0.0.1:8003
 2:20.12 wptserve INFO Stopped http server on 127.0.0.1:8002
 2:20.13 wptserve INFO Stopped http server on 127.0.0.1:8443
 2:20.13 wptserve INFO Stopped http server on 127.0.0.1:8444
 2:20.13 wptserve INFO Stopped http server on 127.0.0.1:8445
 2:20.14 wptserve INFO Stopped http server on 127.0.0.1:8446
 2:20.14 wptserve INFO Stopped WebTransport over HTTP/3 server on 127.0.0.1:11000
 2:20.14 wptserve INFO Stopped http server on 127.0.0.1:9000
 2:20.36 wptserve INFO Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8888))
 2:20.36 wptserve INFO Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8889))
 2:20.41 INFO Removed font: Ahem.ttf
 2:20.43 INFO Closing logging queue
 2:20.43 INFO queue closed
[taskcluster 2026-04-28 20:26:19.808Z] === Task Finished ===
[taskcluster 2026-04-28 20:26:20.943Z] Successful task run with exit code: 0 completed in 214.737 seconds
View more details on Community-TC Integration