Skip to content

Playwright migration from Cypress#18987

Merged
Skrypt merged 75 commits intomainfrom
skrypt/playwright
Mar 28, 2026
Merged

Playwright migration from Cypress#18987
Skrypt merged 75 commits intomainfrom
skrypt/playwright

Conversation

@Skrypt
Copy link
Copy Markdown
Contributor

@Skrypt Skrypt commented Mar 11, 2026

Summary

  • Migrate functional tests from Cypress (TypeScript/Node.js) to Playwright (C#), eliminating the JavaScript toolchain dependency and unifying the test stack under .NET
  • Remove all Cypress infrastructure: cypress-commands package, npm dependencies (package.json, package-lock.json), JS test files, and the custom test_generator.js
  • Add C# Playwright test helpers: OrchardTestFixture, OrchardTestServer, EmbeddedRecipeHarvester, TenantHelper, AuthHelper, FeatureHelper, ButtonHelper, SelectorHelper, ConfigurationHelper, and TestUtils
  • Serve test recipes from embedded resources via EmbeddedRecipeHarvester (implements IRecipeHarvester), eliminating shared filesystem state — any .recipe.json added to Fixtures/ is discovered automatically
  • Enable parallel test execution with full fixture isolation: per-fixture databases (dropped and recreated for clean state), env vars captured once at process start via CaptureAndClear and injected per-host through builder.Configuration and tenants.json
  • Fix SaaS tenant setup on external databases: table prefix was skipped when DatabaseConfigurationPreset hid the provider dropdown — now set independently of dropdown visibility
  • Fix SharedViewCompilerProvider static-to-instance field change to prevent view cache corruption when multiple test hosts run in-process
  • Replace custom InMemoryLoggerProvider with FakeLoggerProvider from Microsoft.Extensions.Diagnostics.Testing
  • Update all CI workflows (main_ci, pr_ci, preview_ci, release_ci, functional_all_db): remove Node.js setup steps, add Playwright browser caching, align artifact upload paths with per-fixture App_Data_Tests_* directories, remove dead ORCHARD_APP env var, and add Docker Hub credentials for service container image pulls
  • Add run-db-tests.sh helper script to run functional tests locally against all database providers via Docker

Test plan

  • All CMS recipe tests pass (Agency, Blog, ComingSoon, Headless, Migrations)
  • SaaS multi-tenant tests pass (including tenant creation with table prefix on external databases)
  • MVC tests pass
  • All 48 tests pass across all 4 databases locally (SQLite, PostgreSQL, MySQL, SQL Server) via run-db-tests.sh all
  • Run functional_all_db CI workflow to confirm parallel execution across database providers
  • Confirm Playwright browser caching works in CI
  • Confirm no regressions in non-test code (only SharedViewCompilerProvider static→instance change)

🤖 Generated with Claude Code

Refactor all Cypress functional tests in a 1:1 migration to Playwright.
@Skrypt Skrypt requested a review from sebastienros March 11, 2026 19:30
The CI runners have Yarn 1.x globally but the project uses Yarn 4.9.4
via packageManager field. Adding corepack enable ensures the correct
Yarn version is used. Also removes unnecessary cd into subdirectory
for functional_all_db.yml since yarn workspace commands run from root.
@hishamco
Copy link
Copy Markdown
Member

I did a lot of work on Playwright previously in Orchard Core Contrib (OCC). Moreover, I attempted to utilize Playwright from xUnit, which is much easier than TypeScript

Thanks for this initiative, it could be a good start before integrating Playwright using .NET

@Skrypt
Copy link
Copy Markdown
Contributor Author

Skrypt commented Mar 11, 2026

Nothing prevents from having both.

@Skrypt Skrypt marked this pull request as ready for review March 11, 2026 23:35
@Piedone
Copy link
Copy Markdown
Member

Piedone commented Mar 12, 2026

Why not C# instead of TS?

@Skrypt
Copy link
Copy Markdown
Contributor Author

Skrypt commented Mar 12, 2026

I could say because it's a 1:1 migration but that would not be a strong argument. As I said previously to Hisham, nothing prevents using both. For some reason, C# is always a strong point but for this migration it seemed easier to simply move it to its equivalent.

@Skrypt
Copy link
Copy Markdown
Contributor Author

Skrypt commented Mar 12, 2026

In general, TypeScript is considered the superior choice for Playwright because the tool was built natively for the Node.js ecosystem. However, C# is a strong alternative if your team is already deeply integrated into the .NET platform. [1, 2, 3, 4]
Why Choose TypeScript?

  • Feature Priority: New features, such as the UI Mode, built-in HTML reporters, and advanced VS Code extension capabilities, typically debut in TypeScript first.
  • Native Test Runner: The playwright-test runner is purpose-built for TypeScript, offering a seamless "all-in-one" experience with a single configuration file (playwright.config.ts).
  • Community and Docs: Most online tutorials, StackOverflow answers, and the majority of official documentation examples favor TypeScript.
  • Execution Speed: Tests often run slightly faster in the Node.js environment compared to the .NET port. [3, 4, 5, 6, 7, 8, 9, 10]

Why Choose C#?

  • Team Alignment: If your developers and QAs are already C# experts, using Playwright for .NET allows them to leverage existing skills, share utility code, and perform easier code reviews.
  • Enterprise Integration: It integrates natively with [MSTest, NUnit, or xUnit](https://playwright.dev/dotnet/docs/test-runners), making it easier to fit into existing .NET CI/CD pipelines and Visual Studio workflows.
  • Language Power: C# provides robust static typing and features like nullable reference types that some developers find more powerful for large-scale automation architectures. [1, 11, 12, 13, 14, 15]

Key Comparison

Feature [3, 5, 6, 7, 12, 14] TypeScript C# (.NET)
Release Cycle First to receive updates Slightly behind TS
Setup Fast, out-of-the-box runner Requires standard .NET test runners
Reporting Integrated, rich HTML reports Often requires 3rd-party tools
Ecosystem Massive web-centric community Strong enterprise support

Verdict: Use TypeScript for the most "complete" Playwright experience with the latest features. Choose C# if you want your tests to live in the same world as your .NET backend or existing C# test suites. [1, 16, 17, 18]
Does your team have a specific CI/CD pipeline (like Azure DevOps or GitHub Actions) where these tests will be running?

[1] [https://club.ministryoftesting.com](https://club.ministryoftesting.com/t/is-playwright-c-worth-using/83635)
[2] [https://ray.run](https://ray.run/discord-forum/threads/92311-net-vs-typescript)
[3] [https://ray.run](https://ray.run/discord-forum/threads/92311-net-vs-typescript)
[4] [https://www.linkedin.com](https://www.linkedin.com/pulse/typescript-vs-c-playwright-real-insights-from-test-automation-yang-wjxhc)
[5] [https://ray.run](https://ray.run/discord-forum/threads/92311-net-vs-typescript)
[6] [https://www.linkedin.com](https://www.linkedin.com/pulse/typescript-vs-c-playwright-real-insights-from-test-automation-yang-wjxhc)
[7] [https://www.reddit.com](https://www.reddit.com/r/QualityAssurance/comments/1ggtncg/playwright_with_c_or_other_language/)
[8] [https://www.reddit.com](https://www.reddit.com/r/QualityAssurance/comments/1ggtncg/playwright_with_c_or_other_language/)
[9] [https://club.ministryoftesting.com](https://club.ministryoftesting.com/t/is-playwright-c-worth-using/83635)
[10] [https://club.ministryoftesting.com](https://club.ministryoftesting.com/t/is-playwright-c-worth-using/83635/2)
[11] [https://www.youtube.com](https://www.youtube.com/watch?v=gm66RISO__M)
[12] [https://playwright.dev](https://playwright.dev/dotnet/docs/test-runners)
[13] [https://playwright.dev](https://playwright.dev/dotnet/docs/writing-tests#:~:text=Test%20Isolation%E2%80%8B%20The%20Playwright%20NUnit%2C%20MSTest%2C%20xUnit%2C,multiple%20tests%20run%20in%20a%20single%20Browser.)
[14] [https://www.linkedin.com](https://www.linkedin.com/pulse/optimal-language-choices-playwright-why-javascript-typescript-mane-x80ff)
[15] [https://news.ycombinator.com](https://news.ycombinator.com/item?id=27460329)
[16] [https://club.ministryoftesting.com](https://club.ministryoftesting.com/t/is-playwright-c-worth-using/83635)
[17] [https://testdriver.ai](https://testdriver.ai/articles/is-playwright-c-the-right-choice-for-your-testing-needs#:~:text=Full%20Functionality%20and%20Feature%20Parity%20One%20of,browsers%2C%20and%20utilize%20the%20same%20API%20capabilities.)
[18] [https://www.reddit.com](https://www.reddit.com/r/Playwright/comments/1jb29zu/is_playwright_the_best_alternative_to_selenium_in/)

@Piedone
Copy link
Copy Markdown
Member

Piedone commented Mar 12, 2026

Playwright has full support for C#, and the OC codebase is largely C#. These tests are not client-side code, so I don't see why we'd then go with TS, as opposed to all the other tests, which are also C# (and using xUnit). The AI-generated arguments don't really stand here.

@Skrypt
Copy link
Copy Markdown
Contributor Author

Skrypt commented Mar 12, 2026

I'm using it for testing the new media gallery. Which makes sense since it is a VueJs application. I don't see any issue having both kind of tests. My goal is to not waste too much time on refactoring these tests in C# either. I just want to have tests for TUS protocol "resuming uploads" in the Media Gallery. I think, it is just a question of tastes which I won't debate on. Playwright allows for both, I do not have a favor between C# or Typescript. I just know that the tool is built with Typescript first so to me it makes sense having all kinds of tests with it.

@sebastienros
Copy link
Copy Markdown
Member

I think we should try to migrate this PR to C#, if not just to simplify how people run these tests, and hopefully write some (nobody has written a JS one in 10 years)

@Skrypt
Copy link
Copy Markdown
Contributor Author

Skrypt commented Mar 12, 2026

@copilot Can you refactor this PR by using C# for the functional test instead. You can rename the project .csproj and folder to OrchardCore.Tests.Functional

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 12, 2026

@Skrypt I've opened a new pull request, #18991, to work on those changes. Once the pull request is ready, I'll request review from you.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates the repository’s end-to-end functional test suite from Cypress to Playwright, updating local tooling and CI workflows to run the new Playwright-based tests.

Changes:

  • Added a new Playwright workspace (test/OrchardCore.Tests.Playwright) with configs, helpers, fixtures, and initial CMS/MVC specs.
  • Removed the legacy Cypress functional test suite and its custom Cypress command package.
  • Updated docs, solution/workspaces, gitignore, and GitHub Actions workflows to run Playwright tests (and moved Node to v20 in CI).

Reviewed changes

Copilot reviewed 67 out of 80 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
yarn.lock Adds Playwright dependencies and workspace entry.
test/OrchardCore.Tests.Playwright/tsconfig.json TypeScript config for Playwright tests/helpers.
test/OrchardCore.Tests.Playwright/tests/mvc/mvc.spec.ts MVC smoke spec migrated to Playwright.
test/OrchardCore.Tests.Playwright/tests/cms/saas.spec.ts SaaS recipe functional specs in Playwright.
test/OrchardCore.Tests.Playwright/tests/cms/saas-setup.spec.ts Default tenant SaaS setup spec for CMS run ordering.
test/OrchardCore.Tests.Playwright/tests/cms/migrations.spec.ts Migrations recipe specs in Playwright.
test/OrchardCore.Tests.Playwright/tests/cms/headless.spec.ts Headless recipe specs in Playwright.
test/OrchardCore.Tests.Playwright/tests/cms/comingsoon.spec.ts ComingSoon recipe specs in Playwright.
test/OrchardCore.Tests.Playwright/tests/cms/blog.spec.ts Blog recipe specs in Playwright.
test/OrchardCore.Tests.Playwright/tests/cms/agency.spec.ts Agency recipe specs in Playwright.
test/OrchardCore.Tests.Playwright/playwright.config.ts Playwright runner configuration and project definitions.
test/OrchardCore.Tests.Playwright/package.json New Yarn workspace package for Playwright tests.
test/OrchardCore.Tests.Playwright/helpers/utils.ts Shared test utilities (tenant/config helpers).
test/OrchardCore.Tests.Playwright/helpers/urls.ts URL navigation helper(s).
test/OrchardCore.Tests.Playwright/helpers/tenants.ts Tenant creation/setup helpers for tests.
test/OrchardCore.Tests.Playwright/helpers/selectors.ts data-cy selector helpers (Cypress parity).
test/OrchardCore.Tests.Playwright/helpers/recipes.ts Recipe run/import helpers.
test/OrchardCore.Tests.Playwright/helpers/features.ts Feature enable/disable helpers.
test/OrchardCore.Tests.Playwright/helpers/configuration.ts Site configuration helper(s).
test/OrchardCore.Tests.Playwright/helpers/buttons.ts Shared button-click helpers.
test/OrchardCore.Tests.Playwright/helpers/auth.ts Login helper for Orchard admin.
test/OrchardCore.Tests.Playwright/helpers/app-lifecycle.ts Build/host/ready-check lifecycle utilities for CI/local runs.
test/OrchardCore.Tests.Playwright/global-teardown.ts Stops hosted app and cleans migrations recipe.
test/OrchardCore.Tests.Playwright/global-setup.ts Hosts the app and prepares test environment.
test/OrchardCore.Tests.Playwright/fixtures/migrations.recipe.json Adds migrations recipe fixture for tests.
test/OrchardCore.Tests.Playwright/OrchardCore.Tests.Playwright.csproj Adds placeholder project to include Playwright test assets in solution.
test/OrchardCore.Tests.Functional/test_generator.js Removes Cypress-era fixture generator.
test/OrchardCore.Tests.Functional/package.json Removes Cypress functional test package config.
test/OrchardCore.Tests.Functional/mvc-tests/cypress/support/index.js Removes Cypress MVC support bootstrap.
test/OrchardCore.Tests.Functional/mvc-tests/cypress/plugins/index.js Removes Cypress MVC plugins entry.
test/OrchardCore.Tests.Functional/mvc-tests/cypress/integration/mvc-test.js Removes Cypress MVC spec.
test/OrchardCore.Tests.Functional/mvc-tests/cypress.json Removes Cypress MVC config.
test/OrchardCore.Tests.Functional/cypress-commands/src/utils.js Removes Cypress command utils source.
test/OrchardCore.Tests.Functional/cypress-commands/src/urls.js Removes Cypress URL command.
test/OrchardCore.Tests.Functional/cypress-commands/src/test-runner.js Removes Cypress test runner implementation.
test/OrchardCore.Tests.Functional/cypress-commands/src/tenants.js Removes Cypress tenant commands.
test/OrchardCore.Tests.Functional/cypress-commands/src/recipe.js Removes Cypress recipe commands.
test/OrchardCore.Tests.Functional/cypress-commands/src/index.js Removes Cypress commands index.
test/OrchardCore.Tests.Functional/cypress-commands/src/features.js Removes Cypress feature commands.
test/OrchardCore.Tests.Functional/cypress-commands/src/cySelectors.js Removes Cypress data-cy selector commands.
test/OrchardCore.Tests.Functional/cypress-commands/src/configuration.js Removes Cypress configuration commands.
test/OrchardCore.Tests.Functional/cypress-commands/src/buttons.js Removes Cypress button commands.
test/OrchardCore.Tests.Functional/cypress-commands/rollup.config.js Removes rollup bundling config for Cypress commands package.
test/OrchardCore.Tests.Functional/cypress-commands/package.json Removes Cypress commands package manifest.
test/OrchardCore.Tests.Functional/cypress-commands/package-lock.json Removes Cypress commands package lockfile.
test/OrchardCore.Tests.Functional/cypress-commands/dist/utils.mjs Removes built Cypress utils artifact.
test/OrchardCore.Tests.Functional/cypress-commands/dist/utils.js Removes built Cypress utils artifact.
test/OrchardCore.Tests.Functional/cypress-commands/dist/test-runner.mjs Removes built Cypress test runner artifact.
test/OrchardCore.Tests.Functional/cypress-commands/dist/test-runner.js Removes built Cypress test runner artifact.
test/OrchardCore.Tests.Functional/cypress-commands/dist/index.mjs Removes built Cypress command bundle.
test/OrchardCore.Tests.Functional/cypress-commands/dist/index.js Removes built Cypress command bundle.
test/OrchardCore.Tests.Functional/cypress-commands/README.md Removes Cypress commands package docs.
test/OrchardCore.Tests.Functional/cypress-commands/LICENSE Removes Cypress commands package license file.
test/OrchardCore.Tests.Functional/cypress-commands/.npmignore Removes Cypress commands publish ignore file.
test/OrchardCore.Tests.Functional/cms-tests/cypress/support/index.js Removes Cypress CMS support bootstrap.
test/OrchardCore.Tests.Functional/cms-tests/cypress/plugins/index.js Removes Cypress CMS plugins entry.
test/OrchardCore.Tests.Functional/cms-tests/cypress/integration/sass-test.js Removes Cypress SaaS spec.
test/OrchardCore.Tests.Functional/cms-tests/cypress/integration/migrations-test.js Removes Cypress migrations spec.
test/OrchardCore.Tests.Functional/cms-tests/cypress/integration/headless-test.js Removes Cypress headless spec.
test/OrchardCore.Tests.Functional/cms-tests/cypress/integration/comingsoon-test.js Removes Cypress coming soon spec.
test/OrchardCore.Tests.Functional/cms-tests/cypress/integration/blog-test.js Removes Cypress blog spec.
test/OrchardCore.Tests.Functional/cms-tests/cypress/integration/agency-test.js Removes Cypress agency spec.
test/OrchardCore.Tests.Functional/cms-tests/cypress/integration/000-saas-setup.js Removes Cypress SaaS setup bootstrap spec.
test/OrchardCore.Tests.Functional/cms-tests/cypress.json Removes Cypress CMS config.
test/OrchardCore.Tests.Functional/Readme.md Removes Cypress functional test documentation.
test/OrchardCore.Tests.Functional/OrchardCore.Tests.Functional.csproj Removes Cypress functional tests placeholder project.
test/OrchardCore.Tests.Functional/.gitignore Removes Cypress-specific ignores (now handled at repo root).
src/docs/reference/libraries/README.md Updates test library list from Cypress to Playwright.
package.json Adds workspace + root scripts to run Playwright tests.
OrchardCore.slnx Replaces Cypress functional test project with Playwright one.
AGENTS.md Updates agent guidance to run Playwright functional tests.
.gitignore Ignores Playwright reports and test-results.
.github/workflows/release_ci.yml Runs Playwright functional tests + uploads reports on failure.
.github/workflows/preview_ci.yml Runs Playwright functional tests + uploads reports on failure.
.github/workflows/pr_ci.yml Runs Playwright functional tests + uploads reports on failure.
.github/workflows/main_ci.yml Runs Playwright functional tests + uploads reports on failure.
.github/workflows/functional_all_db.yml Updates functional test matrix workflow to Playwright and Node 20.
Files not reviewed (1)
  • test/OrchardCore.Tests.Functional/cypress-commands/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/OrchardCore.Tests.Playwright/playwright.config.ts Outdated
Comment thread test/OrchardCore.Tests.Playwright/OrchardCore.Tests.Playwright.csproj Outdated
Copilot AI and others added 2 commits March 12, 2026 16:05
* Initial plan

* Refactor Playwright functional tests from TypeScript to C#

- Remove TypeScript Playwright test project (test/OrchardCore.Tests.Playwright)
- Create C# Playwright test project at test/OrchardCore.Tests.Functional
- Add xUnit v3 test classes mirroring all original TS tests
- Add helper classes for auth, tenants, configuration, lifecycle, etc.
- Update all CI workflows to use dotnet test instead of yarn/npm
- Update OrchardCore.slnx, Directory.Packages.props, AGENTS.md
- Revert root package.json (remove Playwright workspace/scripts)

Co-authored-by: Skrypt <3228637+Skrypt@users.noreply.github.com>

* Fix code review feedback: use base-32 encoding and static members

Co-authored-by: Skrypt <3228637+Skrypt@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Skrypt <3228637+Skrypt@users.noreply.github.com>
@Skrypt
Copy link
Copy Markdown
Contributor Author

Skrypt commented Mar 12, 2026

@copilot Fix the Github CI workflow that is failing.

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 12, 2026

@Skrypt I've opened a new pull request, #18992, to work on those changes. Once the pull request is ready, I'll request review from you.

Comment thread src/OrchardCore/OrchardCore.Mvc.Core/SharedViewCompilerProvider.cs Outdated
@Skrypt
Copy link
Copy Markdown
Contributor Author

Skrypt commented Mar 27, 2026

https://github.com/OrchardCMS/OrchardCore/actions/runs/23652309194/job/68900142655
Here is the result of running the "Functional - All Databases" workflow after enabling 4 parallel threads.
It is failing to run on SQL Server but intermittently will also fail on other DB's that are using an external service in the Github action. @copilot

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 68 out of 80 changed files in this pull request and generated 5 comments.

Files not reviewed (1)
  • test/OrchardCore.Tests.Functional/cypress-commands/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/OrchardCore.Tests.Functional/README.md Outdated
Comment thread test/OrchardCore.Tests.Functional/xunit.runner.json Outdated
Comment thread test/OrchardCore.Tests.Functional/Helpers/OrchardTestServer.cs Outdated
Comment thread test/OrchardCore.Tests.Functional/Helpers/OrchardTestFixture.cs Outdated
Comment thread .github/workflows/pr_ci.yml
Replace AppLifecycleHelper recipe file copy/delete with
EmbeddedRecipeHarvester that serves test recipes directly from embedded
assembly resources via IRecipeHarvester — no filesystem writes, no
shared lock, no ownership tracking. Any .recipe.json added to the
Fixtures/ folder is discovered automatically.

Remove all SetEnvironmentVariable calls from OrchardTestServer. Database
configuration is now injected per-host via builder.Configuration (which
takes priority over env vars), making each host self-contained with zero
process-global side effects.

Replace SqliteConnection.ClearAllPools() with a targeted ClearPool()
scoped to the fixture's specific database file.

These changes remove every shared-mutable-state blocker, enabling full
parallel execution of functional test fixtures.
@Skrypt
Copy link
Copy Markdown
Contributor Author

Skrypt commented Mar 27, 2026

@copilot This last commit seems to make the parallel execution even worst because of tenant initialization time.
https://github.com/OrchardCMS/OrchardCore/actions/runs/23654113880/job/68906559842

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 68 out of 80 changed files in this pull request and generated 3 comments.

Files not reviewed (1)
  • test/OrchardCore.Tests.Functional/cypress-commands/package-lock.json: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread test/OrchardCore.Tests.Functional/README.md Outdated
Comment thread test/OrchardCore.Tests.Functional/Helpers/TenantHelper.cs Outdated
Comment thread test/OrchardCore.Tests.Functional/Helpers/EmbeddedRecipeHarvester.cs Outdated
Skrypt and others added 10 commits March 27, 2026 11:53
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…ter.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Replace Substring concatenation with string.Concat and AsSpan to
satisfy the CA1845 analyzer rule.
Authenticate PostgreSQL and MySQL service container image pulls with
DOCKERHUB_USERNAME/DOCKERHUB_PASSWORD secrets to avoid Docker Hub
anonymous rate limits (100 pulls/6h) on shared GitHub Actions runners.
ShellSettingsManager re-adds EnvironmentVariablesConfigurationProvider
with highest priority, overriding the per-fixture connection strings set
via builder.Configuration. All fixtures ended up using the same database,
causing 'Table already exists' errors under parallel execution.

Replace static readonly capture with CaptureAndClear that reads the CI
env vars once at static initialization and immediately removes them.
Each fixture's database config is then applied exclusively through
tenants.json and builder.Configuration without env var interference.
Fix two issues that caused SaaS functional tests to fail on PostgreSQL,
MySQL, and SQL Server:

1. CreateTenantAsync skipped setting the table prefix when
   DatabaseConfigurationPreset was true (external DB on parent), because
   the #DatabaseProvider dropdown was not rendered and the entire block
   was guarded by its presence. Move table prefix handling outside the
   dropdown check so it always runs when the field is visible.

2. EnsureDatabaseExists kept leftover tables from previous runs, causing
   "database, table prefix and schema are already in use" validation
   errors. Replace with EnsureCleanDatabase that drops and recreates
   the fixture database. Uses pg_terminate_backend for Postgres < 13
   compatibility.

Also add run-db-tests.sh helper script to run functional tests against
all database providers locally via Docker, and force English locale so
test assertions match on non-English machines. Update README with usage.
Describe CaptureAndClear pattern instead of claiming no env vars are
mutated, note that fixture databases are dropped and recreated for
clean state, and clarify that maxParallelThreads defaults to CPU core
count when omitted from xunit.runner.json.
@Skrypt
Copy link
Copy Markdown
Contributor Author

Skrypt commented Mar 27, 2026

Full parallel build now working:
https://github.com/OrchardCMS/OrchardCore/actions/runs/23660563435
No more regressions.

@Skrypt Skrypt merged commit 652cc25 into main Mar 28, 2026
12 checks passed
@Skrypt Skrypt deleted the skrypt/playwright branch March 28, 2026 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants