Skip to content

Use TestContainers for AWS Local Stack #19157

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hishamco wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Use TestContainers for AWS Local Stack #19157

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2ec32b2
Use TestContainers for AWS Local Stack
hishamco Apr 15, 2026
0cad839
Add integration tests to GitHub workflow
hishamco Apr 16, 2026
82df941
Try to use latest image
hishamco Apr 17, 2026
a9abdb2
Revert "Try to use latest image"
hishamco Apr 17, 2026
51e1c63
Run integration tests on linux only
hishamco Apr 17, 2026
bcbc8e6
Tweaks
hishamco Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 0 additions & 67 deletions .github/workflows/integration_tests.yml
View file
Open in desktop

This file was deleted.

3 changes: 2 additions & 1 deletion Directory.Packages.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
<PackageVersion Include="SixLabors.ImageSharp.Web.Providers.AWS" Version="3.2.0" />
<PackageVersion Include="StackExchange.Redis" Version="2.12.8" />
<PackageVersion Include="StyleCop.Analyzers" Version="1.1.118" />
<PackageVersion Include="Testcontainers.LocalStack" Version="4.11.0" />
Copy link
Copy Markdown
Contributor

@Skrypt Skrypt Apr 17, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR looks good although @Piedone really wanted that we use S3Mock because it is lighter in size and executes faster. There is a Java version but no C# one on Testcontainers. Also, let's keep the integration_tests.yml file and nothing else, I don't think @Piedone will like it.

Copy link
Copy Markdown
Member Author

@hishamco hishamco Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the build trigger, I didn't notice any delay

Copy link
Copy Markdown
Contributor

@Skrypt Skrypt Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

That's 1min 55 seconds execution time added on every PR.

Copy link
Copy Markdown
Member

@Piedone Piedone Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. This pulls in everything from AWS. We just need S3.

Copy link
Copy Markdown
Member Author

@hishamco hishamco Apr 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will check I remembered there's an option to specify the service, such as S3 for instance, but I need to make sure

Copy link
Copy Markdown
Member

@Piedone Piedone Apr 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do that if and when we'll do that. No need to waste time until that uncertain future.

Copy link
Copy Markdown
Member Author

@hishamco hishamco Apr 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR based on the meeting feedback, but you or Jasmin can discuss that as a topic in the upcoming meeting

Copy link
Copy Markdown
Member

@Piedone Piedone Apr 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this stage, this PR makes things worse in two ways, what Jasmin and I pointed out here. There's no need to discuss this at the meeting, especially not among the two of us, since we're in agreement. It's you who needs to reflect on the feedback.

Copy link
Copy Markdown
Member Author

@hishamco hishamco Apr 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't get me wrong. I understand your feedback very well :) But it would be nice if @Skrypt mentioned the concern in the last meeting - to save the time -

Based on the discussion, @sebastienros suggested using TestContainers, which is why I submitted the PR, but not forcing acceptance of it

Copy link
Copy Markdown
Contributor

@Skrypt Skrypt Apr 19, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not raise a concern not knowing which Docker image is available from testcontainer. I asked questions in that meeting and I said I would try ; which is what you did here. Now, if that makes things worst now we know. Can't know without trying. Thanks for doing the work and saving me time. Now we know that we should'nt assume that testcontainer is better all the time.

<PackageVersion Include="xunit.v3.mtp-v2" Version="3.2.2" />
<PackageVersion Include="YesSql" Version="5.4.7" />
<PackageVersion Include="YesSql.Abstractions" Version="5.4.7" />
Expand Down Expand Up @@ -110,4 +111,4 @@
<PackageVersion Include="System.IO.Hashing" Version="10.0.5" />
<PackageVersion Include="System.Text.Json" Version="10.0.5" />
</ItemGroup>
</Project>
</Project>
126 changes: 43 additions & 83 deletions test/OrchardCore.Tests.Integration/AmazonS3/AwsFileStoreTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,43 +1,41 @@
using Amazon;
using Amazon.Runtime;
using Amazon.S3;
using Amazon.S3.Model;
using Moq;
using OrchardCore.FileStorage;
using OrchardCore.FileStorage.AmazonS3;
using OrchardCore.Modules;
using Testcontainers.LocalStack;
using Xunit;

namespace OrchardCore.Tests.Integration.AmazonS3;

/// <summary>
/// Integration tests for <see cref="AwsFileStore"/> that run against an S3-compatible emulator (e.g. Adobe S3Mock).
/// Set the <c>S3_EMULATOR_URL</c> environment variable (e.g. <c>http://127.0.0.1:9090</c>) to run these tests.
/// </summary>
public sealed class AwsFileStoreTests : IAsyncLifetime
public class AwsFileStoreTests : IAsyncLifetime
{
private const string EnvVar = "S3_EMULATOR_URL";

private readonly ITestOutputHelper _output;
private AwsFileStore _store;
private AmazonS3Client _s3Client;
private string _bucketName;

public AwsFileStoreTests(ITestOutputHelper output) => _output = output;
private readonly LocalStackContainer _localStackContainer = new LocalStackBuilder("localstack/localstack:2.0")
.Build();

private static string GetServiceUrl()
=> System.Environment.GetEnvironmentVariable(EnvVar);
static AwsFileStoreTests()
{
System.Environment.SetEnvironmentVariable("AWS_ACCESS_KEY_ID", CommonCredentials.AwsAccessKey);
System.Environment.SetEnvironmentVariable("AWS_SECRET_ACCESS_KEY", CommonCredentials.AwsSecretKey);
}

public async ValueTask InitializeAsync()
{
var serviceUrl = GetServiceUrl();
await _localStackContainer.StartAsync()
.ConfigureAwait(false);

_bucketName = $"test-{Guid.NewGuid():N}";

_s3Client = new AmazonS3Client(
new BasicAWSCredentials("test", "test"),
new AmazonS3Config
{
ServiceURL = serviceUrl,
ServiceURL = _localStackContainer.GetConnectionString(),
ForcePathStyle = true,
AuthenticationRegion = "us-east-1",
RequestChecksumCalculation = RequestChecksumCalculation.WHEN_REQUIRED,
Expand All @@ -56,39 +54,6 @@ public async ValueTask InitializeAsync()
_store = new AwsFileStore(clock, options, _s3Client);
}

public async ValueTask DisposeAsync()
{
if (_s3Client is not null && _bucketName is not null)
{
try
{
// Delete all objects first (S3 requires empty bucket before deletion).
var listResponse = await _s3Client.ListObjectsV2Async(new ListObjectsV2Request
{
BucketName = _bucketName,
});

if (listResponse.S3Objects.Count > 0)
{
await _s3Client.DeleteObjectsAsync(new DeleteObjectsRequest
{
BucketName = _bucketName,
Objects = listResponse.S3Objects.Select(o => new KeyVersion { Key = o.Key }).ToList(),
});
}

await _s3Client.DeleteBucketAsync(_bucketName);
}
catch (Exception ex)
{
_output.WriteLine($"Best effort cleanup failed: {ex.Message}");
}

_s3Client.Dispose();
}

}

private async Task<string> CreateTestFileAsync(string path, string content = "test content")
{
using var stream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(content));
Expand All @@ -104,14 +69,14 @@ private async Task<string> ReadFileContentAsync(string path)

// -- File operations --

[S3MockFact]
[Fact]
public async Task CreateFile_ReturnsPath()
{
var result = await CreateTestFileAsync("folder/file.txt");
Assert.Equal("folder/file.txt", result);
}

[S3MockFact]
[Fact]
public async Task GetFileInfo_ReturnsCorrectMetadata()
{
var content = "hello world";
Expand All @@ -126,14 +91,14 @@ public async Task GetFileInfo_ReturnsCorrectMetadata()
Assert.False(info.IsDirectory);
}

[S3MockFact]
[Fact]
public async Task GetFileInfo_NonExistent_ReturnsNull()
{
var info = await _store.GetFileInfoAsync("does-not-exist.txt");
Assert.Null(info);
}

[S3MockFact]
[Fact]
public async Task GetFileStream_ReturnsContent()
{
var expected = "stream content test";
Expand All @@ -144,7 +109,7 @@ public async Task GetFileStream_ReturnsContent()
Assert.Equal(expected, actual);
}

[S3MockFact]
[Fact]
public async Task DeleteFile_ReturnsTrue()
{
await CreateTestFileAsync("delete-me.txt");
Expand All @@ -155,7 +120,7 @@ public async Task DeleteFile_ReturnsTrue()
Assert.Null(await _store.GetFileInfoAsync("delete-me.txt"));
}

[S3MockFact]
[Fact]
public async Task CopyFile_CreatesNewFile()
{
var content = "copy me";
Expand All @@ -167,7 +132,7 @@ public async Task CopyFile_CreatesNewFile()
Assert.Equal(content, await ReadFileContentAsync("copied.txt"));
}

[S3MockFact]
[Fact]
public async Task CopyFile_SamePath_Throws()
{
await CreateTestFileAsync("same.txt");
Expand All @@ -176,14 +141,14 @@ await Assert.ThrowsAnyAsync<Exception>(
() => _store.CopyFileAsync("same.txt", "same.txt"));
}

[S3MockFact]
[Fact]
public async Task CopyFile_SourceNotFound_Throws()
{
await Assert.ThrowsAsync<FileStoreException>(
() => _store.CopyFileAsync("ghost.txt", "dest.txt"));
}

[S3MockFact]
[Fact]
public async Task CreateFile_OverwriteTrue_Succeeds()
{
await CreateTestFileAsync("overwrite.txt", "v1");
Expand All @@ -195,7 +160,7 @@ public async Task CreateFile_OverwriteTrue_Succeeds()
Assert.Equal("v2", content);
}

[S3MockFact]
[Fact]
public async Task CreateFile_OverwriteFalse_Throws()
{
await CreateTestFileAsync("no-overwrite.txt");
Expand All @@ -207,7 +172,7 @@ await Assert.ThrowsAnyAsync<FileStoreException>(

// -- Directory operations --

[S3MockFact]
[Fact]
public async Task GetDirectoryInfo_Root_ReturnsEntry()
{
var info = await _store.GetDirectoryInfoAsync(string.Empty);
Expand All @@ -216,7 +181,7 @@ public async Task GetDirectoryInfo_Root_ReturnsEntry()
Assert.True(info.IsDirectory);
}

[S3MockFact]
[Fact]
public async Task GetDirectoryInfo_Existing_ReturnsEntry()
{
await _store.TryCreateDirectoryAsync("my-folder");
Expand All @@ -227,14 +192,14 @@ public async Task GetDirectoryInfo_Existing_ReturnsEntry()
Assert.True(info.IsDirectory);
}

[S3MockFact]
[Fact]
public async Task GetDirectoryInfo_NonExistent_ReturnsNull()
{
var info = await _store.GetDirectoryInfoAsync("no-such-folder");
Assert.Null(info);
}

[S3MockFact]
[Fact]
public async Task CreateDirectory_Succeeds()
{
var result = await _store.TryCreateDirectoryAsync("new-dir");
Expand All @@ -245,7 +210,7 @@ public async Task CreateDirectory_Succeeds()
Assert.NotNull(info);
}

[S3MockFact]
[Fact]
public async Task DeleteDirectory_WithContents_DeletesAll()
{
await _store.TryCreateDirectoryAsync("dir-to-delete");
Expand All @@ -259,7 +224,7 @@ public async Task DeleteDirectory_WithContents_DeletesAll()
Assert.Null(await _store.GetFileInfoAsync("dir-to-delete/file2.txt"));
}

[S3MockFact]
[Fact]
public async Task DeleteDirectory_Root_Throws()
{
await Assert.ThrowsAsync<FileStoreException>(
Expand All @@ -268,7 +233,7 @@ await Assert.ThrowsAsync<FileStoreException>(

// -- Move --

[S3MockFact]
[Fact]
public async Task MoveFile_MovesToNewPath()
{
var content = "move me";
Expand All @@ -280,7 +245,7 @@ public async Task MoveFile_MovesToNewPath()
Assert.Equal(content, await ReadFileContentAsync("dst.txt"));
}

[S3MockFact]
[Fact]
public async Task MoveFile_AcrossDirectories()
{
await _store.TryCreateDirectoryAsync("dir-a");
Expand All @@ -295,7 +260,7 @@ public async Task MoveFile_AcrossDirectories()

// -- Directory content listing --

[S3MockFact]
[Fact]
public async Task GetDirectoryContent_ListsFilesAndDirs()
{
await CreateTestFileAsync("root-file.txt");
Expand All @@ -311,23 +276,18 @@ public async Task GetDirectoryContent_ListsFilesAndDirs()
Assert.Contains(entries, e => e.Name == "root-file.txt" && !e.IsDirectory);
Assert.Contains(entries, e => e.Name == "sub-dir" && e.IsDirectory);
}
}

/// <summary>
/// Skips the test when the S3 emulator URL is not configured.
/// Set the <c>S3_EMULATOR_URL</c> environment variable to run these tests.
/// </summary>
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
internal sealed class S3MockFactAttribute : FactAttribute
{
public S3MockFactAttribute(
[System.Runtime.CompilerServices.CallerFilePath] string sourceFilePath = null,
[System.Runtime.CompilerServices.CallerLineNumber] int sourceLineNumber = -1)
: base(sourceFilePath, sourceLineNumber)
public async ValueTask DisposeAsync()
{
if (string.IsNullOrEmpty(System.Environment.GetEnvironmentVariable("S3_EMULATOR_URL")))
{
Skip = "S3 emulator is not configured. Set S3_EMULATOR_URL to run this test.";
}
await _localStackContainer.StopAsync();

GC.SuppressFinalize(this);
}

//public sealed class LocalStackDefaultConfiguration : AwsFileStoreTests
//{
// public LocalStackDefaultConfiguration() : base(new LocalStackBuilder(TestSession.GetImageFromDockerfile()).Build())
// {
// }
//}
}
8 changes: 8 additions & 0 deletions test/OrchardCore.Tests.Integration/AmazonS3/CommonCredentials.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
namespace OrchardCore.Tests.Integration.AmazonS3;

public static class CommonCredentials
{
public const string AwsAccessKey = "AKIAIOSFODNN7EXAMPLE";

public const string AwsSecretKey = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY";
}
1 change: 1 addition & 0 deletions test/OrchardCore.Tests.Integration/OrchardCore.Tests.Integration.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

<ItemGroup>
<PackageReference Include="Moq" />
<PackageReference Include="Testcontainers.LocalStack" />
<PackageReference Include="xunit.v3.mtp-v2" />
</ItemGroup>

Expand Down
Loading