-
Notifications
You must be signed in to change notification settings - Fork 126
Add the Docker Compose Detector #1785
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 8 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
0059cf2
Add the DockerComposeComponentDetector and tests
jpinz a5a66f1
Update the DockerfileComponentDetector to enable nullability and use …
jpinz e1a765a
Merge branch 'main' into jupinzer/docker_compose_detector
jpinz 330d5fc
Address copilot PR comments. And add new tests for DockerReferenceUti…
jpinz fbfec90
Address PR comments.
jpinz 7e8f1ef
Add logging to some of the utils in DockerReferenceUtility for parse …
jpinz 382480d
Merge branch 'main' into jupinzer/docker_compose_detector
jpinz d1a648d
Merge branch 'main' into jupinzer/docker_compose_detector
jpinz 6d054fc
I missed passing in the logger to the DockerReference parsing from th…
jpinz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| # Docker Compose Detection | ||
|
|
||
| ## Requirements | ||
|
|
||
| Docker Compose detection depends on the following to successfully run: | ||
|
|
||
| - One or more Docker Compose files matching the patterns: `docker-compose.yml`, `docker-compose.yaml`, `docker-compose.*.yml`, `docker-compose.*.yaml`, `compose.yml`, `compose.yaml`, `compose.*.yml`, `compose.*.yaml` | ||
|
|
||
| The `DockerComposeComponentDetector` is a **DefaultOff** detector and must be explicitly enabled via the `--DetectorArgs` parameter. | ||
|
|
||
| ## Detection strategy | ||
|
|
||
| The Docker Compose detector parses YAML compose files to extract Docker image references from service definitions. | ||
|
|
||
| ### Service Image Detection | ||
|
|
||
| The detector looks for the `services` section and extracts the `image` field from each service: | ||
|
|
||
| ```yaml | ||
| services: | ||
| web: | ||
| image: nginx:1.21 | ||
| db: | ||
| image: postgres:14 | ||
| ``` | ||
|
|
||
| Services that only define a `build` directive without an `image` field are skipped, as they do not reference external Docker images. | ||
|
|
||
| ### Full Registry References | ||
|
|
||
| The detector supports full registry image references: | ||
|
|
||
| ```yaml | ||
| services: | ||
| app: | ||
| image: ghcr.io/myorg/myapp:v2.0 | ||
| ``` | ||
|
|
||
| ### Variable Resolution | ||
|
|
||
| Images containing unresolved variables (e.g., `${TAG}` or `${REGISTRY:-docker.io}`) are skipped to avoid reporting incomplete or incorrect references. The detector checks for `$`, `{`, or `}` characters in image references. | ||
|
|
||
| ## Known limitations | ||
|
|
||
| - **DefaultOff Status**: This detector must be explicitly enabled using `--DetectorArgs DockerCompose=EnableIfDefaultOff` | ||
| - **Variable Resolution**: Image references containing unresolved environment variables or template expressions are not reported, which may lead to under-reporting in compose files that heavily use variable substitution | ||
| - **Build-Only Services**: Services that only specify a `build` directive without an `image` field are not reported | ||
| - **No Dependency Graph**: All detected images are registered as independent components without parent-child relationships |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
123 changes: 123 additions & 0 deletions
123
src/Microsoft.ComponentDetection.Detectors/dockercompose/DockerComposeComponentDetector.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| namespace Microsoft.ComponentDetection.Detectors.DockerCompose; | ||
|
|
||
| using System; | ||
| using System.Collections.Generic; | ||
| using System.IO; | ||
| using System.Threading; | ||
| using System.Threading.Tasks; | ||
| using Microsoft.ComponentDetection.Common; | ||
| using Microsoft.ComponentDetection.Contracts; | ||
| using Microsoft.ComponentDetection.Contracts.Internal; | ||
| using Microsoft.ComponentDetection.Contracts.TypedComponent; | ||
|
jpinz marked this conversation as resolved.
|
||
| using Microsoft.Extensions.Logging; | ||
| using YamlDotNet.RepresentationModel; | ||
|
|
||
| public class DockerComposeComponentDetector : FileComponentDetector, IDefaultOffComponentDetector | ||
| { | ||
| public DockerComposeComponentDetector( | ||
| IComponentStreamEnumerableFactory componentStreamEnumerableFactory, | ||
| IObservableDirectoryWalkerFactory walkerFactory, | ||
| ILogger<DockerComposeComponentDetector> logger) | ||
| { | ||
| this.ComponentStreamEnumerableFactory = componentStreamEnumerableFactory; | ||
| this.Scanner = walkerFactory; | ||
| this.Logger = logger; | ||
| } | ||
|
|
||
| public override string Id => "DockerCompose"; | ||
|
|
||
| public override IList<string> SearchPatterns { get; } = | ||
| [ | ||
| "docker-compose.yml", "docker-compose.yaml", | ||
| "docker-compose.*.yml", "docker-compose.*.yaml", | ||
| "compose.yml", "compose.yaml", | ||
| "compose.*.yml", "compose.*.yaml", | ||
| ]; | ||
|
|
||
| public override IEnumerable<ComponentType> SupportedComponentTypes => [ComponentType.DockerReference]; | ||
|
|
||
| public override int Version => 1; | ||
|
|
||
| public override IEnumerable<string> Categories => [nameof(DetectorClass.DockerCompose)]; | ||
|
|
||
| protected override async Task OnFileFoundAsync(ProcessRequest processRequest, IDictionary<string, string> detectorArgs, CancellationToken cancellationToken = default) | ||
| { | ||
| var singleFileComponentRecorder = processRequest.SingleFileComponentRecorder; | ||
| var file = processRequest.ComponentStream; | ||
|
|
||
| try | ||
| { | ||
| this.Logger.LogInformation("Discovered Docker Compose file: {Location}", file.Location); | ||
|
|
||
| string contents; | ||
| using (var reader = new StreamReader(file.Stream)) | ||
| { | ||
| contents = await reader.ReadToEndAsync(cancellationToken); | ||
| } | ||
|
|
||
| var yaml = new YamlStream(); | ||
| yaml.Load(new StringReader(contents)); | ||
|
|
||
| if (yaml.Documents.Count == 0) | ||
| { | ||
| return; | ||
| } | ||
|
|
||
| foreach (var document in yaml.Documents) | ||
| { | ||
| if (document.RootNode is YamlMappingNode rootMapping) | ||
| { | ||
| this.ExtractImageReferences(rootMapping, singleFileComponentRecorder); | ||
| } | ||
| } | ||
| } | ||
| catch (Exception e) | ||
| { | ||
| this.Logger.LogError(e, "Failed to parse Docker Compose file: {Location}", file.Location); | ||
| } | ||
| } | ||
|
|
||
| private static YamlMappingNode? GetMappingChild(YamlMappingNode parent, string key) | ||
| { | ||
| foreach (var entry in parent.Children) | ||
| { | ||
| if (entry.Key is YamlScalarNode scalarKey && string.Equals(scalarKey.Value, key, StringComparison.OrdinalIgnoreCase)) | ||
| { | ||
| return entry.Value as YamlMappingNode; | ||
| } | ||
| } | ||
|
|
||
| return null; | ||
| } | ||
|
|
||
| private void ExtractImageReferences(YamlMappingNode rootMapping, ISingleFileComponentRecorder recorder) | ||
| { | ||
| var services = GetMappingChild(rootMapping, "services"); | ||
| if (services == null) | ||
| { | ||
| return; | ||
| } | ||
|
|
||
| foreach (var serviceEntry in services.Children) | ||
| { | ||
| if (serviceEntry.Value is not YamlMappingNode serviceMapping) | ||
| { | ||
| continue; | ||
| } | ||
|
|
||
| // Extract direct image: references | ||
| foreach (var entry in serviceMapping.Children) | ||
| { | ||
| var key = (entry.Key as YamlScalarNode)?.Value; | ||
| if (string.Equals(key, "image", StringComparison.OrdinalIgnoreCase)) | ||
| { | ||
| var imageRef = (entry.Value as YamlScalarNode)?.Value; | ||
| if (!string.IsNullOrWhiteSpace(imageRef)) | ||
| { | ||
| DockerReferenceUtility.TryRegisterImageReference(imageRef, recorder, this.Logger); | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.