Add field maturity and remove experimental pipeline

.gitignore

@@ -8,13 +8,8 @@ build
 *.iml
 .vscode/*
 
-# experimental exclusions
-experimental/generated/elasticsearch/6
-experimental/generated/docs
-
 # subset exclusions
 generated/ecs/subset
-experimental/generated/ecs/subset
 
 # patches are vital to cross-branch testing but don't want in GitHub
 *.patch

CHANGELOG.next.md

@@ -28,8 +28,12 @@ Thanks, you're awesome :-) -->
 
 #### Added
 
+* Add `alpha` field maturity attribute, mirroring `beta` with mutual exclusion. #XXXX
+* Document field stability definitions (alpha, beta, GA) in design principles. #XXXX
+
 #### Improvements
 
+* Remove the `experimental/` build pipeline and unused `cgroup.*` fields; alpha and beta fields now live in `schemas/`. #XXXX
 * Increase composable template `total_fields.limit` from 2000 to 2500. #2584
 
 #### Deprecated

Makefile

@@ -6,22 +6,21 @@ FIND             := find . -type f -not -path './build/*' -not -path './.git/*'
 OPEN_DOCS        ?= "--open"
 PYTHON           := build/ve/bin/python
 SUBSETS_DIR      := schemas/subsets/
-EXP_SUBSETS_DIR  := experimental/schemas/subsets/
 VERSION          := $(shell cat version)
 SEMCONV_VERSION  := $(shell cat otel-semconv-version)
 
 #
 # Targets (sorted alphabetically)
 #
 
-# Default build generates main and experimental artifacts
+# Default build generates all artifacts
 .PHONY: all
-all: generate experimental
+all: generate
 
 # Check verifies that all of the committed files that are generated are
 # up-to-date.
 .PHONY: check
-check: generate experimental test fmt misspell_warn makelint
+check: generate test fmt misspell_warn makelint
 	# Check if diff is empty.
 	git diff | cat
 	git update-index --refresh
@@ -39,7 +38,7 @@ check_license_headers:
 # Clean deletes all temporary and generated content.
 .PHONY: clean
 clean:
-	rm -rf build generated/elasticsearch/composable/component experimental/generated/elasticsearch/composable/component
+	rm -rf build generated/elasticsearch/composable/component
 	# Clean generated documentation files
 	@echo "Removing generated documentation files..."
 	@rm -f docs/reference/index.md docs/reference/ecs-field-reference.md docs/reference/ecs-otel-alignment-details.md docs/reference/ecs-otel-alignment-overview.md
@@ -98,11 +97,6 @@ docs:
 		$(PWD)/build/docs/docs-builder serve; \
 	fi
 
-# Alias to generate experimental artifacts
-.PHONY: experimental
-experimental: ve
-	$(PYTHON) scripts/generator.py --include experimental/schemas --subset "${SUBSETS_DIR}" "${EXP_SUBSETS_DIR}" --semconv-version "${SEMCONV_VERSION}" --out experimental
-
 # Format code and files in the repo.
 .PHONY: fmt
 fmt: ve
@@ -134,7 +128,7 @@ misspell:
 		chmod +x ./build/misspell/install-misspell.sh ; \
 		./build/misspell/install-misspell.sh -b ./build/misspell/bin >> /dev/null 2>&1 ; \
 	fi
-	./build/misspell/bin/misspell -error README.md CONTRIBUTING.md schemas/* docs/* experimental/schemas/*
+	./build/misspell/bin/misspell -error README.md CONTRIBUTING.md schemas/* docs/*
 
 # Warn re misspell removal
 .PHONY: misspell_warn

USAGE.md

@@ -173,11 +173,11 @@ python scripts/generator.py \
 **Combines with other options**:
 
 ```bash
-# Generate from ECS v9.0.0 + experimental + custom fields
+# Generate from ECS v9.0.0 + custom fields
 python scripts/generator.py \
   --semconv-version v1.38.0 \
   --ref v9.0.0 \
-  --include experimental/schemas ../myproject/fields/custom
+  --include ../myproject/fields/custom
 ```
 
 Loads schemas from git history (tags, branches, commits). Requires git.

_config.yml

@@ -5,6 +5,5 @@ exclude:
   - scripts/
   - venv/
   - generated/
-  - experimental/
   - "*.py"
   - Makefile

docs/reference/ecs-principles-design.md

@@ -37,9 +37,27 @@
 * Several potential IP addresses in a single event (`source.ip`, `destination.ip`, `host.ip`).
 
 
+## Field stability [_field_stability]
+
+Every ECS field carries a maturity level that tells consumers what to expect:
+
+**Alpha.** An alpha field is newly introduced and under active development. Its schema — including field name, type, expected values, and semantics — may change or the field may be removed entirely in any future release without prior notice. Alpha fields are intended for early feedback and exploratory use only. Consumers should not depend on alpha fields for production workloads, alerting rules, or long-lived dashboards. No backwards-compatibility guarantees are provided.
+
+**Beta.** A beta field has reached a reasonable level of stability and is expected to become part of the stable schema, but has not yet received a final commitment. The field name and type are unlikely to change, though adjustments to allowed values, descriptions, or minor semantic refinements may still occur. Beta fields are suitable for broader adoption, including dashboards, detection rules, and integrations, with the understanding that minor migration effort may occasionally be required. Breaking changes to beta fields are not expected but may still occur.
+
+**GA (stable).** The default. GA fields are backwards-compatible; breaking changes are not made outside of major versions.
+
+### Criteria for each level [_field_stability_criteria]
+
+These criteria give schema authors and reviewers a shared rubric for assigning maturity:
+
+* **Alpha when:** field definitions are reasonably complete but may still evolve; usage patterns are not yet well understood; field definitions have not yet been completely validated against real-world data; open questions remain.
+* **Beta when:** field definitions are complete; usage patterns are well understood across multiple scenarios; field definitions have been validated against real-world data; impact of changes to ingestion and consumption is understood; open questions are minimal.
+
+
 ## Reuse [_reuse]
 
 Introducing extra fields may seem necessary when adding or expanding a concept. But, try to use an existing field or reuse an existing field set to avoid duplicating fields. Leveraging consistent fields across event sources helps build more straightforward queries and visualizations.
 
 For example, imagine an app or framework produces a unique ID for each log it emits. Instead of adding a custom `.id` field specific to that app, consider the `event.id` field.