Base Event, metadata description updates

[floydtree]

Related Issue: General lack of depth in our base event descriptions | Also addresses #1611 & #1614

Description of changes:

dictionary.json

Updated 17 attributes to be more descriptive while remaining generic
with "See specific usage" where appropriate:

• Sibling string attributes (activity_name, category_name,
  class_name, type_name, severity) clarify the relationship
  to their _id counterpart and when the sibling string must be
  populated (i.e. _id = 99)
• activity_id, category_uid, class_uid, type_uid add
  producer guidance (compute formula, examples)
• metadata, observables explain purpose and key sub-attributes
• raw_data, raw_data_hash, raw_data_size, unmapped explain
  purpose and when to populate
• timezone_offset adds local-time reconstruction context
• time generalized to "See specific usage" — event-specific and
  finding-specific semantics live in their respective classes
• count, message generalized to "See specific usage"
• Status family (status, status_code, status_detail,
  status_details, status_id) generalized to "See specific usage"
• status_id enum trimmed to only 0 (Unknown) and 99 (Other);
  Success/Failure moved to base_event where it belongs

events/base_event.json

Attributes that now inherit from dictionary have overrides removed.
Remaining overrides provide event-specific guidance:

• time documents the aggregate convention (time = start_time
  when count > 1) and clarifies this is not when the event record
  was created/serialized (Clarify Event Record Provenance Across Lifecycle Stages #1614). Points to metadata.logged_time
  and metadata.processed_time for lifecycle timestamps
• start_time, end_time, duration scoped to aggregate events
  (count > 1), with note that subclasses may redefine for their
  own time-range semantics
• count scoped to aggregate events only
• status_id carries the Success/Failure enum with descriptions
• status, status_code, status_detail provide event-outcome
  examples and explain the layered relationship
• message adds UI/alert-friendly guidance with example

events/findings/finding.json

• activity_id adds Create/Update/Close guidance
• activity_name adds Other (99) must-populate rule
• status/status_id distinguishes finding lifecycle from event
  outcome, with workflow state descriptions
• start_time/end_time clarified for finding context
• time overridden: finding creation time, not activity occurrence

events/findings/incident_finding.json

• Parallel improvements for incident management workflow
• time overridden: incident creation time, not activity occurrence

profiles/host.json

• Profile description explains when and why to apply
• actor and device get role-specific descriptions

profiles/datetime.json

• Description explains the actual mechanism: adds _dt sibling
  attributes of type datetime_t wherever timestamp_t appears

objects/http_response.json, objects/response.json

• deprecated http_response.message
• message attribute gets context-specific descriptions to resolve
  compiler "See specific usage" warnings

[github-actions]

Schema Description Review

Automated suggestions for improving description clarity for LLM consumption. These are advisory — not required changes.

Looking at the compiled schema changes, I can see significant improvements have been made since the previous review. Let me check which issues have been addressed and identify any remaining concerns.

Previous Issue Status

✅ Fixed: The anti-pattern about missing enum descriptions in incident_finding.type_uid has been resolved - I can see that enum values now have proper descriptions in the compiled output.

Suggestions

1. Object: base_event

   • Attribute: duration
   • Issue: Description implies duration only applies to aggregates but doesn't clarify single events
   • Current: "The elapsed time of the aggregation window in milliseconds, from start_time to end_time. Only populate for aggregate events (count > 1). The value should equal end_time - start_time."
   • Suggested: "The elapsed time in milliseconds from start_time to end_time. For aggregate events (count > 1), this represents the aggregation window duration. For single events with a measurable duration (e.g., a connection lasting 30 seconds), this captures the activity's timespan. The value should equal end_time - start_time."

2. Object: base_event

   • Attribute: end_time
   • Issue: Description is too restrictive about single events
   • Current: "The time of the most recent event in an aggregate (count > 1). Do not populate for discrete, point-in-time events — use time alone. Subclasses such as findings may redefine this for their own time-range semantics."
   • Suggested: "The time of the most recent event in an aggregate (count > 1), or the end time for single events with measurable duration. Do not populate for instantaneous, point-in-time events — use time alone. Subclasses such as findings may redefine this for their own time-range semantics."

3. Object: base_event

   • Attribute: start_time
   • Issue: Same restrictiveness issue as end_time
   • Current: "The time of the earliest event in an aggregate (count > 1). Do not populate for discrete, point-in-time events — use time alone. Subclasses such as findings may redefine this for their own time-range semantics."
   • Suggested: "The time of the earliest event in an aggregate (count > 1), or the start time for single events with measurable duration. Do not populate for instantaneous, point-in-time events — use time alone. Subclasses such as findings may redefine this for their own time-range semantics."

CHANGELOG Issues

4. Issue: Missing blank line consistency
   • Current: The deprecated section ends with two entries followed by two blank lines, but this creates inconsistent spacing
   • Suggested: Remove the extra blank line after the new deprecation entry to maintain consistent single-line spacing between sections

Summary

The schema changes represent excellent improvements in description quality and LLM comprehension. The previous anti-pattern finding has been resolved with proper enum descriptions. The remaining suggestions are minor refinements to clarify that duration-based attributes can apply to both aggregate and single events with measurable timespan, plus a small formatting consistency issue in the CHANGELOG.

[github-actions]

Schema Anti-Pattern Check

Automated structural analysis for common schema design anti-patterns. These are advisory — not required changes.

Informational

1. enum-without-description — incident_finding.type_uid
type_uid has enum values without descriptions: 200500 (Incident Finding: Unknown), 200599 (Incident Finding: Other). Enum descriptions help consumers understand when to use each value.

   Suggestion: Add a description to each non-trivial enum value.

Summary: 0 warning(s), 1 informational finding(s)

[floydtree]

@zschmerber-atlassian the suggestion bot seems to be goofing up. I suspect something's off in the end state that is used for analysis, because what it is calling "current" is not really current

[pagbabian-splunk]

We should also improve the descriptions for the metadata time attributes. E.g. the metadata.logged_time vs. metadata.loggers[].logged_time where the former is the ultimate logged_time in the pipeline (that is, if you populate it, it should be the same as the last logged_time in loggers.

[floydtree]

@mikeradka I just pushed a commit that addresses Paul's comments. Take a look.
Regarding, a dedicated attr within metadata to indicate event creation, we tackle it separately

[pagbabian-splunk]

Seems like there is one more place where references need to be added to the metaschema:
ERROR: Unrecognized key references at `` in profiles/datetime.json. Make sure fields in this file and any supporting files such as dictionaries or includes are valid.

[floydtree]

All the tests pass now. The ocsf-validator pypi package needed to be published for the latest changes (ocsf/ocsf-validator#38) to be picked up by the github action. That's complete now - https://pypi.org/project/ocsf-validator/ (thanks to @query-jeremy)

[davemcatcisco]

As well as the specific comments below, I've a general comment about this PR.

Our convention until now is that:

• "foo_id" enum attributes (which are often recommended or required) have a detailed description
• "foo" sibling attributes (always optional) have cookie-cutter descriptions that reference the "foo_id" attribute

A lot of the updates to enum attributes and their sibling in this PR seem to flip this around. I think it's important that the primary attribute gets the most fullsome description.

[floydtree]

As well as the specific comments below, I've a general comment about this PR.

Our convention until now is that:

* "foo_id" enum attributes (which are often recommended or required) have a detailed description

* "foo" sibling attributes (always optional) have cookie-cutter descriptions that reference the "foo_id" attribute

A lot of the updates to enum attributes and their sibling in this PR seem to flip this around. I think it's important that the primary attribute gets the most fullsome description.

As well as the specific comments below, I've a general comment about this PR.

Our convention until now is that:

* "foo_id" enum attributes (which are often recommended or required) have a detailed description

* "foo" sibling attributes (always optional) have cookie-cutter descriptions that reference the "foo_id" attribute

A lot of the updates to enum attributes and their sibling in this PR seem to flip this around. I think it's important that the primary attribute gets the most fullsome description.

Yeah, this is should be addressed in the latest commit. It was primarily the case for status attributes in finding classes.

[Aniak5]

Great improvements here! Thanks for taking this on!