Skip to content

Add context.Context support so a running program can be cancelled #36

Description

@gh123man

Operation / feature

A context.Context-aware run API, equivalent to gojq's
RunWithContext:

func (p *Program) RunFuncContext(ctx context.Context, input []byte, fn func([]byte) error) error
func (p *Program) RunContext(ctx context.Context, input []byte) ([]byte, error)

Why

There is currently no way to abort a running program. A query whose output is unbounded
runs to completion no matter what the caller wants:

range(1000000000) | {value: .}

RunAll additionally materialises every output before returning, so that query is an
unbounded memory commitment as well as an unbounded CPU one. RunFunc at least lets the
caller stop accumulating — though today the callback can't actually halt the walk either
(#31), so even a caller who counts outputs keeps getting invoked.

For a library aimed at high-throughput log processing this also matters for ordinary
operational reasons: per-record deadlines, shutdown, and load-shedding all want
cancellation, not just pathological-input defence.

The workaround available today is an output-count cap inside the callback, which is what we
did. It bounds output cardinality but nothing else — it can't interrupt time spent inside a
single output, and it doesn't help queries that are slow rather than prolific.

Reference

  • gojq: Code.RunWithContext
    checks ctx.Err() during iteration and yields the context error as a result.

Expected allocation tier

  • Zero allocations on the hot path (must be implementable on raw bytes
    without buffering structured data; see docs/CONSTRAINTS.md).

A ctx.Err() check between outputs (and at generator/reduce/foreach loop boundaries) is
a nil-pointer comparison in the common case and allocates nothing. Keeping the existing
non-ctx methods as the zero-overhead path and having them delegate with
context.Background() would avoid any cost for callers that don't need it.

Context

Found while replacing gojq with fastjq in the Datadog Agent. The call site executes
remote-config-authored jq transforms in-process during package configuration; the gojq
version passed the operation's context.Context straight through to RunWithContext, and
that cancellation was lost in the port.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions