Operation / feature
Two related gaps in expressing output cardinality through the current API:
1. Run cannot distinguish "no output" from "one null output". Both return
("null", nil):
package main
import (
"fmt"
"github.com/DataDog/fastjq"
)
func main() {
for _, q := range []string{`empty`, `null`, `.missing`} {
p, _ := fastjq.Compile(q)
out, err := p.Run([]byte(`{}`))
fmt.Printf("%-10s out=%q err=%v\n", q, string(out), err)
}
}
empty out="null" err=<nil>
null out="null" err=<nil>
.missing out="null" err=<nil>
In jq these are different: empty produces zero outputs, null and .missing each
produce one. A caller that needs to tell "the filter matched nothing" from "the filter
matched and the value is null" — a boolean predicate, an existence check — has to abandon
Run and hand-roll RunFunc with a found flag.
2. There is no lazy / first-output API. gojq exposes code.Run(v).Next(), which
evaluates lazily, so taking the first output means later sources are never evaluated and
their errors never occur. RunFunc walks eagerly, so a later source's error surfaces even
when the first output was fine:
p, _ := fastjq.Compile(`.a[], .bad.x`)
_, err := p.RunAll([]byte(`{"a":[1,2,3],"bad":"z"}`))
// err = Cannot index string with string "x"
// gojq's .Next() returns 1 and never touches .bad.x
Combined with #31 (a callback error can't stop the walk), "give me the first output" is
currently not expressible without also swallowing genuine errors from outputs the caller
never asked for.
Proposal
Either of these would resolve both halves:
RunFirst(input []byte) ([]byte, bool, error) — returns the first output and whether
there was one, stopping evaluation there. Directly serves the common "predicate" and
"extract one field" cases, and is the minimal fix.
- A lazy iterator (
p.Iter(input) returning something with Next() ([]byte, bool) plus
an Err()), matching gojq's shape. More work, but it makes short-circuiting composable
and would let first(...)/limit(...) semantics fall out naturally for callers.
If neither is in scope, then at minimum Run's doc comment should say that a null return
is ambiguous and point callers at RunFunc — right now the ambiguity is silent.
Reference
- jq manual, empty: "
empty returns no results at all"
- gojq's lazy iterator:
Code.Run →
Iter.Next()
Expected allocation tier
RunFirst should be strictly cheaper than the status quo — it can stop at the first output
instead of walking everything, and needs no result slice. A RunFirstWithBuffer variant
would keep it zero-alloc for repeated calls, matching RunWithBuffer.
Context
Found while replacing gojq with fastjq in the Datadog Agent, where the two call sites are
"evaluate a boolean predicate over a config document" and "extract one scalar" — both of
which want first-output-or-nothing semantics.
Operation / feature
Two related gaps in expressing output cardinality through the current API:
1.
Runcannot distinguish "no output" from "onenulloutput". Both return("null", nil):In jq these are different:
emptyproduces zero outputs,nulland.missingeachproduce one. A caller that needs to tell "the filter matched nothing" from "the filter
matched and the value is null" — a boolean predicate, an existence check — has to abandon
Runand hand-rollRunFuncwith afoundflag.2. There is no lazy / first-output API. gojq exposes
code.Run(v).Next(), whichevaluates lazily, so taking the first output means later sources are never evaluated and
their errors never occur.
RunFuncwalks eagerly, so a later source's error surfaces evenwhen the first output was fine:
Combined with #31 (a callback error can't stop the walk), "give me the first output" is
currently not expressible without also swallowing genuine errors from outputs the caller
never asked for.
Proposal
Either of these would resolve both halves:
RunFirst(input []byte) ([]byte, bool, error)— returns the first output and whetherthere was one, stopping evaluation there. Directly serves the common "predicate" and
"extract one field" cases, and is the minimal fix.
p.Iter(input)returning something withNext() ([]byte, bool)plusan
Err()), matching gojq's shape. More work, but it makes short-circuiting composableand would let
first(...)/limit(...)semantics fall out naturally for callers.If neither is in scope, then at minimum
Run's doc comment should say that anullreturnis ambiguous and point callers at
RunFunc— right now the ambiguity is silent.Reference
emptyreturns no results at all"Code.Run→Iter.Next()Expected allocation tier
without buffering structured data; see
docs/CONSTRAINTS.md).RunFirstshould be strictly cheaper than the status quo — it can stop at the first outputinstead of walking everything, and needs no result slice. A
RunFirstWithBuffervariantwould keep it zero-alloc for repeated calls, matching
RunWithBuffer.Context
Found while replacing gojq with fastjq in the Datadog Agent, where the two call sites are
"evaluate a boolean predicate over a config document" and "extract one scalar" — both of
which want first-output-or-nothing semantics.