Numeric semantics diverge from jq: >2^53 arithmetic loses precision, overflow becomes null, output not canonical

[gh123man]

Description

Three numeric divergences from jq that share one root cause — arithmetic goes through
float64, while pass-through copies the input bytes. Grouped because they'd likely be
addressed (or documented) together.

The existing Limitations section covers nan/infinite serializing to null, but none
of the three below are mentioned, and the first two are silent data corruption rather than a
visible null.

1. Integer arithmetic silently loses precision past 2^53

Pass-through is exact, so the value looks fine until a query touches it:

query	input	fastjq	jq 1.8.1 / gojq
.x	{"x":9007199254740993}	9007199254740993 ✓	9007199254740993
.x + 1	{"x":9007199254740993}	9007199254740992 ✗	9007199254740994
.x * 1	{"x":9007199254740993}	9007199254740992 ✗	9007199254740993

gojq uses math/big for integers; jq 1.8.1 preserves integer literals and does exact
arithmetic within int64 range. The fact that pass-through is correct is what makes this
dangerous — a test that reads a large ID through .x passes, and the same ID silently
corrupts the moment someone adds + 1 or | floor.

2. Arithmetic overflow silently becomes null

.x * 10   over {"x":1e308}   → null        (jq: 1.7976931348623157e+308)

This follows from the documented infinite → null normalization, but overflow arising from
arithmetic is a different situation from a literal nan/infinite in the source: the
caller didn't write an infinity, they wrote a multiplication. Quietly replacing a value with
null is worse than either erroring or saturating — in a config-transform context it means
a key silently loses its value.

Also in this area, .x + 0 over {"x":1e400} returns a 300-plus-digit decimal expansion
rather than 1e400, null, or an error, which looks unintended:

.x + 0    over {"x":1e400}   → 1000000000000...000 (309 digits)

3. Number output is the verbatim input literal, not canonical jq form

.x     over {"x":1.0}   → 1.0     (jq: 1)
.x     over {"x":1E2}   → 1E2     (jq: 100)
.x     over {"x":1.50}  → 1.50    (jq: 1.5)
.x + 0 over {"x":1.0}   → 1       (canonicalizes once touched)

So the same value formats differently depending on whether an operation touched it. I
suspect this one is working as intended — it's the direct consequence of the zero-copy
design and I wouldn't want it changed at the cost of the allocation budget. But it means
fastjq output is not byte-comparable with jq's, which is worth stating explicitly for
anyone diffing the two or writing golden-file tests.

Reproducer

package main

import (
	"fmt"

	"github.com/DataDog/fastjq"
)

func main() {
	cases := []struct{ q, in string }{
		{`.x`, `{"x":9007199254740993}`},
		{`.x + 1`, `{"x":9007199254740993}`},
		{`.x * 1`, `{"x":9007199254740993}`},
		{`.x * 10`, `{"x":1e308}`},
		{`.x + 0`, `{"x":1e400}`},
		{`.x`, `{"x":1.0}`},
		{`.x`, `{"x":1E2}`},
		{`.x`, `{"x":1.50}`},
		{`.x + 0`, `{"x":1.0}`},
	}
	for _, c := range cases {
		p, err := fastjq.Compile(c.q)
		if err != nil {
			fmt.Printf("%-8s %-24s compile: %v\n", c.q, c.in, err)
			continue
		}
		out, err := p.RunAll([]byte(c.in))
		if err != nil {
			fmt.Printf("%-8s %-24s err: %v\n", c.q, c.in, err)
			continue
		}
		fmt.Printf("%-8s %-24s => %s\n", c.q, c.in, out[0])
	}
}

Suggested resolution

I'm not assuming big-int support is wanted — it may well conflict with the allocation
budget. Ranked by cost:

1. Document all three under Limitations, next to the existing nan/infinite note.
   That alone would have saved us from shipping (1) unnoticed.
2. Make overflow visible — return an error rather than null when arithmetic overflows,
   and fix the 1e400 decimal-expansion formatting.
3. Exact int64 arithmetic where both operands are integral and the result fits in int64,
   falling back to float64 otherwise. That covers the realistic cases (IDs, counters,
   timestamps in nanoseconds) without a bignum dependency, and stays zero-alloc.

Environment

• fastjq version: c3336f252fa23bbda9a611024f6398904330cd9a (master, untagged)
• Go version: go version go1.25.4 darwin/arm64
• OS / arch: macOS, arm64