Make the v20 optical byte map executable: name its eleven fields, CRC-gate the decoder

[vishk23]

The v20 (2,140-byte) optical record's byte layout is established and published in #423 — but nothing in the tree can read a field out of one. Whoop5RawOptical (from #546) gets the shape right: five 422-byte blocks, a 21-byte head, two 200-byte reading slots. It keeps the head as an opaque [UInt8] blob and never verifies the frame's checksum.

This PR makes the layout executable: the eleven fields inside the head are named and typed, and the decoder is CRC-gated. Decoder and tests only — no storage, no analytics, no consumers. It should be reviewable and useful on its own.

1. The eleven fields (block-relative)

The 21-byte head is fully accounted for — 1+1+2+1+2+1+4+2+1+4+2 = 21, no residue. Both detector groups are 7 bytes with the identical internal shape select:1, range:4, offset:2, which is what makes the 7-byte stride visible in a raw hex dump of block 0.

rel	width	name	rel	width	name
0	u8	sample_count	8:11	u32 LE	range_a
1	u8	source_a	12:13	i16 LE	offset_a
2:3	u16 LE	drive_a	14	u8	detector_b_select
4	u8	source_b	15:18	u32 LE	range_b
5:6	u16 LE	drive_b	19:20	i16 LE	offset_b
7	u8	detector_a_select	21:220 / 221:420	50 x i32 LE	readings_a / readings_b

Names are exactly the neutral ones from the #423 write-up, so the code and the public record agree. They describe what each field is observed to do and assert nothing about registers, units, or wavelength. The raw 21 bytes are still preserved — rawHeader round-trips them, and a test asserts it reproduces the on-wire bytes. The named view is an interpretation of those bytes, not a replacement for them.

2. Correction: the CRC input range is [8:2136], not [26:2136]

If you built anything against the earlier number, it rejects every real record. My 2026-07-17 comment in #423 (and the body of #546) said the five blocks fill [26:2136] "where the CRC32 sits". The tiling is right; the CRC range was wrong.

zlib.crc32(rec[8:2136]) == u32LE@2136   # 29,203 / 29,203

It starts at the record-class byte. This matches @digitalerdude's independent v26 finding earlier in the thread — CRC-32 over bytes 8..83 validating for all 1,080 of their records at a completely different frame size. Same frame family, same rule, so byte 8 is general rather than a v20 quirk. That cross-format agreement is what makes me confident enough to correct the record rather than just patch a constant.

And the useful consequence: this is not a v20-specific CRC at all — it is the ordinary WHOOP 5 envelope rule the repo already implements. @2:3 declares 2132, so the envelope's payload span [8, declaredLength+8-4) is [8, 2136). The decoder therefore calls verifyFrame(_:family:.whoop5) / Framing.frameCrcOk rather than growing its own checksum.

Small new finding while wiring that up: header bytes 6:7 — listed as "unidentified" in the #423 table — are the envelope's CRC16-Modbus over bytes [0:6], holding 29,203/29,203. They read as a constant 0xB3B5 in that capture only because bytes 0–5 are constant. One fewer unknown byte, and it means the whole record sits inside the already-known envelope.

3. CRC gating (the actual defect being fixed)

The previous decoder documented that callers "should also use the normal envelope CRC verification path" and then didn't. Its own shape test built an all-zero 2,140-byte buffer with three magic bytes set and decoded it successfully. Bad bytes could become data.

decode now verifies the envelope and both checksums before reading a single payload field, per the BLE safety contract. Consequences, all intended:

• parseFrame's field annotation and Whoop5OpticalExperimentAnalyzer now skip corrupt records instead of annotating/analysing them.
• Hand-built test frames must carry real checksums. Both platforms' fixtures now seal, and OpticalExperimentAnalysisTests re-seals after it mutates a header byte.

4. Readings: signed 20-bit, already sign-extended on the wire

The containers arrive sign-extended into 4 bytes, so a plain signed 32-bit read is the entire decode — no masking, no re-extension from bit 19. That is measured, not assumed: across all 4,380,450 populated containers in the corpus the fourth byte is only ever 0x00 (4,057,474) or 0xFF (322,976) and never anything else, and decoded values span [-148,044, 524,287].

Worth stating because the wrong version is not obviously wrong from the numbers: masking to 20 bits and zero-extending, or reading the container unsigned, turns the real sample -22,101 (wire ab a9 ff ff) into 1,048,491 or 4,294,945,195. Both look like plausible ADC counts. A golden vector pins that exact value.

The positive rail is exactly 2^19-1 = 524,287 — hit and never exceeded — so sampleMin/sampleMax are exposed and asserted rather than left as folklore.

5. Tests

A shared r20_optical_oracle.json, byte-identical in Packages/WhoopProtocol/Tests/WhoopProtocolTests/Resources/ and android/app/src/test/resources/, same idiom as decoder_oracle.json (#869, #889). Two real captured records; the expected values were generated by an independent third implementation of the layout (Tools/gen_r20_oracle.py, plain struct.unpack + zlib.crc32), so Swift and Kotlin each have to reproduce them rather than agreeing with each other by construction. A test asserts the two copies stay identical, and the generator writes both at once.

Covered on both platforms: full per-field decode of all five blocks; the [25, 0, 0, 25, 25] pattern with sample_count == 0 yielding no readings (not 50 zeros); block 3 confirmed as the drive-off dark control; negative readings sign-extending; saturation exactly at 2^19-1; the CRC range asserted directly, including that the retracted [26:2136] does not match; and five single-bit corruptions — in a reading, in sample_count, in drive_a, in the CRC trailer, and in the declared length — each of which must be rejected. The generator proves each mutation really does break a checksum, so the fixture can never assert "rejected" for a frame that is in fact valid.

Results (both run on this branch's head):

• Swift — swift test in Packages/WhoopProtocol: 422 tests, 0 failures, of which 10 are Whoop5RawOpticalTests.
• Kotlin — ./gradlew :app:testFullDebugUnitTest: 3187 tests, 1 failed, 5 skipped. Whoop5RawOpticalTest is 9 tests, 0 failures, 0 skipped, read out of the JUnit XML rather than trusted from the exit code.

On that one Kotlin failure — it is pre-existing on main and unrelated to this PR. com.noop.data.DeepCaptureMigrationTest.repositoryInsertV18Aux_insertsThenPrunes fails with expected:<my-whoop> but was:<null>. #888 made the v18-aux retention sweep amortized — WhoopRepository.insert now prunes only once banked >= V18_AUX_PRUNE_EVERY_ROWS (10,000) — but the test still inserts a single row and asserts the prune fired, so it cannot pass. Nothing in com.noop.data references Whoop5RawOptical. It goes unnoticed because Kotlin unit tests don't run on PRs. Happy to send that as a separate one-line PR if you'd like it.

6. Full-corpus verification

Both suites carry an opt-in test that runs the decoder over an entire deep-buffer capture and asserts the published invariants on every record — the block pattern, drive_a inside its six-value set, drive_b == 2 * drive_a, offsets multiples of 800, reserved zero, reading counts matching sample_count, every sample inside the signed 20-bit domain, and the decoded timestamp matching the recorder's own strap_ts.

WHOOP_R20_CORPUS=/path/to/buffers2140.jsonl swift test --filter testFullCorpus
WHOOP_R20_CORPUS=/path/to/buffers2140.jsonl ./gradlew :app:testFullDebugUnitTest --tests '*Whoop5RawOpticalTest.fullCorpus*'

It skips when the variable is unset (the capture is far too large to commit), so CI is unaffected. Against the 29,203-record WHOOP 5.0/MG capture behind #423:

R20 corpus: 29203/29203 decoded, 0 invariant mismatches   # Swift
R20 corpus: 29203/29203 decoded, 0 invariant mismatches   # Kotlin

Two golden vectors prove the offsets; this proves they hold at scale, on both platforms, which is the claim #423 actually makes.

7. Scope, and what is deliberately not claimed

Not wired into storage or analytics. No production consumer reads any of these fields. Persistence and downstream analytics for this record are being designed separately; keeping them out means this can be reviewed as what it is — a byte-layout decoder with vectors.

Unchanged from #546, and restated in the source: the two slots under one head belong to one shared measurement configuration and must not be labelled as two wavelengths. drive_* is named only because a block with both drive fields at zero produces a dark measurement with no pulse (block 3). Its units are not established, what source_* values {1,2,3,4} select is unknown, and nothing in this corpus identifies an emission band.

Everything here was measured from NOOP's own BLE captures via the #454 recorder — offsets, widths, endianness, value sets and counts are all reproducible from the capture alone.

Thanks to @digitalerdude, whose public capture and v26 CRC-from-byte-8 result are what corroborate §2 across two frame sizes.

Refs #423.