Add UTS test specs for LiveObjects path-based API#473
Open
paddybyers wants to merge 31 commits into
Open
Conversation
paddybyers
force-pushed
the
uts-integration-proxy
branch
from
f355624 to
2e12074
Compare
paddybyers
force-pushed
the
uts-liveobjects
branch
from
f8f1b71 to
08789a6
Compare
paddybyers
force-pushed
the
uts-liveobjects
branch
from
08789a6 to
97d02db
Compare
Complete portable test suite covering the LiveObjects path-based API: 21 files across unit tests (pure + mock WebSocket), integration tests (sandbox), and proxy integration tests. Covers PathObject, Instance, BatchContext, LiveCounter/LiveMap CRDTs, ObjectsPool sync state machine, value types, subscriptions, and GC. Includes table-driven validation tests, bytes/binary data coverage, and REST fixture provisioning. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove client-side allocated_port/port_base patterns from all proxy test specs and helper docs. Port is now auto-assigned by the proxy when omitted from create_proxy_session(). Matches uts-proxy v0.2.0. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Align all ~330 LiveObjects UTS test specs with the squashed spec revision a397e34 (LiveObjects path-based API). Key changes: - Add parent_references.md (20 tests): RTLO3f, RTLO4g/4h, RTLO4f, RTO5c10 - Add public_object_message.md (13 tests): PAOM1-3, PAOOP1-3 - Thread ObjectMessage through all CRDT operations and LiveObjectUpdate - Add RTO25 (access preconditions) and RTO26 (write preconditions) - Update subscription model: subscribe returns Subscription object - Add RTO24 (PathObjectSubscriptionRegister) dispatch tests - Add parentReferences maintenance tests to live_map.md (+8 tests) - Add post-sync parentReferences rebuild tests to objects_pool.md (+3) - Rename "consume"/"consumption" to "evaluate"/"evaluation" in value_types - Remove batch.md (Batch API deferred from current spec revision) - Remove subscribeIterator and LiveObject#unsubscribe tests - Update PLAN.md to reflect new file structure and test counts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
paddybyers
force-pushed
the
uts-liveobjects
branch
from
97d02db to
2fba05e
Compare
sacOO7
reviewed
Jun 1, 2026
sacOO7
reviewed
Jun 1, 2026
sacOO7
reviewed
Jun 1, 2026
sacOO7
reviewed
Jun 1, 2026
sacOO7
reviewed
Jun 1, 2026
Collaborator
There was a problem hiding this comment.
- We also need to update
Spec File Countssection at uts/README.md. - Looks like the
Directory structuresection needs an update as well. We could potentially remove it altogether—I’m not sure it serves much purpose anymore. WDYT?
sacOO7
reviewed
Jun 2, 2026
sacOO7
reviewed
Jun 2, 2026
sacOO7
reviewed
Jun 2, 2026
sacOO7
reviewed
Jun 2, 2026
sacOO7
reviewed
Jun 2, 2026
sacOO7
reviewed
Jun 2, 2026
sacOO7
reviewed
Jun 2, 2026
sacOO7
reviewed
Jun 2, 2026
sacOO7
reviewed
Jun 2, 2026
sacOO7
reviewed
Jun 2, 2026
sacOO7
reviewed
Jun 2, 2026
Instead of just checking score == 100 after re-sync (which proves nothing since OBJECT_SYNC resets the pool regardless), replay a previously-applied serial after re-sync. If appliedOnAckSerials was properly cleared, the replayed message applies normally and score reaches 110. If not cleared, dedup (RTO9a3) rejects it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add missing protocol message actions from TR2: OBJECT (19), OBJECT_SYNC (20), and ANNOTATION (21). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2 tasks
Member
Author
sacOO7
reviewed
Jun 9, 2026
| @@ -0,0 +1,385 @@ | |||
| # LiveObject Subscribe Tests | |||
|
|
|||
| Spec points: `RTLO4b`, `RTLO4b3`, `RTLO4b4c1`, `RTLO4b4c3a`, `RTLO4b4c3c`, `RTLO4b4d`, `RTLO4b4e`, `RTLO4b6`, `RTLO4b7` | |||
Collaborator
sacOO7
reviewed
Jun 9, 2026
sacOO7
approved these changes
Jun 9, 2026
ttypic
approved these changes
Jun 9, 2026
The `provision_objects_via_rest` helper described the legacy pre-V2 REST shape
(`POST .../objects` with a `{ "messages": [ { operation: { action, ... } } ] }`
envelope), which no longer matches the LiveObjects objects REST API.
Update it to the V2 contract (per the LiveObjects OpenAPI specification):
- endpoint is singular `POST .../object`;
- the body is a single operation object, or a bare JSON array (batch) — no
`messages` envelope;
- an operation is identified by its payload key (`mapSet`/`mapRemove`/`mapCreate`/
`counterInc`/`counterCreate`) with an `objectId` or `path` target (and optional
idempotency `id`); values are `{string|number|boolean|bytes|objectId}`;
- `mapCreate.semantics` is the integer 0 (LWW) with `{data:}`-wrapped entries.
Also update the only call site (RTPO15 in objects_lifecycle_test.md) to the V2
operation shape so the spec stays internally consistent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The REST fixture-provisioning helper still POSTed to https://sandbox-rest.ably.io, the legacy prod-sandbox host. Every other objects integration spec — and the whole UTS integration corpus (realtime + rest) — provisions apps and routes clients via https://sandbox.realtime.ably-nonprod.net (raw HTTP) / endpoint: "nonprod:sandbox" (SDK clients). The objects integration specs were migrated to the nonprod host in e57d340, but standard_test_pool.md was missed, leaving provision_objects_via_rest posting fixtures to a different backend than the app/key were provisioned on — so RTPO15 (rest-provisioned-data-sync) could never resolve. Point the helper at the canonical nonprod host to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Realtime client option blocks in objects_lifecycle_test.md (10) and objects_sync_test.md (5) set no host, so the SDK falls back to the production endpoint (realtime.ably.io, REC1a) — meaning the clients never reach the nonprod sandbox the "Sandbox Setup" provisions the app on. Every other UTS integration spec (realtime + rest) configures clients with endpoint: "nonprod:sandbox"; add it here to match so the tests actually exercise the sandbox. Added missing auto-connect false to clientOptions
[AIT-1071] fix(uts/objects): Fix liveobjects UTS test spec
Member
Author
|
Could we avoid merge commits like 61f16f8 ? |
…uidance
Fixes found while evaluating the derived ably-java ObjectsFaultsTest against
the sandbox through uts-proxy:
- objects_faults.md: restructure the RTO20e test to the RTO20e1 sequence
(mutation in flight while SYNCING, then channel enters FAILED -> 92008).
The previous steps mutated after FAILED, which per RTO26b fails with 90001
and never reaches publishAndApply - confirmed against objects-features.md,
ably-js source and both ably-js/ably-java behaviour.
- objects_faults.md: rule match.action values must be strings ("20"), not
JSON numbers (uts-proxy rejects numbers with HTTP 400); bound every await
with a timeout; extend the action-number table (ACK, ERROR); align the
Common Cleanup with multi-client tests.
- connection_resume.md: RTN15g rule used invalid action type "refuse"
(silently passes through) - corrected to "refuse_connection".
- helpers/proxy.md: document the match.action string format, the v0.3.0
action-name resolution limit (names beyond AUTH need numeric strings), and
add a wall-clock timeout convention.
- docs: add wall-clock timeout guidance for integration specs
(writing-derived-tests.md new section, integration-testing.md and
writing-test-specs.md pointers) - virtual-time test frameworks (e.g.
kotlinx-coroutines runTest) fire spec timeouts instantly unless waits run
on the real clock.
- PLAN.md: update the objects_faults.md summary row to the corrected RTO20e
scenario.
proxy.md is cross-module infrastructure documentation, not a realtime helper: it is consumed by the realtime (7 proxy specs), rest (rest_fallback.md) and objects (objects_faults.md) modules plus both process docs, unlike the genuinely module-local helper specs (mock_websocket.md, mock_http.md, standard_test_pool.md) which are each used only by their own module. Its old location forced cross-module reach-ins and implied it was realtime-specific. - git mv uts/realtime/integration/helpers/proxy.md -> uts/docs/proxy.md, leaving a one-line stub at the old path so external deep links keep resolving - update all 19 references across 12 files, normalising the pre-existing stale `uts/test/` prefix carried by the realtime proxy specs' references and replacing two mentions of the long-gone in-repo `uts/test/proxy/` directory with a link to the ably/uts-proxy repo - update the directory-tree diagrams in uts/README.md, docs/integration-testing.md and docs/writing-test-specs.md - adjust proxy.md's own cross-reference to sibling style (writing-derived-tests.md) No content changes to the spec itself beyond that one internal reference. Note: ~80 unit/proxy spec files still carry stale `uts/test/...` legacy paths unrelated to proxy.md — left for a separate cleanup PR.
Align the uts/objects test specs with the canonical objects-features.md on main, and fold in the test-correctness fixes from the objects UTS audit. Reconciliation with main: - Rename the internal CRDT type LiveMap/LiveCounter -> InternalLiveMap/ InternalLiveCounter (271 refs), and rename the four CRDT spec files to internal_live_*.md, repointing all cross-references. Blueprint .create() factories, public PathObject/Instance, and LiveMapUpdate/ LiveMapValue are deliberately left unchanged. - get() precondition RTO23b -> RTO23e: get() now performs ensure-active-channel (RTL33) -- re-attaches on DETACHED, rejects 90001 only on FAILED -- instead of unconditionally throwing 90001. - Rename blueprint value types LiveMapValueType/LiveCounterValueType -> LiveMap/LiveCounter (static .create() factories), matching main. - Consolidate RTO25 (access) / RTO26 (write) preconditions as dedicated sections in realtime_object.md, fixing 8 references to two precondition files that never existed; drop the mislabeled RTO25b-via-get() tests. - public_object_message: replace the ably-js-private _derivedFrom with a spec-neutral derivedFrom (RTLCV4g5/RTLMV4j5 keep this retained Create local-only and unnamed). Test-correctness fixes (objects UTS audit): - Quiescence-barrier pattern for negative subscription assertions. - Tombstone / echo-dedup handling; map-clear (RTLM24) and LWW-reject (RTLM9b) semantics; path depth-coverage (RTO24b2a1) fixes. - ACK-serial helper + SITE_CODE in standard_test_pool; explicit REST per-op cardinality; assorted spec-ID and expected-event corrections. Validated against main objects-features.md and cross-checked with ably-js; spec naming is preferred over ably-js where the two differ.
…specs
Implementing the reconciled objects specs against ably-js surfaced a
serial-ordering defect: the canonical ack_serial value "ack-{msgSerial}:{i}"
and the "s:1"/"s:2" serials in the RTO24a/RTO24c1 tests sort BEFORE the
standard pool's "t:0" entry timeserials under the string LWW comparison
(RTLM9), so a strictly-derived SDK rejects those operations as stale -
every local apply-on-ACK MAP_SET on a pool entry fails, and the RTO24c1
root MAP_SET never fires its subscription.
- ack_serial redefined to the sort-safe "t:" + (msgSerial + 1) + ":" + i
form, with a NOTE documenting the ordering constraint.
- All 8 inline auto-ACK harness copies now call ack_serial(msg.msgSerial, i)
instead of duplicating the literal, per the helper's own "never hardcode"
convention; the 3 literal "ack-0:0" uses in the echo-dedup and
ack-after-echo tests reference ack_serial(0, 0).
- RTO24a/RTO24c1 serials "s:1"/"s:2" changed to "t:1"/"t:2"; the adjacent
rationale comments rewritten to state both the site-level newness and
entry-level LWW ordering facts.
- Removed the stale note-only Test IDs RTLC12b/increment-requires-publish-0
and RTLM20d/echo-messages-false-0 (their sections carry no steps or
assertions - the preconditions live in realtime_object.md RTO26) so tag
sweeps stop reporting them as permanently missing tests.
- Added the missing RTLCV3c/no-validation-at-create-0 section to
value_types.md (no validation at blueprint creation; rejection happens
at evaluation per RTLCV4a).
Validated by the derived ably-js suite (ably/ably-js#2257): 309 passing,
0 failing with these values; the original values reproduce 7 failures.
…l helpers
Correct three self-inconsistencies in the objects UTS unit specs (verified
against the ably-js reference tests and SDK):
- SI-1: counter object-states double-counted the created value. Counter sync is
additive (RTLC6c+RTLC6d/RTLC16: data = count + createOp.count), so `count:N` +
`createOp:N` materialised 2N and contradicted every consumer asserting N. Set
the residual `count` to 0 (createOp keeps the initial value) across all 16
affected fixtures in standard_test_pool, objects_pool, parent_references and
path_object_subscribe.
- SI-2: inbound MAP_SET/MAP_REMOVE on existing pool entries used bare serials
("99"/"100"/"101") that sort before the pool baseline "t:0" under lexicographic
LWW (RTLM9e), so they were rejected as stale and silently defeated the tests.
- SI-3: RTO20f reused the apply-on-ACK serial, which RTO9a3 echo-dedup discards
before the newness check (value stays 110, contradicting its own assertion of
120 and the sibling RTO20 dedup test). Use a serial that is not the ACK serial
but sorts below it, so it genuinely exercises RTLC7c.
Add a named serial vocabulary to the Canonical Constants so the LWW ordering
intent is explicit and used at every synthetic inbound serial: POOL_SERIAL
(baseline), remote_serial(i) (remote op that wins vs the baseline) and
below_ack_serial(i) (a non-ACK serial that loses to the ACK serial).
…e RTPO19e2 comment
standard_test_pool.md: a createOp is a full ObjectOperation whose
`action` and `objectId` are mandatory (OOP2) and validated by SDKs
before merging — yet every fixture in the module writes the terse form
`createOp: { counterCreate: {...} }` and build_object_state copied it
verbatim, producing wire-invalid states as literally written. Both SDK
suites already compensate (ably-java's builders embed action and inject
objectId; ably-js spells both out in every fixture). Codify that:
build_object_state now fills in missing `objectId` (from the state's
objectId) and `action` (COUNTER_CREATE/MAP_CREATE by payload), keeping
all terse fixtures valid without edits. Explicit fields are preserved,
so deliberately-mismatched negative fixtures remain expressible.
path_object_subscribe.md: rewrite the stale RTPO19e2 comment (it
described an older two-message MAP_SET flow) to match the actual
fixture: a single OBJECT_SYNC updating counter:score via replaceData
(100 -> 200, RTLC6), with root intentionally omitted — per RTO5c2a the
root must never be removed from the pool, so the update stays reachable
and dispatches with message omitted.
Review comments (PR #499): - Grammar: "a Internal*" -> "an Internal*" at the 4 flagged sites plus a 5th the review missed (internal_live_map.md tombstone prose). - Add objectsGCGracePeriod: 86400000 to the 3 minimal mock CONNECTED connectionDetails (RTO23a, RTO23e-DETACHED, RTO23e-FAILED) — the other 12 blocks already carried it, and 86400000 is the spec's own default (RTO10b3); omitting it invited derived-test failures for fixture reasons unrelated to each test's intent. - Link the LiveObjects REST API docs (built from the OpenAPI spec) in the V2-format REST provisioning section. Naming audit (LiveMap/LiveCounter vs InternalLive*): - RTPO3 heading + requirement row said path resolution walks "LiveMaps"; per RTPO3a1 the walked objects are InternalLiveMaps — the section body was renamed but the heading/table lagged. - build_object_state comment: createOp validation happens on the internal CRDT object — "a LiveCounter rejects" -> "an InternalLiveCounter rejects" (comment was added after the mass rename, so no sweep could catch it). Behavioral audit: - internal_live_map_api.md header over-claimed RTLM24 and RTLM13: the file has no MAP_CLEAR test (RTLM24 is the inbound apply, tested in internal_live_map.md — the main spec defines no public clear()) and no values() test (RTLM13 is untested everywhere, a known gap). Header corrected to RTLM5, RTLM10–RTLM12, RTLM20–RTLM21, RTLMV4, RTLCV4 — verified against the tests actually present. All claims cross-validated against objects-features.md (RTLM5/10-13/ 20/21/24, RTO10b1/b3, RTPO3a1, RTLC6d/RTLM6d).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
uts/objects/Test plan
LiveMap#clear())🤖 Generated with Claude Code