replay_plan: the oracle approach has no NSRTs for this env - #108
Merged
Conversation
Every domino in a captured scene was standing, but two of the four came back with roll = pi and read as Toppled before anything moved. A task whose goal is Toppled(target) was then satisfied in its own initial state: the planner returned a length-0 plan and the run reported SOLVED. A domino is a box, so turning it 180 degrees about its own width axis leaves it exactly where it was. Both orientations describe the same physical domino and a marker-based pose estimate returns either one arbitrarily -- in a single capture, some dominoes come back at roll 0 and others at roll +-pi. Roll is only meaningful modulo pi, so it is now folded into [-pi/2, pi/2): standing (0 or +-pi) folds to ~0, and knocked over (+-pi/2) keeps the magnitude that Toppled and Upright are defined on. Yaw is untouched, so the push direction is unaffected. This is a regression from the toppled-domino support in plan PR 6. Before it, both conversions hard-coded roll = 0, so a flipped estimate could not surface. The offline check against scenes 0000/0001 missed it because every domino in those two captures happened to come back near roll 0. The fold lives in the domino env rather than in domino_env_euler, so the geometry helper stays a faithful decomposition and the "a domino is a box" fact sits with the domain that knows it. Tests: the invariant that no domino starts toppled when built from a capture whose records are flipped, the same for a mid-episode observation, and the fold itself over eight angles. Verified against the real capture that exposed this -- all four dominoes now read 0.00 deg and the goal is no longer true at init.
There was no command that ran the oracle arm on this env: oracle.yaml un-skips fan, and launch_simp.py takes only -c, so running Stage 1 meant editing a shared config that fan runs also use. This is a thin launcher of the kind the repo already uses -- it only un-skips the env and arm it runs. It also documents the trap that sent Stage 1 off the rails: a bare `python predicators/main.py --env pybullet_domino_real --approach oracle` inherits none of the env's flags from envs/all.yaml, and this env does not work on the settings.py defaults. Most sharply, domino_use_domino_blocks_as_target defaults False, which sizes the domino component with the target held as a separate object -- so a 4-domino scene allocates 3 slots and task construction dies with "perceived 4 dominoes but only 3 slots". The scene path is set here rather than in settings.py because envs/all.yaml already sets it, and a config value beats the settings.py default -- so editing settings.py has no effect on a launcher run. Lands with the roll fold so that one branch both runs Stage 1 and answers it truthfully.
replay_plan has been dead for pybullet_domino_real since the CogMan
rewiring landed:
NotImplementedError: Ground-truth NSRTs not implemented for env:
pybullet_domino_real
It built its CogMan with create_approach("oracle", ...) on the reasoning
that the approach is never consulted for control -- an override policy is
always set, so _reset_policy takes that branch and never calls solve. That
part is right, but OracleApproach builds ground-truth NSRTs in its
*constructor*, and this env has none: it is planned over processes, which
is why the process-planning oracle's _get_current_nsrts returns an empty
set. So the replay died before rendering a frame.
Uses random_options instead: a plain BaseApproach that needs nothing but
the option set, which is all a fixed-plan replay requires.
Verified by rendering a single Push on a real captured scene --
`steps=59 goal_reached=True`, MP4 written.
Narrows this PR to the roll fold alone. The launcher is preserved verbatim on the oracle-domino-real-launcher branch.
…search/predicators into fix-replay-plan-approach
…search/predicators into fix-replay-plan-approach
amburger66
marked this pull request as ready for review
July 31, 2026 19:02
amburger66
enabled auto-merge (squash)
July 31, 2026 19:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
scripts/domino_debug/replay_plan.pyhas been dead forpybullet_domino_realsince the CogMan rewiring landed in #104:It dies before rendering a frame.
Cause
The rewiring built CogMan with
create_approach("oracle", ...), on the reasoning that the approach is never consulted for control — an override policy is always set, so_reset_policytakes that branch and never callssolve. That reasoning is correct, but it missed thatOracleApproachbuilds ground-truth NSRTs in its constructor.This env has none. It is planned over processes — which is exactly why the process-planning oracle's
_get_current_nsrts()returns an empty set. Soget_gt_nsrtsraises during construction, before the override ever matters.Fix
Use
random_options: a plainBaseApproachneeding nothing but the option set, which is all a fixed-plan replay requires. The comment now says why, so the next person doesn't reach fororacleagain.Verification
Rendered a single
Pushon a real captured scene:Gates: mypy clean over 709 files, pylint 13/13 on
scripts/domino_debug/, yapf/isort/docformatter clean.Branched off
fix-domino-roll-symmetry(#106) so the roll fold is present, since without it the captured scene's dominoes read as already toppled.