Preserve torch.Size through provenance tracking - #3467
Draft
Mohit-Ak wants to merge 1 commit into
Draft
Conversation
torch.Size subclasses tuple, so it was being handled by the list/tuple/dict pytree branches of track_provenance and extract_provenance. tree_flatten decomposes it into its plain int leaves and tree_unflatten rebuilds it as an ordinary tuple, so every torch.Size that passed through the provenance machinery came back out with its type erased. That matters because torch APIs overload on the distinction. Tensor.new() reads a torch.Size as a shape but a tuple as data, so inside ProvenanceTensor.__torch_function__ the detach_provenance() call turned Multinomial.sample()'s counts = samples.new(self._extended_shape(...)) into a 1-element tensor instead of a correctly shaped one, and the following scatter_add_ raised "index 2 is out of bounds for dimension 0 with size 1". Register torch.Size on both singledispatch functions and mark it as a leaf in the two pytree helpers so nested occurrences (args/kwargs) survive as well.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3436.
The problem
torch.Sizesubclassestuple. That means it matches thelist/tuple/dictbranches oftrack_provenanceandextract_provenance, which run it throughtree_flatten/tree_unflatten. Those decompose theSizeinto its plain int leaves and rebuild it as an ordinarytuple, so anytorch.Sizepassing through the provenance machinery comes back out with its type erased:That looks harmless, but torch overloads on exactly this distinction:
Tensor.new()treats atorch.Sizeas a shape and atupleas data.Multinomial.sample()doescounts = samples.new(self._extended_shape(sample_shape)).zero_(). Underrender_modelthe sample is aProvenanceTensor, so__torch_function__routes the args throughdetach_provenance(), the shape degrades to a tuple,countscomes back with shape(1,)instead of(3,), and the next line fails:This is why the issue only shows up for
Multinomialand only insiderender_model— you need a distribution that reconstructs a tensor from its own shape, plus provenance tracking being active.The fix
Two parts, and both are needed:
torch.Sizeontrack_provenanceandextract_provenance. ASizeholds plain ints, so there is no provenance to add or extract — return it unchanged.is_leaf=_is_sizeto thetree_map/tree_flattencalls in the pytree branches.Part 2 is the one that actually repairs the reported crash. The failing path is
detach_provenance([args, kwargs]), where theSizeis nested inside a list, so the pytree helper flattens it before singledispatch ever gets a chance to see it. I checked this rather than assuming — registering the dispatch alone leaves the bug in place for every nested case:is_leafSizeSizein args tupleSizenested in listSizein kwargsThe control row confirms ordinary tuples still flatten normally.
Testing
Added to
tests/ops/test_provenance.py: a parametrized round-trip test covering bare/2-D/emptySizeplus nesting in args, kwargs, and mixed with a tensor; aTensor.new()test pinning the shape-vs-data behavior; and aget_model_relationstest that is the issue's reproducer.RED→GREEN, verified in a detached worktree so the buggy code ran against the new tests:
Wider suites at the committed SHA:
The 3 failures are
test_gaussian_funsor—ModuleNotFoundError: No module named 'funsor'. They fail identically on pristine upstream (checked in a clean worktree), so they're a missing optional dep, not this change.make lintgates on the touched files:ruff checkclean,black --checkclean,mypy pyro/ops/provenance.pyclean.