Skip to content

Derive generators on inductives with struct params - #37

Closed
mwhicks1 wants to merge 3 commits into
mainfrom
specimen-struct-param
Closed

Derive generators on inductives with struct params#37
mwhicks1 wants to merge 3 commits into
mainfrom
specimen-struct-param

Conversation

@mwhicks1

Copy link
Copy Markdown
Contributor

Support structure-parameterized output types in the constrained deriver derive_generator/derive_mutual could not produce values of an inductive parameterized by a structure (e.g. Strata's LExpr T where T : LExprParams carries the metadata/identifier/type-annotation types). The structure parameter T.mono : LExprParamsT rides along as an argument of every constructor, and the deriver tried to generate it — yielding a Type 1-in-Type universe error — and had no way to generate the per-constructor metadata fields (m : T.base.Metadata). This mirrors the gap that Specimen.DeriveArbitrary fixes for the unconstrained deriving Arbitrary path via expandStructBinders.

Three coordinated changes make the fully-parameterized relation derivable:

  1. Don't lift fixed, ungeneratable subterms during conclusion flattening (Utils.lean). collectUnmatchableSubterms now skips a subterm whose free variables are all fixed inputs and whose type has no Arbitrary instance (e.g. T.mono). Ordinary fixed value subterms like n * n : Nat still flatten (they have Arbitrary), so checker behavior is unchanged. The fixed-input set is the conclusion's non-output bare-variable arguments (DeriveConstrainedProducer).

  2. Emit per-field producer instance binders for structure parameters (MakeConstrainedProducerInstance.lean). New expandStructInstBinders / mkProducerParamInstBinders mirror expandStructBinders: a sort param α keeps [Arbitrary α] [DecidableEq α]; a structure param T expands to [Arbitrary T.base.Metadata] … . Threaded through both the single-instance and mutual-def emission paths (the latter places struct-field binders innermost, where the value param they reference is in scope).

  3. Drop implicit constructor arguments from conclusion outputs (MExp.lean). dropImplicitCtorArgsExpr removes implicit-position args of each data constructor in the produced value (e.g. LExpr.const's T, Option.some's α) so the implicit-allowing emission re-infers them instead of mis-placing them positionally.

StrataLexprGen now derives a generator for the genuine @LExpr.HasTypeA T … (abstract T, no monomorphization) and its embedded #eval confirms soundness: 60/60 sampled terms type-check, using the LExpr typechecking algorithm, at the requested type. Full test suite green.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Comment thread SpecimenTest/StrataLexprGen.lean Outdated
Comment thread Specimen/Utils.lean Outdated
Comment thread Specimen/Utils.lean Outdated
-- The field's declared type, read as the codomain of the projection's
-- signature `∀ (_ : sName ..), fieldType`. Structure fields here are the
-- metadata-configuration types, whose field types do not depend on the value.
let projType ← forallTelescopeReducing (← inferType (mkConst projName))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we call mkConst without specifying universe levels -- this is fine for Strata in practice since the parameterized types are all Type 0. However, in the future, if we have types parameterized by universe-polymorphic structures (see example below), the projType would be computed incorrectly. To fix this, we would need to extract the universe levels from ty (the type expression for the structure) and pass them to mkConst. I don't think universe-polymorphic structures are that common in practice, so it's fine to leave it out, but wanted to bring this up just in case.

Here's an example of a universe-polymorphic structure parameter:

-- Universe-polymorphic structure type
structure Config.{v} where
    Carrier : Type v
    default : Carrier

-- Config.{0} gives Carrier : Type 0, Config.{1} gives Carrier : Type 1
inductive Tree (C : Config.{u}) : Type u where
    | leaf : C.Carrier → Tree C
    | node : Tree C → Tree C → Tree C

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than try to address this, I left a "Limitation" comment that documents the weakness.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

@ngernest

Copy link
Copy Markdown
Collaborator

Looks generally good to me! Left a few minor comments

@mwhicks1
mwhicks1 force-pushed the specimen-delegated-producer branch from 60fa8b9 to 5dfee8e Compare June 24, 2026 16:42
mwhicks1 added a commit that referenced this pull request Jun 24, 2026
- StrataLexprGen: generate non-integer rationals in Arbitrary Rat via
  mkRat numerator denominator (was Rat.ofInt only).
- Utils: simplify isFixedUngenerable's instance probe to a guarded
  let-bind returning inst.isNone (was a nested match).
- Rename 'generatable' -> 'generable': isFixedUngeneratable ->
  isFixedUngenerable (Utils) and generatableVars -> generableVars
  (SearchTree).
- MakeConstrainedProducerInstance: document the universe-polymorphic
  structure-parameter limitation of mkConst projName (not supported;
  rare in practice).
@mwhicks1
mwhicks1 force-pushed the specimen-struct-param branch from ad94058 to 862613e Compare June 24, 2026 17:13
@ngernest

Copy link
Copy Markdown
Collaborator

Looks great, thanks!

@ngernest
ngernest self-requested a review June 24, 2026 17:33
ngernest
ngernest previously approved these changes Jun 24, 2026
@mwhicks1
mwhicks1 force-pushed the specimen-delegated-producer branch from 5dfee8e to 89e0d5c Compare June 24, 2026 21:25
mwhicks1 added a commit that referenced this pull request Jun 24, 2026
- StrataLexprGen: generate non-integer rationals in Arbitrary Rat via
  mkRat numerator denominator (was Rat.ofInt only).
- Utils: simplify isFixedUngenerable's instance probe to a guarded
  let-bind returning inst.isNone (was a nested match).
- Rename 'generatable' -> 'generable': isFixedUngeneratable ->
  isFixedUngenerable (Utils) and generatableVars -> generableVars
  (SearchTree).
- MakeConstrainedProducerInstance: document the universe-polymorphic
  structure-parameter limitation of mkConst projName (not supported;
  rare in practice).
@mwhicks1
mwhicks1 force-pushed the specimen-struct-param branch from 862613e to b76ef06 Compare June 24, 2026 21:26
mwhicks1 added a commit that referenced this pull request Jun 25, 2026
- StrataLexprGen: generate non-integer rationals in Arbitrary Rat via
  mkRat numerator denominator (was Rat.ofInt only).
- Utils: simplify isFixedUngenerable's instance probe to a guarded
  let-bind returning inst.isNone (was a nested match).
- Rename 'generatable' -> 'generable': isFixedUngeneratable ->
  isFixedUngenerable (Utils) and generatableVars -> generableVars
  (SearchTree).
- MakeConstrainedProducerInstance: document the universe-polymorphic
  structure-parameter limitation of mkConst projName (not supported;
  rare in practice).
@mwhicks1
mwhicks1 force-pushed the specimen-struct-param branch from b76ef06 to c2e1bbb Compare June 25, 2026 01:27
mwhicks1 added 3 commits July 1, 2026 14:40
`derive_generator`/`derive_mutual` could not produce values of an inductive
parameterized by a *structure* (e.g. Strata's `LExpr T` where `T : LExprParams`
carries the metadata/identifier/type-annotation types). The structure parameter
`T.mono : LExprParamsT` rides along as an argument of every constructor, and the
deriver tried to *generate* it — yielding a `Type 1`-in-`Type` universe error —
and had no way to generate the per-constructor metadata fields (`m :
T.base.Metadata`). This mirrors the gap that `Specimen.DeriveArbitrary` fixes for
the unconstrained `deriving Arbitrary` path via `expandStructBinders`.

Three coordinated changes make the fully-parameterized relation derivable:

1. Don't lift fixed, ungeneratable subterms during conclusion flattening
   (Utils.lean). `collectUnmatchableSubterms` now skips a subterm whose free
   variables are all fixed inputs *and* whose type has no `Arbitrary` instance
   (e.g. `T.mono`). Ordinary fixed value subterms like `n * n : Nat` still flatten
   (they have `Arbitrary`), so checker behavior is unchanged. The fixed-input set
   is the conclusion's non-output bare-variable arguments (DeriveConstrainedProducer).

2. Emit per-field producer instance binders for structure parameters
   (MakeConstrainedProducerInstance.lean). New `expandStructInstBinders` /
   `mkProducerParamInstBinders` mirror `expandStructBinders`: a sort param `α`
   keeps `[Arbitrary α] [DecidableEq α]`; a structure param `T` expands to
   `[Arbitrary T.base.Metadata]` … . Threaded through both the single-instance and
   mutual-`def` emission paths (the latter places struct-field binders innermost,
   where the value param they reference is in scope).

3. Drop implicit constructor arguments from conclusion outputs (MExp.lean).
   `dropImplicitCtorArgsExpr` removes implicit-position args of each data
   constructor in the produced value (e.g. `LExpr.const`'s `T`, `Option.some`'s
   `α`) so the implicit-allowing emission re-infers them instead of mis-placing
   them positionally.

StrataLexprGen now derives a generator for the genuine `@LExpr.HasTypeA T …`
(abstract `T`, no monomorphization) and its embedded `#eval` confirms soundness:
60/60 sampled terms type-check at the requested type. Full test suite green.
Vendor `LExpr.typeCheck` (and its `LMonoTy.isArrow` dependency) verbatim from
`Strata/DL/Lambda/Denote/LExprAnnotated.lean` into LambdaCore, and use it in the
StrataLexprGen soundness `#eval` in place of the hand-rolled `typeCheckP`. This
makes the check authoritative: `LExpr.typeCheck` is the actual Strata checker,
proved equivalent to `HasTypeA` upstream. Sampled terms remain 25/25 (and 60/60
across the trial set) well-typed; full suite green.
- StrataLexprGen: generate non-integer rationals in Arbitrary Rat via
  mkRat numerator denominator (was Rat.ofInt only).
- Utils: simplify isFixedUngenerable's instance probe to a guarded
  let-bind returning inst.isNone (was a nested match).
- Rename 'generatable' -> 'generable': isFixedUngeneratable ->
  isFixedUngenerable (Utils) and generatableVars -> generableVars
  (SearchTree).
- MakeConstrainedProducerInstance: document the universe-polymorphic
  structure-parameter limitation of mkConst projName (not supported;
  rare in practice).
@mwhicks1
mwhicks1 force-pushed the specimen-struct-param branch from c2e1bbb to 111d1b1 Compare July 1, 2026 18:44
@mwhicks1
mwhicks1 changed the base branch from specimen-delegated-producer to main July 1, 2026 18:44
@mwhicks1
mwhicks1 dismissed ngernest’s stale review July 1, 2026 18:44

The base branch was changed.

| .Enumerator => ``Enum

let arbitraryTypeParamInstances ← mkTypeClassInstanceBinders typeParams #[producerUnconstrainedClass, ``DecidableEq]
let arbitraryTypeParamInstances0 ← mkTypeClassInstanceBinders typeParams #[producerUnconstrainedClass, ``DecidableEq]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of my PRs modifies the way we track bracketed typeclass instance arguments to only collect those that are truly necessary. It does so by traversing the dependency graph and propagating bottom up the needed dependencies from the schedules. My only concern is how well this change would adapt to that.

@mwhicks1

mwhicks1 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Canceling this in favor of PR #47

@mwhicks1 mwhicks1 closed this Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants