diff --git a/.scratch/value-walk/issues/01-walk.md b/.scratch/value-walk/issues/01-walk.md index e0c4840b..eea9fc6c 100644 --- a/.scratch/value-walk/issues/01-walk.md +++ b/.scratch/value-walk/issues/01-walk.md @@ -19,16 +19,16 @@ visitors over `BinStream::walk`, and the same visitor verifies a repair over `Bi ## Proposed surface +In `ltk_meta::walk`: + ```rust -impl Kind { +/// The one question the walk asks of a `Kind`. Sealed: implemented for `Kind` only. +pub trait TreeKind: Copy + sealed::Sealed { /// Whether a value of this kind is a node: `Struct` or `Embedded`. - pub fn is_node(self) -> bool; + fn is_node(self) -> bool; } -``` +impl TreeKind for Kind {} -In `ltk_meta::walk`: - -```rust /// A value the walk can cross. Sealed: implemented for `&'a PropertyValueEnum` and for /// `ValueView<'a, M>`, and by nothing else. pub trait TreeValue<'a>: Copy + sealed::Sealed { @@ -37,7 +37,7 @@ pub trait TreeValue<'a>: Copy + sealed::Sealed { fn kind(&self) -> Kind; /// Whether entering this value can reach a node: a `Struct` or `Embedded` whose class hash - /// is not 0, or a container, optional or map whose item kind [`Kind::is_node`]. + /// is not 0, or a container, optional or map whose item kind [`TreeKind::is_node`]. fn holds_node(&self) -> Result; /// This value as a node, if it is a `Struct` or `Embedded` with a class hash that is not 0. fn as_node(&self) -> Result, Error>; @@ -52,7 +52,8 @@ pub trait TreeValue<'a>: Copy + sealed::Sealed { } /// A node the walk can visit: a class and properties. Sealed: implemented for the owned -/// tree's node, for `StructView<'a, M>`, and for `ObjectView<'a, M>` as a root. +/// tree's node and for `StructView<'a, M>`. An object's root is walked as a `StructView` +/// over the same bytes. pub trait TreeNode<'a>: Copy + sealed::Sealed { type Value: TreeValue<'a, Node = Self>; type Properties: Iterator>; @@ -97,7 +98,6 @@ impl<'a, M> TreeValue<'a> for &'a PropertyValueEnum { type Node = OwnedNode<' impl<'a, M: Default> TreeValue<'a> for ValueView<'a, M> { type Node = StructView<'a, M>; /* ... */ } impl<'a, M> TreeNode<'a> for OwnedNode<'a, M> { type Value = &'a PropertyValueEnum; /* ... */ } impl<'a, M: Default> TreeNode<'a> for StructView<'a, M> { type Value = ValueView<'a, M>; /* ... */ } -impl<'a, M: Default> TreeNode<'a> for ObjectView<'a, M> { type Value = ValueView<'a, M>; /* ... */ } /// What a callback answers. `ltk_ritobin::cst::visitor::Visit`'s shape (W21). #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] @@ -132,7 +132,8 @@ pub trait Visitor<'a, V: TreeValue<'a>> { /// descended on `Continue`. fn enter_property(&mut self, field: BinHash, value: V, node: &Node<'_, 'a, V>) -> Result { Ok(Visit::Continue) } - /// Once per property descended. Not called for a leaf. Never after an `Abort`. + /// Once per property that holds a node and was entered. Not called for a leaf. Never + /// after an `Abort`. fn exit_property(&mut self, field: BinHash, value: V, node: &Node<'_, 'a, V>) -> Result { Ok(Visit::Continue) } } @@ -208,14 +209,14 @@ impl<'a, M: Default> ObjectView<'a, M> { } impl ObjectStream<'_, R, M> { /// `view()?` then `walk`. - pub fn walk(&mut self, visitor: &mut W) -> Result - where W: for<'a> Visitor<'a, ValueView<'a, M>>; + pub fn walk(&mut self, visitor: &mut W) -> Result + where E: From, W: for<'a> Visitor<'a, ValueView<'a, M>, Error = E>; } impl BinStream { /// Walks every object in file order, one buffered object at a time. Holds one object's /// bytes at any moment and nothing of the tree. - pub fn walk(&mut self, visitor: &mut W) -> Result - where W: for<'a> Visitor<'a, ValueView<'a, M>>; + pub fn walk(&mut self, visitor: &mut W) -> Result + where E: From, W: for<'a> Visitor<'a, ValueView<'a, M>, Error = E>; } ``` @@ -252,7 +253,7 @@ to decode and a visitor reading a leaf can too; the tree's errors convert throug `Stop` is an outcome, not an error. Blocked by #219: `Node::value_path`, `Trail::to_value_path` and `MapKey` are its types. -`Kind::is_node`, the traits, `Leaf`, `Visitor`, `Node`, `Trail` and every entry point do not +`TreeKind`, the tree traits, `Leaf`, `Visitor`, `Node`, `Trail` and every entry point do not depend on it and can land first behind those methods. - [ ] The fixture tree of `value-walk.md` [section 7](https://github.com/LeagueToolkit/league-toolkit/blob/main/docs/design/value-walk.md#s7) is walked twice, owned and as an `ObjectView` of its bytes, through one generic visitor, and a diff --git a/crates/ltk_meta/README.md b/crates/ltk_meta/README.md index d77e4abf..bedde551 100644 --- a/crates/ltk_meta/README.md +++ b/crates/ltk_meta/README.md @@ -191,6 +191,84 @@ fn main() -> Result<(), Box> { } ``` +## Walking a bin + +`ltk_meta::walk` is one traversal for both trees. A `Visitor` is called once per node, in pre-order and file order, and answers a `Visit` that continues, prunes a property, stops or aborts. The walk visits every node of an object once, in pre-order and file order, and asks the visitor before entering each property. The visitor is generic over the tree: the same `Census` runs over an owned `Bin` and over a `BinStream`, where nothing is materialised. + +```rust +use ltk_hash::BinHash; +use ltk_meta::{ + walk::{Node, TreeValue, Visit, Visitor}, + Error, +}; + +/// Counts nodes and records the address of every `Struct` of one class. +#[derive(Default)] +struct Census { + nodes: usize, + hits: Vec<(BinHash, String)>, +} + +impl<'a, V: TreeValue<'a>> Visitor<'a, V> for Census { + type Error = Error; + + fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + self.nodes += 1; + if *node.class_hash() == 0x1e6b_a0c4 { + self.hits.push((node.object_hash(), node.trail().to_string())); + } + Ok(Visit::Continue) + } +} + +fn main() -> Result<(), Box> { + let bin = ltk_meta::Bin::from_reader(&mut std::fs::File::open("data.bin")?)?; + let mut census = Census::default(); + bin.walk(&mut census)?; + println!("{} nodes, {} hits", census.nodes, census.hits.len()); + + let mut stream = ltk_meta::concrete::BinStream::mount(std::fs::File::open("data.bin")?)?; + let mut census = Census::default(); + stream.walk(&mut census)?; + Ok(()) +} +``` + +**In parallel.** The walk over one object is sequential by contract: one visitor, pre-order, `Stop` and `Skip` as ordered decisions. Objects are independent of one another, and every view, node and trail type is `Send`. A sweep parallelises across objects with one visitor instance per worker and a reduce at the end. Nothing in the crate schedules this; the split is the caller's. + +```rust +let objects: Vec<_> = bin.objects.values().collect(); +let workers = std::thread::available_parallelism().map_or(1, |n| n.get()); +let per_worker = objects.len().div_ceil(workers).max(1); + +let counted = std::thread::scope(|scope| { + let workers: Vec<_> = objects + .chunks(per_worker) + .map(|chunk| { + scope.spawn(move || { + let mut census = Census::default(); + for object in chunk { + object.walk(&mut census)?; + } + Ok::<_, Error>(census) + }) + }) + .collect(); + + let mut all = Census::default(); + for worker in workers { + let census = worker.join().expect("a worker panicked")?; + all.nodes += census.nodes; + all.hits.extend(census.hits); + } + Ok::<_, Error>(all) +})?; + +println!("{} nodes, {} hits", counted.nodes, counted.hits.len()); +``` + +Across many files the same shape applies one level up: one task per file, each mounting its own `BinStream` and walking it sequentially. The per-object walk is microseconds; decompression and I/O are where a sweep spends its time. + ## Creating one programmatically The `concrete` module pins the metadata parameter, which is what you want unless you are attaching per-node metadata of your own: diff --git a/crates/ltk_meta/src/lib.rs b/crates/ltk_meta/src/lib.rs index 858da7ad..adb1fda8 100644 --- a/crates/ltk_meta/src/lib.rs +++ b/crates/ltk_meta/src/lib.rs @@ -94,6 +94,97 @@ while let Some(mut object) = objects.next()? { [`Bin::from_reader`] is itself `BinStream::mount` plus [`BinStream::into_bin`], so the eager tree and the streaming surface are one parser and cannot drift. +### Walking a bin + +[`walk`] is one traversal for both trees. A [`Visitor`](walk::Visitor) is called once per +node, in pre-order and file order, and answers a [`Visit`](walk::Visit) that continues, prunes +a property, stops or aborts. The same visitor runs over an owned [`Bin`] and over a +[`BinStream`], where nothing is materialised. + +``` +use ltk_hash::BinHash; +use ltk_meta::{ + walk::{Node, TreeValue, Visit, Visitor}, + Error, +}; + +/// Counts nodes and records the address of every `Struct` of one class. +#[derive(Default)] +struct Census { + nodes: usize, + hits: Vec<(BinHash, String)>, +} + +impl<'a, V: TreeValue<'a>> Visitor<'a, V> for Census { + type Error = Error; + + fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + self.nodes += 1; + if *node.class_hash() == 0x1e6b_a0c4 { + self.hits.push((node.object_hash(), node.trail().to_string())); + } + Ok(Visit::Continue) + } +} + +# let bin = ltk_meta::concrete::Bin::builder().build(); +let mut census = Census::default(); +bin.walk(&mut census)?; +# Ok::<(), Error>(()) +``` + +The walk over one object is sequential by contract. Objects are independent, and every view, +node and trail type is `Send`. A sweep parallelises across objects: one visitor instance per +worker, reduced at the end. The split is the caller's; the crate schedules nothing. + +``` +# use ltk_hash::BinHash; +# use ltk_meta::{walk::{Node, TreeValue, Visit, Visitor}, Error}; +# #[derive(Default)] +# struct Census { nodes: usize, hits: Vec<(BinHash, String)> } +# impl<'a, V: TreeValue<'a>> Visitor<'a, V> for Census { +# type Error = Error; +# fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { +# self.nodes += 1; +# Ok(Visit::Continue) +# } +# } +# let bin = ltk_meta::concrete::Bin::builder().build(); +let objects: Vec<_> = bin.objects.values().collect(); +let workers = std::thread::available_parallelism().map_or(1, |n| n.get()); +let per_worker = objects.len().div_ceil(workers).max(1); + +let counted = std::thread::scope(|scope| { + let workers: Vec<_> = objects + .chunks(per_worker) + .map(|chunk| { + scope.spawn(move || { + let mut census = Census::default(); + for object in chunk { + object.walk(&mut census)?; + } + Ok::<_, Error>(census) + }) + }) + .collect(); + + let mut all = Census::default(); + for worker in workers { + let census = worker.join().expect("a worker panicked")?; + all.nodes += census.nodes; + all.hits.extend(census.hits); + } + Ok::<_, Error>(all) +})?; + +println!("{} nodes, {} hits", counted.nodes, counted.hits.len()); +# Ok::<(), Error>(()) +``` + +Across many files the same shape applies one level up: one task per file, each mounting its +own [`BinStream`] and walking it sequentially. The per-object walk is microseconds; +decompression and I/O are where a sweep spends its time. + ### Modifying a bin file ```no_run @@ -234,3 +325,5 @@ mod error; pub use error::*; pub mod traits; + +pub mod walk; diff --git a/crates/ltk_meta/src/stream/view.rs b/crates/ltk_meta/src/stream/view.rs index ebddb286..ece75d04 100644 --- a/crates/ltk_meta/src/stream/view.rs +++ b/crates/ltk_meta/src/stream/view.rs @@ -141,6 +141,11 @@ impl<'a, M> ObjectView<'a, M> { find_property(self.properties(), name_hash.into()) } + /// The object as a struct: its class hash and its properties, the path hash left behind. + pub(crate) fn as_struct(&self) -> StructView<'a, M> { + StructView::from_parts(self.class_hash, self.property_count, self.properties) + } + /// The object's raw bytes — its whole declared range, size field included. /// /// This is the range a byte-exact copy of the object covers, which is what the delta diff --git a/crates/ltk_meta/src/stream/view/value.rs b/crates/ltk_meta/src/stream/view/value.rs index 1da4055f..730fb2a3 100644 --- a/crates/ltk_meta/src/stream/view/value.rs +++ b/crates/ltk_meta/src/stream/view/value.rs @@ -616,6 +616,17 @@ impl<'a, M> StructView<'a, M> { }) } + /// A view over `count` properties at `properties`, carrying `class_hash`. An object's + /// root as a struct. + pub(crate) fn from_parts(class_hash: BinHash, count: u16, properties: Cursor<'a>) -> Self { + Self { + class_hash, + property_count: count, + properties, + meta: PhantomData, + } + } + /// The class this is an instance of, or `0` for a null pointer. #[must_use] pub fn class_hash(&self) -> BinHash { diff --git a/crates/ltk_meta/src/walk.rs b/crates/ltk_meta/src/walk.rs new file mode 100644 index 00000000..0f79b774 --- /dev/null +++ b/crates/ltk_meta/src/walk.rs @@ -0,0 +1,670 @@ +//! One read-only traversal over every node of a bin object, driven by a [`Visitor`]. +//! +//! The walk is written once, against two sealed traits - [`TreeValue`] and [`TreeNode`] - that +//! the owned tree (`&PropertyValueEnum`) and the streaming view ([`ValueView`]) both +//! implement. A visitor is generic over the value type and runs over either unchanged: +//! [`BinObject::walk`] and [`Bin::walk`] over the owned tree, [`ObjectView::walk`] and +//! [`BinStream::walk`] over a buffered object's bytes. +//! +//! The visitor sees nodes in pre-order, in file order, each exactly once. A node is an object, +//! or a `Struct` or `Embedded` value whose class hash is not 0. It is entered and exited, and +//! so is every property of it that can hold a node. Every callback answers a [`Visit`]; the +//! walk returns a [`WalkOutcome`], or the visitor's own error. +//! +//! The walk carries a [`Trail`]: the steps from the object's root to the current position, +//! borrowing the tree and allocating nothing per step. A visitor renders it for a node it +//! reports on and for nothing else. +//! +//! ``` +//! use ltk_hash::BinHash; +//! use ltk_meta::{ +//! concrete::{values, Bin, BinObject}, +//! walk::{Node, TreeValue, Visit, Visitor}, +//! Error, +//! }; +//! +//! /// Every node's address, in pre-order. +//! #[derive(Default)] +//! struct Addresses(Vec); +//! +//! impl<'a, V: TreeValue<'a>> Visitor<'a, V> for Addresses { +//! type Error = Error; +//! +//! fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { +//! self.0.push(format!("{:08x} {}", node.object_hash(), node.trail())); +//! Ok(Visit::Continue) +//! } +//! } +//! +//! let inner = values::Struct { +//! class_hash: 0xC1A5_0002u32.into(), +//! properties: Default::default(), +//! meta: Default::default(), +//! }; +//! let bin = Bin::builder() +//! .object( +//! BinObject::builder(0x0100_0001u32, 0xC1A5_0001u32) +//! .property(0x0000_0001u32, inner) +//! .build(), +//! ) +//! .build(); +//! +//! let mut addresses = Addresses::default(); +//! bin.walk(&mut addresses)?; +//! assert_eq!(addresses.0, ["01000001 ", "01000001 00000001"]); +//! # Ok::<(), Error>(()) +//! ``` + +mod owned; +mod tree; +mod view; + +#[cfg(test)] +mod tests; + +pub use owned::{OwnedChildren, OwnedNode, OwnedProperties}; +pub use tree::{Child, Leaf, TreeKind, TreeNode, TreeValue}; +pub use view::{ViewChildren, ViewProperties}; + +use std::{ + fmt, io, + ops::ControlFlow::{self, Break, Continue}, +}; + +use ltk_hash::BinHash; + +use crate::{ + stream::{BinStream, ObjectStream, ObjectView, ValueView}, + Bin, BinObject, BinOverride, Error, PropertyValueEnum, +}; + +/// What a callback answers. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum Visit { + /// Ends the walk immediately. No exit callback runs for anything open. + Abort, + /// Ends the walk after unwinding: every open property and node gets its exit, + /// innermost first. The walk does not resume. + Stop, + /// Skips ahead, locally: + /// - from [`Visitor::enter_node`]: the node's properties are not walked; its + /// [`Visitor::exit_node`] runs regardless. + /// - from [`Visitor::enter_property`]: the value is not descended - the prune. + /// [`Visitor::exit_property`] runs regardless for a value that holds a node. + /// - from [`Visitor::exit_property`]: the node's remaining properties are pruned; the walk + /// jumps to the node's [`Visitor::exit_node`]. + /// - from [`Visitor::exit_node`]: the parent property's remaining items are pruned; the + /// walk jumps to the parent's [`Visitor::exit_property`]. + Skip, + /// Carries on. + Continue, +} + +/// How a walk ended. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum WalkOutcome { + /// Every object was walked to the end. + Completed, + /// A callback answered [`Visit::Stop`] and the walk unwound. + Stopped, + /// A callback answered [`Visit::Abort`]. + Aborted, +} + +/// What a walk calls. +/// +/// Generic over the tree's value type: one visitor runs over the owned tree +/// (`V = &PropertyValueEnum`) and over the view (`V = ValueView<'a, M>`) alike. +/// +/// Every callback has a default that continues. A visitor implements only what it reads. +#[expect( + unused_variables, + reason = "the defaults name their parameters for the reader and use none of them" +)] +pub trait Visitor<'a, V: TreeValue<'a>> { + /// The visitor's own error. The tree's errors convert into it: a `?` on a tree call + /// inside a callback needs nothing more than `From`. + type Error: From; + + /// Called at every node the walk reaches, before any of its properties. + /// + /// # Errors + /// + /// The visitor's own. An error ends the walk at once, as an [`Visit::Abort`] does. + fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + Ok(Visit::Continue) + } + + /// Called once for every node entered: after its properties, after a [`Visit::Skip`], + /// and while unwinding for a [`Visit::Stop`]. Never after an [`Visit::Abort`]. + /// + /// # Errors + /// + /// The visitor's own. An error ends the walk at once, as an [`Visit::Abort`] does. + fn exit_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + Ok(Visit::Continue) + } + + /// Called for every property of a node, in file order, leaves included. The value is the + /// tree's, undecoded until read. Only a value that [`TreeValue::holds_node`] is descended + /// on [`Visit::Continue`]; a leaf is a call and nothing more. + /// + /// # Errors + /// + /// The visitor's own. An error ends the walk at once, as an [`Visit::Abort`] does. + fn enter_property( + &mut self, + field: BinHash, + value: V, + node: &Node<'_, 'a, V>, + ) -> Result { + Ok(Visit::Continue) + } + + /// Called once for every property that holds a node and was entered: after its nodes, + /// after a [`Visit::Skip`], and while unwinding for a [`Visit::Stop`]. Not called for a + /// leaf. Never after an [`Visit::Abort`]. + /// + /// # Errors + /// + /// The visitor's own. An error ends the walk at once, as an [`Visit::Abort`] does. + fn exit_property( + &mut self, + field: BinHash, + value: V, + node: &Node<'_, 'a, V>, + ) -> Result { + Ok(Visit::Continue) + } +} + +/// A `&mut W` is a visitor. A `&mut dyn Visitor<'a, V, Error = E>` passes where one is wanted. +impl<'a, V: TreeValue<'a>, W: Visitor<'a, V> + ?Sized> Visitor<'a, V> for &mut W { + type Error = W::Error; + + fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + (**self).enter_node(node) + } + + fn exit_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + (**self).exit_node(node) + } + + fn enter_property( + &mut self, + field: BinHash, + value: V, + node: &Node<'_, 'a, V>, + ) -> Result { + (**self).enter_property(field, value, node) + } + + fn exit_property( + &mut self, + field: BinHash, + value: V, + node: &Node<'_, 'a, V>, + ) -> Result { + (**self).exit_property(field, value, node) + } +} + +/// One node, as the walk hands it to a visitor. +pub struct Node<'t, 'a, V: TreeValue<'a>> { + object_hash: BinHash, + inner: V::Node, + trail: &'t Trail, +} + +impl<'t, 'a, V: TreeValue<'a>> Node<'t, 'a, V> { + fn new(object_hash: BinHash, inner: V::Node, trail: &'t Trail) -> Self { + Self { + object_hash, + inner, + trail, + } + } + + /// The path hash of the object this node is in, or is. + #[must_use] + pub fn object_hash(&self) -> BinHash { + self.object_hash + } + + /// The class hash this node carries. Never 0. + #[must_use] + pub fn class_hash(&self) -> BinHash { + self.inner.class_hash() + } + + /// The node itself: its properties in file order, lookup by field, and + /// [`TreeNode::to_struct`]. + #[must_use] + pub fn inner(&self) -> V::Node { + self.inner + } + + /// Where the node is: empty at the root. + #[must_use] + pub fn trail(&self) -> &'t Trail { + self.trail + } + + /// Whether this node is the object itself. + #[must_use] + pub fn is_root(&self) -> bool { + self.trail.is_empty() + } +} + +impl<'a, V: TreeValue<'a>> Clone for Node<'_, 'a, V> { + fn clone(&self) -> Self { + *self + } +} +impl<'a, V: TreeValue<'a>> Copy for Node<'_, 'a, V> {} + +impl<'a, V: TreeValue<'a>> fmt::Debug for Node<'_, 'a, V> { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("Node") + .field("object_hash", &self.object_hash) + .field("class_hash", &self.class_hash()) + .field("trail", &format_args!("{}", self.trail)) + .finish() + } +} + +/// One step of a [`Trail`]: a field, an index or a map entry. A key is the tree's value. +#[derive(Clone, Copy, Debug, PartialEq)] +pub enum TrailStep { + /// A property of a node, by the field's name hash. + Field(BinHash), + /// A container element by position, or the value of a present optional, which is always 0. + Index(usize), + /// A map entry, by its key. + Key(V), +} + +/// The steps from an object's root to the walk's position. +/// +/// Borrows the tree - a map key is the tree's own value, never a copy. Descending a map of ten +/// thousand entries allocates nothing. Text is made only by `Display`. +/// +/// Beside the steps the trail keeps the **class context**: for each `Field` step, the class +/// hash of the node the field was read on. It is what a name table is asked with. +#[derive(Debug)] +pub struct Trail { + steps: Vec>, + classes: Vec, +} + +impl Trail { + fn new() -> Self { + Self { + steps: Vec::new(), + classes: Vec::new(), + } + } + + /// The steps, root first. + #[must_use] + pub fn steps(&self) -> &[TrailStep] { + &self.steps + } + + /// How many steps the trail holds. + #[must_use] + pub fn len(&self) -> usize { + self.steps.len() + } + + /// Whether the trail is at the root. + #[must_use] + pub fn is_empty(&self) -> bool { + self.steps.is_empty() + } + + /// The class of the node each field step was read on, one per `Field` step, in order. + /// Never 0: the walk always knows. + #[must_use] + pub fn classes(&self) -> &[BinHash] { + &self.classes + } + + fn push_field(&mut self, field: BinHash, class: BinHash) { + self.steps.push(TrailStep::Field(field)); + self.classes.push(class); + } + + fn push(&mut self, step: TrailStep) { + self.steps.push(step); + } + + fn pop(&mut self) { + if let Some(TrailStep::Field(_)) = self.steps.pop() { + self.classes.pop(); + } + } + + fn clear(&mut self) { + self.steps.clear(); + self.classes.clear(); + } +} + +/// The hash form: `.` between fields, `[i]` for an index, `{key}` for a map entry, every +/// field hash as eight lowercase hex digits. A key that does not decode renders as `{?}`. +impl<'a, V: TreeValue<'a>> fmt::Display for Trail { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + for (i, step) in self.steps.iter().enumerate() { + match step { + TrailStep::Field(field) => { + if i > 0 { + f.write_str(".")?; + } + write!(f, "{field:08x}")?; + } + TrailStep::Index(index) => write!(f, "[{index}]")?, + TrailStep::Key(key) => { + f.write_str("{")?; + match key.leaf() { + Ok(Some(leaf)) => leaf.write_key(f)?, + Ok(None) | Err(_) => f.write_str("?")?, + } + f.write_str("}")?; + } + } + } + Ok(()) + } +} + +/// Walk teardown, propagated up the recursion. +enum Interrupt { + /// A [`Visit::Stop`]: every open exit runs, innermost first. + Unwind, + /// A [`Visit::Abort`], or an error: no further callback runs. + Abort, +} + +/// Where the walk resumes after a node's exit. +enum Resume { + /// With the enclosing property's remaining items. + Siblings, + /// At the enclosing property's exit: the node's exit answered [`Visit::Skip`]. + Parent, +} + +/// One object's walk: the object hash, and the trail below its root. +struct Walker { + object_hash: BinHash, + trail: Trail, +} + +impl<'a, V: TreeValue<'a>> Walker { + fn new() -> Self { + Self { + // A placeholder: `walk_object` sets the hash before any callback reads it. + object_hash: BinHash(0), + trail: Trail::new(), + } + } + + /// Walks one object, `root` under `object_hash`, and reports how it ended. `Completed` + /// leaves the trail empty and the walker ready for the next object. + fn walk_object>( + &mut self, + object_hash: BinHash, + root: V::Node, + visitor: &mut W, + ) -> Result { + self.object_hash = object_hash; + self.trail.clear(); + Ok(match self.walk_node(root, visitor)? { + Continue(_) => WalkOutcome::Completed, + Break(Interrupt::Unwind) => WalkOutcome::Stopped, + Break(Interrupt::Abort) => WalkOutcome::Aborted, + }) + } + + fn node(&self, inner: V::Node) -> Node<'_, 'a, V> { + Node::new(self.object_hash, inner, &self.trail) + } + + fn walk_node>( + &mut self, + node: V::Node, + visitor: &mut W, + ) -> Result, W::Error> { + let walked = match visitor.enter_node(&self.node(node))? { + Visit::Abort => return Ok(Break(Interrupt::Abort)), + Visit::Stop => Break(Interrupt::Unwind), + Visit::Skip => Continue(()), + Visit::Continue => self.walk_properties(node, visitor)?, + }; + if let Break(Interrupt::Abort) = walked { + return Ok(Break(Interrupt::Abort)); + } + + Ok(match (walked, visitor.exit_node(&self.node(node))?) { + (_, Visit::Abort) => Break(Interrupt::Abort), + (Break(Interrupt::Unwind), _) | (_, Visit::Stop) => Break(Interrupt::Unwind), + (_, Visit::Skip) => Continue(Resume::Parent), + (_, Visit::Continue) => Continue(Resume::Siblings), + }) + } + + fn walk_properties>( + &mut self, + node: V::Node, + visitor: &mut W, + ) -> Result, W::Error> { + for property in node.properties() { + let (field, value) = property?; + let visit = visitor.enter_property(field, value, &self.node(node))?; + if !value.holds_node()? { + match visit { + Visit::Abort => return Ok(Break(Interrupt::Abort)), + Visit::Stop => return Ok(Break(Interrupt::Unwind)), + Visit::Skip | Visit::Continue => continue, + } + } + + let walked = match visit { + Visit::Abort => return Ok(Break(Interrupt::Abort)), + Visit::Stop => Break(Interrupt::Unwind), + Visit::Skip => Continue(()), + Visit::Continue => { + self.trail.push_field(field, node.class_hash()); + let walked = self.descend(value, visitor); + self.trail.pop(); + walked? + } + }; + if let Break(Interrupt::Abort) = walked { + return Ok(Break(Interrupt::Abort)); + } + + match ( + walked, + visitor.exit_property(field, value, &self.node(node))?, + ) { + (_, Visit::Abort) => return Ok(Break(Interrupt::Abort)), + (Break(Interrupt::Unwind), _) | (_, Visit::Stop) => { + return Ok(Break(Interrupt::Unwind)) + } + (_, Visit::Skip) => break, + (_, Visit::Continue) => {} + } + } + Ok(Continue(())) + } + + /// Descends a value that holds a node: the node itself, or every item of a container, + /// optional or map. + fn descend>( + &mut self, + value: V, + visitor: &mut W, + ) -> Result, W::Error> { + if let Some(node) = value.as_node()? { + return Ok(match self.walk_node(node, visitor)? { + Break(interrupt) => Break(interrupt), + Continue(_) => Continue(()), + }); + } + + for child in value.children()? { + let (step, item) = child?; + let Some(node) = item.as_node()? else { + continue; + }; + self.trail.push(match step { + Child::Index(index) => TrailStep::Index(index), + Child::Key(key) => TrailStep::Key(key), + }); + let walked = self.walk_node(node, visitor); + self.trail.pop(); + match walked? { + Break(interrupt) => return Ok(Break(interrupt)), + Continue(Resume::Parent) => break, + Continue(Resume::Siblings) => {} + } + } + Ok(Continue(())) + } +} + +/// Walks `objects` in order through one walker. A `Stop` or `Abort` ends the whole walk. +fn walk_all<'a, V, W, N>( + objects: impl IntoIterator, + visitor: &mut W, +) -> Result +where + V: TreeValue<'a, Node = N>, + N: TreeNode<'a, Value = V>, + W: Visitor<'a, V>, +{ + let mut walker = Walker::new(); + for (object_hash, root) in objects { + match walker.walk_object(object_hash, root, visitor)? { + WalkOutcome::Completed => {} + ended => return Ok(ended), + } + } + Ok(WalkOutcome::Completed) +} + +impl BinObject { + /// Walks this object: the root, then every node beneath every property `visitor` enters. + /// + /// # Errors + /// + /// Whatever the visitor raises. The owned tree never fails on its own. + pub fn walk<'a, W>(&'a self, visitor: &mut W) -> Result + where + W: Visitor<'a, &'a PropertyValueEnum>, + { + Walker::new().walk_object(self.path_hash, OwnedNode::from(self), visitor) + } +} + +impl Bin { + /// Walks every object, in file order. A `Stop` or `Abort` ends the whole walk, not the + /// current object. + /// + /// # Errors + /// + /// Whatever the visitor raises. The owned tree never fails on its own. + pub fn walk<'a, W>(&'a self, visitor: &mut W) -> Result + where + W: Visitor<'a, &'a PropertyValueEnum>, + { + walk_all( + self.objects + .values() + .map(|object| (object.path_hash, OwnedNode::from(object))), + visitor, + ) + } +} + +impl BinOverride { + /// Walks every embedded object, in file order, as the file holds them. A record that + /// targets one of them has not been applied. Patch records are not walked: a record's + /// value has no node of its own to stand on. + /// + /// # Errors + /// + /// Whatever the visitor raises. The owned tree never fails on its own. + pub fn walk<'a, W>(&'a self, visitor: &mut W) -> Result + where + W: Visitor<'a, &'a PropertyValueEnum>, + { + walk_all( + self.objects + .values() + .map(|object| (object.path_hash, OwnedNode::from(object))), + visitor, + ) + } +} + +impl<'a, M: Default> ObjectView<'a, M> { + /// Walks this object over its buffered bytes: nothing is materialised, a header is + /// decoded where the walk descends, and a leaf is decoded only when the visitor asks. + /// + /// # Errors + /// + /// A kind byte or header that does not decode, converted into the visitor's error, or + /// whatever the visitor raises. + pub fn walk(&self, visitor: &mut W) -> Result + where + W: Visitor<'a, ValueView<'a, M>>, + { + Walker::new().walk_object(self.path_hash(), self.as_struct(), visitor) + } +} + +impl ObjectStream<'_, R, M> { + /// [`ObjectStream::view`] then [`ObjectView::walk`]. + /// + /// `E` is the visitor's error, named once here: a visitor that runs over every object + /// buffer is bound for every lifetime, and its error type is the one thing the bound + /// holds fixed. + /// + /// # Errors + /// + /// The same as [`ObjectStream::view`] and [`ObjectView::walk`], in the visitor's error. + pub fn walk(&mut self, visitor: &mut W) -> Result + where + E: From, + W: for<'a> Visitor<'a, ValueView<'a, M>, Error = E>, + { + self.view()?.walk(visitor) + } +} + +impl BinStream { + /// Walks every object in file order, one buffered object at a time: [`BinStream::objects`] + /// and [`ObjectStream::walk`] on each. Holds one object's bytes at any moment and nothing + /// of the tree. + /// + /// # Errors + /// + /// The same as [`ObjectStream::walk`], for the object the walk was in. + pub fn walk(&mut self, visitor: &mut W) -> Result + where + E: From, + W: for<'a> Visitor<'a, ValueView<'a, M>, Error = E>, + { + let mut objects = self.objects(); + while let Some(mut object) = objects.next()? { + match object.walk(visitor)? { + WalkOutcome::Completed => {} + ended => return Ok(ended), + } + } + Ok(WalkOutcome::Completed) + } +} diff --git a/crates/ltk_meta/src/walk/owned.rs b/crates/ltk_meta/src/walk/owned.rs new file mode 100644 index 00000000..f133729b --- /dev/null +++ b/crates/ltk_meta/src/walk/owned.rs @@ -0,0 +1,318 @@ +//! The owned tree as the walk sees it: `&PropertyValueEnum` and [`OwnedNode`]. + +use std::{fmt, iter::Enumerate, slice}; + +use indexmap::IndexMap; +use ltk_hash::BinHash; + +use super::{ + tree::{sealed::Sealed, Child, Leaf, TreeKind as _, TreeNode, TreeValue}, + Error, +}; +use crate::{ + property::Kind, + property::{values, NoMeta}, + BinObject, PropertyValueEnum, +}; + +/// The owned tree's node: a class hash and a borrowed property map. +/// +/// [`BinObject`] and [`values::Struct`] both view as one, through `From`. +pub struct OwnedNode<'a, M = NoMeta> { + class_hash: BinHash, + properties: &'a IndexMap>, +} + +impl<'a, M> OwnedNode<'a, M> { + /// A node over `properties`, carrying `class_hash`. + #[must_use] + pub fn new( + class_hash: BinHash, + properties: &'a IndexMap>, + ) -> Self { + Self { + class_hash, + properties, + } + } +} + +impl<'a, M> From<&'a BinObject> for OwnedNode<'a, M> { + fn from(object: &'a BinObject) -> Self { + Self::new(object.class_hash, &object.properties) + } +} + +impl<'a, M> From<&'a values::Struct> for OwnedNode<'a, M> { + fn from(value: &'a values::Struct) -> Self { + Self::new(value.class_hash, &value.properties) + } +} + +// By hand rather than derived: a derived `Copy` would demand `M: Copy` for a borrow. +impl Clone for OwnedNode<'_, M> { + fn clone(&self) -> Self { + *self + } +} +impl Copy for OwnedNode<'_, M> {} + +impl fmt::Debug for OwnedNode<'_, M> { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("OwnedNode") + .field("class_hash", &self.class_hash) + .field("property_count", &self.properties.len()) + .finish() + } +} + +impl Sealed for OwnedNode<'_, M> {} +impl Sealed for &PropertyValueEnum {} + +/// The properties of an [`OwnedNode`], in order. +#[must_use = "iterators are lazy and do nothing unless consumed"] +#[derive(Debug)] +pub struct OwnedProperties<'a, M = NoMeta> { + inner: indexmap::map::Iter<'a, BinHash, PropertyValueEnum>, +} + +impl<'a, M> Iterator for OwnedProperties<'a, M> { + type Item = Result<(BinHash, &'a PropertyValueEnum), Error>; + + fn next(&mut self) -> Option { + self.inner.next().map(|(field, value)| Ok((*field, value))) + } + + fn size_hint(&self) -> (usize, Option) { + self.inner.size_hint() + } +} + +impl ExactSizeIterator for OwnedProperties<'_, M> {} +impl std::iter::FusedIterator for OwnedProperties<'_, M> {} + +impl<'a, M> TreeNode<'a> for OwnedNode<'a, M> { + type Value = &'a PropertyValueEnum; + type Properties = OwnedProperties<'a, M>; + + fn class_hash(&self) -> BinHash { + self.class_hash + } + + fn properties(&self) -> Self::Properties { + OwnedProperties { + inner: self.properties.iter(), + } + } + + fn property(&self, field: BinHash) -> Result, Error> { + Ok(self.properties.get(&field)) + } + + fn to_struct(&self) -> Result { + Ok(values::Struct { + class_hash: self.class_hash, + properties: self + .properties + .iter() + .map(|(field, value)| Ok((*field, strip_meta(value)?))) + .collect::>()?, + meta: NoMeta, + }) + } +} + +/// The values inside an owned container, optional or map. +#[must_use = "iterators are lazy and do nothing unless consumed"] +#[derive(Debug)] +pub struct OwnedChildren<'a, M = NoMeta> { + inner: OwnedChildrenInner<'a, M>, +} + +#[derive(Debug)] +enum OwnedChildrenInner<'a, M> { + Items(Enumerate>>), + Entries(slice::Iter<'a, (PropertyValueEnum, PropertyValueEnum)>), +} + +impl<'a, M> Iterator for OwnedChildren<'a, M> { + type Item = Result<(Child<&'a PropertyValueEnum>, &'a PropertyValueEnum), Error>; + + fn next(&mut self) -> Option { + match &mut self.inner { + OwnedChildrenInner::Items(items) => items + .next() + .map(|(index, value)| Ok((Child::Index(index), value))), + OwnedChildrenInner::Entries(entries) => entries + .next() + .map(|(key, value)| Ok((Child::Key(key), value))), + } + } + + fn size_hint(&self) -> (usize, Option) { + match &self.inner { + OwnedChildrenInner::Items(items) => items.size_hint(), + OwnedChildrenInner::Entries(entries) => entries.size_hint(), + } + } +} + +impl ExactSizeIterator for OwnedChildren<'_, M> {} +impl std::iter::FusedIterator for OwnedChildren<'_, M> {} + +impl<'a, M> TreeValue<'a> for &'a PropertyValueEnum { + type Node = OwnedNode<'a, M>; + type Children = OwnedChildren<'a, M>; + + fn kind(&self) -> Kind { + PropertyValueEnum::kind(self) + } + + fn holds_node(&self) -> Result { + Ok(match self { + PropertyValueEnum::Struct(s) => *s.class_hash != 0, + PropertyValueEnum::Embedded(e) => *e.0.class_hash != 0, + PropertyValueEnum::Container(c) => c.item_kind().is_node(), + PropertyValueEnum::UnorderedContainer(c) => c.0.item_kind().is_node(), + PropertyValueEnum::Optional(o) => o.item_kind().is_node(), + PropertyValueEnum::Map(m) => m.value_kind().is_node(), + _ => false, + }) + } + + fn as_node(&self) -> Result, Error> { + let node = match self { + PropertyValueEnum::Struct(s) => s, + PropertyValueEnum::Embedded(e) => &e.0, + _ => return Ok(None), + }; + Ok((*node.class_hash != 0).then(|| OwnedNode::from(node))) + } + + fn children(&self) -> Result { + let inner = match self { + PropertyValueEnum::Container(c) => { + OwnedChildrenInner::Items(c.items().iter().enumerate()) + } + PropertyValueEnum::UnorderedContainer(c) => { + OwnedChildrenInner::Items(c.0.items().iter().enumerate()) + } + PropertyValueEnum::Optional(o) => OwnedChildrenInner::Items( + o.value() + .map_or(&[][..], slice::from_ref) + .iter() + .enumerate(), + ), + PropertyValueEnum::Map(m) => OwnedChildrenInner::Entries(m.entries().iter()), + _ => OwnedChildrenInner::Items([].iter().enumerate()), + }; + Ok(OwnedChildren { inner }) + } + + fn leaf(&self) -> Result>, Error> { + use PropertyValueEnum as P; + Ok(Some(match *self { + P::None(_) => Leaf::None, + P::Bool(v) => Leaf::Bool(v.value), + P::I8(v) => Leaf::I8(v.value), + P::U8(v) => Leaf::U8(v.value), + P::I16(v) => Leaf::I16(v.value), + P::U16(v) => Leaf::U16(v.value), + P::I32(v) => Leaf::I32(v.value), + P::U32(v) => Leaf::U32(v.value), + P::I64(v) => Leaf::I64(v.value), + P::U64(v) => Leaf::U64(v.value), + P::F32(v) => Leaf::F32(v.value), + P::Vector2(v) => Leaf::Vector2(v.value), + P::Vector3(v) => Leaf::Vector3(v.value), + P::Vector4(v) => Leaf::Vector4(v.value), + P::Matrix44(v) => Leaf::Matrix44(v.value), + P::Color(v) => Leaf::Color(v.value), + P::String(v) => Leaf::String(&v.value), + P::Hash(v) => Leaf::Hash(v.value), + P::WadChunkLink(v) => Leaf::File(v.value), + P::ObjectLink(v) => Leaf::Link(v.value), + P::BitBool(v) => Leaf::Flag(v.value), + P::Container(_) + | P::UnorderedContainer(_) + | P::Optional(_) + | P::Map(_) + | P::Struct(_) + | P::Embedded(_) => return Ok(None), + })) + } + + fn to_value(&self) -> Result { + strip_meta(self) + } +} + +/// A copy of `value` with every metadata slot reset to [`NoMeta`]. +/// +/// # Errors +/// +/// Never: every container held by `value` satisfies the checks its constructor performs. +pub(crate) fn strip_meta(value: &PropertyValueEnum) -> Result { + use PropertyValueEnum as P; + macro_rules! prim { + ($ty:ident, $v:expr) => { + P::$ty(values::$ty::new_with_meta($v.value.clone(), NoMeta)) + }; + } + Ok(match value { + P::None(_) => P::None(values::None { meta: NoMeta }), + P::Bool(v) => prim!(Bool, v), + P::I8(v) => prim!(I8, v), + P::U8(v) => prim!(U8, v), + P::I16(v) => prim!(I16, v), + P::U16(v) => prim!(U16, v), + P::I32(v) => prim!(I32, v), + P::U32(v) => prim!(U32, v), + P::I64(v) => prim!(I64, v), + P::U64(v) => prim!(U64, v), + P::F32(v) => prim!(F32, v), + P::Vector2(v) => prim!(Vector2, v), + P::Vector3(v) => prim!(Vector3, v), + P::Vector4(v) => prim!(Vector4, v), + P::Matrix44(v) => prim!(Matrix44, v), + P::Color(v) => prim!(Color, v), + P::String(v) => prim!(String, v), + P::Hash(v) => prim!(Hash, v), + P::WadChunkLink(v) => prim!(WadChunkLink, v), + P::ObjectLink(v) => prim!(ObjectLink, v), + P::BitBool(v) => prim!(BitBool, v), + P::Struct(s) => P::Struct(strip_struct(s)?), + P::Embedded(e) => P::Embedded(values::Embedded(strip_struct(&e.0)?)), + P::Container(c) => P::Container(strip_container(c)?), + P::UnorderedContainer(c) => { + P::UnorderedContainer(values::UnorderedContainer(strip_container(&c.0)?)) + } + P::Optional(o) => P::Optional(values::Optional::new( + o.item_kind(), + o.value().map(strip_meta).transpose()?, + )?), + P::Map(m) => P::Map(values::Map::new( + m.key_kind(), + m.value_kind(), + m.entries() + .iter() + .map(|(k, v)| Ok((strip_meta(k)?, strip_meta(v)?))) + .collect::>()?, + )?), + }) +} + +fn strip_struct(value: &values::Struct) -> Result { + OwnedNode::from(value).to_struct() +} + +fn strip_container(value: &values::Container) -> Result { + values::Container::new( + value.item_kind(), + value + .items() + .iter() + .map(strip_meta) + .collect::>()?, + ) +} diff --git a/crates/ltk_meta/src/walk/tests.rs b/crates/ltk_meta/src/walk/tests.rs new file mode 100644 index 00000000..5f34789c --- /dev/null +++ b/crates/ltk_meta/src/walk/tests.rs @@ -0,0 +1,1143 @@ +//! The fixture of `value-walk.md` section 7, walked twice: over the owned tree and over an +//! `ObjectView` of the same bytes, through one generic visitor. + +use std::io; + +use glam::{Mat4, Vec2, Vec3, Vec4}; +use ltk_hash::BinHash; +use ltk_primitives::Color; + +use super::{Leaf, Node, TreeNode, TreeValue, Visit, Visitor, WalkOutcome}; +use crate::{ + concrete::{self, values, Bin, BinObject}, + property::values::{Embedded, UnorderedContainer}, + property::Kind, + stream::ValueView, + BinOverride, Error, PropertyValueEnum, +}; + +type BinStream = concrete::BinStream>>; + +const OBJECT: u32 = 0x0100_0001; +const C1: u32 = 0xC1A5_0001; +const C2: u32 = 0xC1A5_0002; +const C3: u32 = 0xC1A5_0003; +const C4: u32 = 0xC1A5_0004; +const C5: u32 = 0xC1A5_0005; +const C6: u32 = 0xC1A5_0006; +const C7: u32 = 0xC1A5_0007; +const C8: u32 = 0xC1A5_0008; +const C9: u32 = 0xC1A5_0009; + +const F_STRUCT: u32 = 0x01; +const F_EMBED: u32 = 0x02; +const F_NULL_STRUCT: u32 = 0x03; +const F_NULL_EMBED: u32 = 0x04; +const F_CONT_STRUCT: u32 = 0x05; +const F_CONT_EMBED: u32 = 0x06; +const F_OPT_STRUCT: u32 = 0x07; +const F_OPT_NULL: u32 = 0x08; +const F_OPT_EMPTY: u32 = 0x09; +const F_MAP_STRUCT: u32 = 0x0A; +const F_MAP_EMBED: u32 = 0x0B; +const F_STRINGS: u32 = 0x0C; +const F_LEAF: u32 = 0x10; +const F_INNER: u32 = 0x11; +/// Leaf properties of the root: `F_LEAVES + kind as u32`. +const F_LEAVES: u32 = 0x100; +/// Maps of the root keyed by each valid key kind, holding `I32`: `F_KEYS + kind as u32`. +const F_KEYS: u32 = 0x200; + +const KEY_A: u32 = 0x0000_00AA; +const KEY_B: u32 = 0x0000_00BB; + +fn node(class: u32, properties: Vec<(u32, PropertyValueEnum)>) -> values::Struct { + values::Struct { + class_hash: class.into(), + properties: properties + .into_iter() + .map(|(field, value)| (BinHash(field), value)) + .collect(), + meta: Default::default(), + } +} + +fn null() -> values::Struct { + values::Struct::default() +} + +fn leaf_of(kind: Kind) -> PropertyValueEnum { + use Kind as K; + match kind { + K::None => values::None::default().into(), + K::Bool => values::Bool::new(true).into(), + K::I8 => values::I8::new(-8).into(), + K::U8 => values::U8::new(8).into(), + K::I16 => values::I16::new(-16).into(), + K::U16 => values::U16::new(16).into(), + K::I32 => values::I32::new(-32).into(), + K::U32 => values::U32::new(32).into(), + K::I64 => values::I64::new(-64).into(), + K::U64 => values::U64::new(64).into(), + K::F32 => values::F32::new(1.5).into(), + K::Vector2 => values::Vector2::new(Vec2::new(1.0, 2.0)).into(), + K::Vector3 => values::Vector3::new(Vec3::new(1.0, 2.0, 3.0)).into(), + K::Vector4 => values::Vector4::new(Vec4::new(1.0, 2.0, 3.0, 4.0)).into(), + K::Matrix44 => values::Matrix44::new(Mat4::from_cols_array(&[ + 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, + ])) + .into(), + K::Color => values::Color::new(Color { + r: 1u8, + g: 2, + b: 3, + a: 4, + }) + .into(), + K::String => values::String::from("weapon").into(), + K::Hash => values::Hash::new(0x1e6b_a0c4u32).into(), + K::WadChunkLink => values::WadChunkLink::new(0x00c9_fd8f_1a2b_3c4du64).into(), + K::ObjectLink => values::ObjectLink::new(0x0BEE_F000u32).into(), + K::BitBool => values::BitBool::new(true).into(), + _ => panic!("{kind:?} is not a leaf kind"), + } +} + +const LEAF_KINDS: [Kind; 21] = [ + Kind::None, + Kind::Bool, + Kind::I8, + Kind::U8, + Kind::I16, + Kind::U16, + Kind::I32, + Kind::U32, + Kind::I64, + Kind::U64, + Kind::F32, + Kind::Vector2, + Kind::Vector3, + Kind::Vector4, + Kind::Matrix44, + Kind::Color, + Kind::String, + Kind::Hash, + Kind::WadChunkLink, + Kind::ObjectLink, + Kind::BitBool, +]; + +fn key_kinds() -> impl Iterator { + LEAF_KINDS.into_iter().filter(Kind::is_valid_map_key) +} + +/// The section 7 fixture: a `Struct` and an `Embedded` at a property, inside a container, +/// inside an optional, as a map value; a null pointer in each position; a container of +/// strings; a map keyed by every kind `Kind::is_valid_map_key` admits; one leaf of every kind. +fn fixture() -> Bin { + let mut object = BinObject::builder(OBJECT, C1) + .property( + F_STRUCT, + node( + C2, + vec![ + (F_LEAF, values::I32::new(1).into()), + ( + F_INNER, + Embedded(node(C9, vec![(F_LEAF, leaf_of(Kind::F32))])).into(), + ), + ], + ), + ) + .property( + F_EMBED, + Embedded(node(C3, vec![(F_LEAF, leaf_of(Kind::String))])), + ) + .property(F_NULL_STRUCT, null()) + .property(F_NULL_EMBED, Embedded(null())) + .property( + F_CONT_STRUCT, + values::Container::from(vec![node(C4, vec![]), null(), node(C4, vec![])]), + ) + .property( + F_CONT_EMBED, + UnorderedContainer(values::Container::from(vec![ + Embedded(node(C5, vec![])), + Embedded(null()), + ])), + ) + .property( + F_OPT_STRUCT, + values::Optional::new(Kind::Struct, Some(node(C6, vec![]).into())).unwrap(), + ) + .property( + F_OPT_NULL, + values::Optional::new(Kind::Struct, Some(null().into())).unwrap(), + ) + .property( + F_OPT_EMPTY, + values::Optional::empty(Kind::Embedded).unwrap(), + ) + .property( + F_MAP_STRUCT, + values::Map::new( + Kind::Hash, + Kind::Struct, + vec![ + (values::Hash::new(KEY_A).into(), node(C7, vec![]).into()), + (values::Hash::new(KEY_B).into(), null().into()), + ], + ) + .unwrap(), + ) + .property( + F_MAP_EMBED, + values::Map::new( + Kind::String, + Kind::Embedded, + vec![( + values::String::from("k").into(), + Embedded(node(C8, vec![])).into(), + )], + ) + .unwrap(), + ) + .property( + F_STRINGS, + values::Container::from(vec![values::String::from("a"), values::String::from("b")]), + ) + .build(); + + for kind in LEAF_KINDS { + object + .properties + .insert(BinHash(F_LEAVES + kind as u32), leaf_of(kind)); + } + for kind in key_kinds() { + let map = values::Map::new( + kind, + Kind::I32, + vec![(leaf_of(kind), values::I32::new(7).into())], + ) + .unwrap(); + object + .properties + .insert(BinHash(F_KEYS + kind as u32), map.into()); + } + + Bin::builder().object(object).build() +} + +fn bytes_of(bin: &Bin) -> Vec { + let mut cursor = io::Cursor::new(Vec::new()); + bin.to_writer(&mut cursor).expect("the bin writes"); + cursor.into_inner() +} + +/// Every node of the fixture: `(trail, class)`, in pre-order. +const EXPECTED_NODES: [(&str, u32); 10] = [ + ("", C1), + ("00000001", C2), + ("00000001.00000011", C9), + ("00000002", C3), + ("00000005[0]", C4), + ("00000005[2]", C4), + ("00000006[0]", C5), + ("00000007[0]", C6), + ("0000000a{000000aa}", C7), + ("0000000b{\"k\"}", C8), +]; + +#[derive(Debug, Clone, PartialEq)] +enum Event { + EnterNode { + object: u32, + class: u32, + trail: String, + classes: Vec, + }, + ExitNode { + class: u32, + trail: String, + }, + EnterProperty { + field: u32, + trail: String, + holds_node: bool, + }, + ExitProperty { + field: u32, + trail: String, + }, +} + +impl Event { + fn is_enter_node(&self) -> bool { + matches!(self, Event::EnterNode { .. }) + } +} + +/// Records every callback and answers each with `answer`. +struct Recorder { + events: Vec, + answer: fn(&Event) -> Result, +} + +impl Recorder { + fn new(answer: fn(&Event) -> Result) -> Self { + Self { + events: Vec::new(), + answer, + } + } + + fn record(&mut self, event: Event) -> Result { + let visit = (self.answer)(&event); + self.events.push(event); + visit + } + + fn nodes(&self) -> Vec<(String, u32)> { + self.events + .iter() + .filter_map(|e| match e { + Event::EnterNode { trail, class, .. } => Some((trail.clone(), *class)), + _ => None, + }) + .collect() + } +} + +fn hashes(classes: &[BinHash]) -> Vec { + classes.iter().map(|h| h.0).collect() +} + +impl<'a, V: TreeValue<'a>> Visitor<'a, V> for Recorder { + type Error = Error; + + fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + self.record(Event::EnterNode { + object: node.object_hash().0, + class: node.class_hash().0, + trail: node.trail().to_string(), + classes: hashes(node.trail().classes()), + }) + } + + fn exit_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + self.record(Event::ExitNode { + class: node.class_hash().0, + trail: node.trail().to_string(), + }) + } + + fn enter_property( + &mut self, + field: BinHash, + value: V, + node: &Node<'_, 'a, V>, + ) -> Result { + self.record(Event::EnterProperty { + field: field.0, + trail: node.trail().to_string(), + holds_node: value.holds_node()?, + }) + } + + fn exit_property( + &mut self, + field: BinHash, + _value: V, + node: &Node<'_, 'a, V>, + ) -> Result { + self.record(Event::ExitProperty { + field: field.0, + trail: node.trail().to_string(), + }) + } +} + +/// Runs one visitor over the owned tree and another over the view of the same bytes. +fn walk_both(bin: &Bin, make: impl Fn() -> W) -> [(W, Result); 2] +where + W: for<'a> Visitor<'a, &'a PropertyValueEnum, Error = Error> + + for<'a> Visitor<'a, ValueView<'a>, Error = Error>, +{ + let mut owned = make(); + let owned_outcome = bin.walk(&mut owned); + + let mut viewed = make(); + let mut stream = BinStream::mount(io::Cursor::new(bytes_of(bin))).expect("the stream mounts"); + let viewed_outcome = stream.walk(&mut viewed); + + [(owned, owned_outcome), (viewed, viewed_outcome)] +} + +fn always_continue(_: &Event) -> Result { + Ok(Visit::Continue) +} + +fn record_both(bin: &Bin, answer: fn(&Event) -> Result) -> [Recorder; 2] { + let [(owned, a), (viewed, b)] = walk_both(bin, || Recorder::new(answer)); + assert_eq!( + a.as_ref().ok(), + b.as_ref().ok(), + "the two trees ended differently" + ); + assert_eq!(owned.events, viewed.events, "the two trees differ"); + [owned, viewed] +} + +#[test] +fn visits_every_node_in_pre_order_over_both_trees() { + let [owned, _] = record_both(&fixture(), always_continue); + let expected: Vec<_> = EXPECTED_NODES + .iter() + .map(|(trail, class)| ((*trail).to_owned(), *class)) + .collect(); + assert_eq!(owned.nodes(), expected); + assert!(owned + .events + .iter() + .all(|e| !matches!(e, Event::EnterNode { object, .. } if *object != OBJECT))); +} + +#[test] +fn a_null_pointer_is_never_visited() { + let [owned, _] = record_both(&fixture(), always_continue); + assert!(!owned + .events + .iter() + .any(|e| matches!(e, Event::EnterNode { class: 0, .. }))); + // The null positions: `F_NULL_STRUCT`, `F_NULL_EMBED`, `[1]` of the container, `[1]` of the + // unordered container, the optional, and `KEY_B` of the map. + for absent in [ + "00000003", + "00000004", + "00000005[1]", + "00000006[1]", + "00000008[0]", + "0000000a{000000bb}", + ] { + assert!( + !owned.nodes().iter().any(|(trail, _)| trail == absent), + "{absent} was visited" + ); + } +} + +#[test] +fn skip_at_a_property_prunes_only_what_is_beneath_it() { + fn skip_struct(event: &Event) -> Result { + Ok(match event { + Event::EnterProperty { + field: F_STRUCT, .. + } => Visit::Skip, + _ => Visit::Continue, + }) + } + let [owned, _] = record_both(&fixture(), skip_struct); + let expected: Vec<_> = EXPECTED_NODES + .iter() + .filter(|(trail, _)| !trail.starts_with("00000001")) + .map(|(trail, class)| ((*trail).to_owned(), *class)) + .collect(); + assert_eq!(owned.nodes(), expected); + assert!(owned.events.contains(&Event::ExitProperty { + field: F_STRUCT, + trail: String::new(), + })); +} + +#[test] +fn exits_pair_with_entries_and_a_leaf_has_none() { + let [owned, _] = record_both(&fixture(), always_continue); + + let entered_nodes = owned.events.iter().filter(|e| e.is_enter_node()).count(); + let exited_nodes = owned + .events + .iter() + .filter(|e| matches!(e, Event::ExitNode { .. })) + .count(); + assert_eq!(entered_nodes, exited_nodes); + assert_eq!(entered_nodes, EXPECTED_NODES.len()); + + let mut descended: Vec<_> = owned + .events + .iter() + .filter_map(|e| match e { + Event::EnterProperty { + field, + trail, + holds_node: true, + } => Some((*field, trail.clone())), + _ => None, + }) + .collect(); + let mut exited: Vec<_> = owned + .events + .iter() + .filter_map(|e| match e { + Event::ExitProperty { field, trail } => Some((*field, trail.clone())), + _ => None, + }) + .collect(); + descended.sort(); + exited.sort(); + assert_eq!(descended, exited); + // The empty optional and the leaves of every kind are entered and never exited. + assert!(descended.contains(&(F_OPT_EMPTY, String::new()))); + assert!(!exited.iter().any(|(field, _)| *field == F_STRINGS)); + assert!(!exited.iter().any(|(field, _)| *field >= F_LEAVES)); + + // Well nested: every exit closes the innermost open entry. + let mut open: Vec<&Event> = Vec::new(); + for event in &owned.events { + match event { + Event::EnterNode { .. } => open.push(event), + Event::EnterProperty { + holds_node: true, .. + } => open.push(event), + Event::EnterProperty { .. } => {} + Event::ExitNode { class, trail } => { + let Some(Event::EnterNode { + class: c, trail: t, .. + }) = open.pop() + else { + panic!("exit_node without an open node"); + }; + assert_eq!((c, t), (class, trail)); + } + Event::ExitProperty { field, trail } => { + let Some(Event::EnterProperty { + field: f, trail: t, .. + }) = open.pop() + else { + panic!("exit_property without an open property"); + }; + assert_eq!((f, t), (field, trail)); + } + } + } + assert!(open.is_empty()); +} + +#[test] +fn skip_from_enter_node_skips_its_properties_and_still_exits() { + fn skip_c2(event: &Event) -> Result { + Ok(match event { + Event::EnterNode { class: C2, .. } => Visit::Skip, + _ => Visit::Continue, + }) + } + let [owned, _] = record_both(&fixture(), skip_c2); + let nodes = owned.nodes(); + assert!(!nodes.iter().any(|(_, class)| *class == C9)); + assert!(!owned + .events + .iter() + .any(|e| matches!(e, Event::EnterProperty { trail, .. } if trail == "00000001"))); + assert!(owned.events.contains(&Event::ExitNode { + class: C2, + trail: "00000001".into(), + })); + assert_eq!(nodes.len(), EXPECTED_NODES.len() - 1); +} + +#[test] +fn skip_from_exit_property_prunes_the_remaining_properties() { + fn skip_after_struct(event: &Event) -> Result { + Ok(match event { + Event::ExitProperty { + field: F_STRUCT, .. + } => Visit::Skip, + _ => Visit::Continue, + }) + } + let [owned, _] = record_both(&fixture(), skip_after_struct); + assert_eq!( + owned.nodes(), + [ + (String::new(), C1), + ("00000001".to_owned(), C2), + ("00000001.00000011".to_owned(), C9), + ] + ); + let root_properties = owned + .events + .iter() + .filter(|e| matches!(e, Event::EnterProperty { trail, .. } if trail.is_empty())) + .count(); + assert_eq!(root_properties, 1); + assert_eq!( + owned.events.last(), + Some(&Event::ExitNode { + class: C1, + trail: String::new(), + }) + ); +} + +#[test] +fn skip_from_exit_node_prunes_the_parent_propertys_remaining_items() { + fn skip_after_first_c4(event: &Event) -> Result { + Ok(match event { + Event::ExitNode { trail, .. } if trail == "00000005[0]" => Visit::Skip, + _ => Visit::Continue, + }) + } + let [owned, _] = record_both(&fixture(), skip_after_first_c4); + let nodes = owned.nodes(); + assert!(!nodes.iter().any(|(trail, _)| trail == "00000005[2]")); + assert!(nodes.iter().any(|(trail, _)| trail == "00000006[0]")); + let position = |event: &Event| owned.events.iter().position(|e| e == event).unwrap(); + let exit_c4 = position(&Event::ExitNode { + class: C4, + trail: "00000005[0]".into(), + }); + let exit_property = position(&Event::ExitProperty { + field: F_CONT_STRUCT, + trail: String::new(), + }); + assert_eq!(exit_property, exit_c4 + 1); +} + +#[test] +fn stop_unwinds_every_open_exit_and_reports_stopped() { + fn stop_at_c9(event: &Event) -> Result { + Ok(match event { + Event::EnterNode { class: C9, .. } => Visit::Stop, + _ => Visit::Continue, + }) + } + let [(owned, outcome), (_, viewed_outcome)] = + walk_both(&fixture(), || Recorder::new(stop_at_c9)); + assert_eq!(outcome.unwrap(), WalkOutcome::Stopped); + assert_eq!(viewed_outcome.unwrap(), WalkOutcome::Stopped); + let at = owned + .events + .iter() + .position(|e| matches!(e, Event::EnterNode { class: C9, .. })) + .unwrap(); + assert_eq!( + &owned.events[at + 1..], + [ + Event::ExitNode { + class: C9, + trail: "00000001.00000011".into(), + }, + Event::ExitProperty { + field: F_INNER, + trail: "00000001".into(), + }, + Event::ExitNode { + class: C2, + trail: "00000001".into(), + }, + Event::ExitProperty { + field: F_STRUCT, + trail: String::new(), + }, + Event::ExitNode { + class: C1, + trail: String::new(), + }, + ] + ); +} + +#[test] +fn stop_from_enter_property_exits_a_property_that_holds_a_node() { + fn stop_at_struct(event: &Event) -> Result { + Ok(match event { + Event::EnterProperty { + field: F_STRUCT, .. + } => Visit::Stop, + _ => Visit::Continue, + }) + } + let [(owned, outcome), _] = walk_both(&fixture(), || Recorder::new(stop_at_struct)); + assert_eq!(outcome.unwrap(), WalkOutcome::Stopped); + assert_eq!( + &owned.events[2..], + [ + Event::ExitProperty { + field: F_STRUCT, + trail: String::new(), + }, + Event::ExitNode { + class: C1, + trail: String::new(), + }, + ] + ); +} + +#[test] +fn abort_runs_no_further_callback_and_reports_aborted() { + fn abort_at_c9(event: &Event) -> Result { + Ok(match event { + Event::EnterNode { class: C9, .. } => Visit::Abort, + _ => Visit::Continue, + }) + } + let [(owned, outcome), (viewed, viewed_outcome)] = + walk_both(&fixture(), || Recorder::new(abort_at_c9)); + assert_eq!(outcome.unwrap(), WalkOutcome::Aborted); + assert_eq!(viewed_outcome.unwrap(), WalkOutcome::Aborted); + assert!(matches!( + owned.events.last(), + Some(Event::EnterNode { class: C9, .. }) + )); + assert_eq!(owned.events, viewed.events); +} + +#[test] +fn a_visitor_error_ends_the_walk_like_an_abort() { + fn fail_at_c9(event: &Event) -> Result { + match event { + Event::EnterNode { class: C9, .. } => Err(Error::EmptyContainer), + _ => Ok(Visit::Continue), + } + } + let [(owned, outcome), (viewed, viewed_outcome)] = + walk_both(&fixture(), || Recorder::new(fail_at_c9)); + assert!(matches!(outcome, Err(Error::EmptyContainer))); + assert!(matches!(viewed_outcome, Err(Error::EmptyContainer))); + assert!(matches!( + owned.events.last(), + Some(Event::EnterNode { class: C9, .. }) + )); + assert_eq!(owned.events, viewed.events); +} + +/// Collects `to_struct` at every node and `leaf` and `to_value` at every property. +#[derive(Default)] +struct Materialiser { + structs: Vec<(String, values::Struct)>, + leaves: Vec<(u32, Option>)>, + values: Vec<(u32, PropertyValueEnum)>, +} + +fn owned_leaf(leaf: Leaf<'_>) -> Leaf<'static> { + match leaf { + Leaf::String(s) => Leaf::String(Box::leak(s.to_owned().into_boxed_str())), + Leaf::None => Leaf::None, + Leaf::Bool(v) => Leaf::Bool(v), + Leaf::I8(v) => Leaf::I8(v), + Leaf::U8(v) => Leaf::U8(v), + Leaf::I16(v) => Leaf::I16(v), + Leaf::U16(v) => Leaf::U16(v), + Leaf::I32(v) => Leaf::I32(v), + Leaf::U32(v) => Leaf::U32(v), + Leaf::I64(v) => Leaf::I64(v), + Leaf::U64(v) => Leaf::U64(v), + Leaf::F32(v) => Leaf::F32(v), + Leaf::Vector2(v) => Leaf::Vector2(v), + Leaf::Vector3(v) => Leaf::Vector3(v), + Leaf::Vector4(v) => Leaf::Vector4(v), + Leaf::Matrix44(v) => Leaf::Matrix44(v), + Leaf::Color(v) => Leaf::Color(v), + Leaf::Hash(v) => Leaf::Hash(v), + Leaf::File(v) => Leaf::File(v), + Leaf::Link(v) => Leaf::Link(v), + Leaf::Flag(v) => Leaf::Flag(v), + } +} + +impl<'a, V: TreeValue<'a>> Visitor<'a, V> for Materialiser { + type Error = Error; + + fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + self.structs + .push((node.trail().to_string(), node.inner().to_struct()?)); + Ok(Visit::Continue) + } + + fn enter_property( + &mut self, + field: BinHash, + value: V, + node: &Node<'_, 'a, V>, + ) -> Result { + if node.is_root() { + self.leaves.push((field.0, value.leaf()?.map(owned_leaf))); + self.values.push((field.0, value.to_value()?)); + } + Ok(Visit::Continue) + } +} + +#[test] +fn to_struct_equals_the_eager_parse_on_a_root_and_a_nested_node() { + let bin = fixture(); + let object = &bin.objects[&BinHash(OBJECT)]; + let [(owned, _), (viewed, _)] = walk_both(&bin, Materialiser::default); + assert_eq!(owned.structs, viewed.structs); + + let (root_trail, root) = &owned.structs[0]; + assert!(root_trail.is_empty()); + assert_eq!(root.class_hash.0, C1); + assert_eq!(root.properties, object.properties); + + let (nested_trail, nested) = &owned.structs[1]; + assert_eq!(nested_trail, "00000001"); + assert_eq!( + PropertyValueEnum::Struct(nested.clone()), + object.properties[&BinHash(F_STRUCT)] + ); +} + +#[test] +fn leaves_and_values_agree_between_the_trees_for_every_kind() { + let bin = fixture(); + let object = &bin.objects[&BinHash(OBJECT)]; + let [(owned, _), (viewed, _)] = walk_both(&bin, Materialiser::default); + assert_eq!(owned.leaves, viewed.leaves); + assert_eq!(owned.values, viewed.values); + + for kind in LEAF_KINDS { + let field = F_LEAVES + kind as u32; + let (_, leaf) = owned.leaves.iter().find(|(f, _)| *f == field).unwrap(); + let leaf = leaf.expect("a leaf kind decodes"); + assert_eq!(leaf.kind(), kind); + } + assert_eq!( + owned + .leaves + .iter() + .filter(|(f, leaf)| *f < F_LEAVES && leaf.is_some()) + .count(), + 0, + "a complex kind is not a leaf" + ); + for (field, value) in &owned.values { + assert_eq!(value, &object.properties[&BinHash(*field)], "{field:x}"); + } +} + +#[test] +fn map_keys_of_every_kind_agree_between_the_trees() { + let bin = fixture(); + let [(owned, _), (viewed, _)] = walk_both(&bin, Materialiser::default); + for kind in key_kinds() { + let field = F_KEYS + kind as u32; + let find = |m: &Materialiser| { + m.values + .iter() + .find(|(f, _)| *f == field) + .unwrap() + .1 + .clone() + }; + let (a, b) = (find(&owned), find(&viewed)); + assert_eq!(a, b, "{kind:?}"); + let PropertyValueEnum::Map(map) = a else { + panic!("{kind:?}: not a map") + }; + assert_eq!(map.key_kind(), kind); + assert_eq!(map.entries()[0].0, leaf_of(kind)); + } +} + +#[test] +fn the_class_context_holds_the_class_of_every_enclosing_node() { + let [owned, _] = record_both(&fixture(), always_continue); + // A field step is read on exactly one node: the context is the open nodes, root first. + let mut open: Vec = Vec::new(); + for event in &owned.events { + match event { + Event::EnterNode { + class, + classes, + trail, + .. + } => { + assert_eq!(classes, &open, "{trail}"); + open.push(*class); + } + Event::ExitNode { .. } => { + open.pop(); + } + _ => {} + } + } + assert!(owned.events.contains(&Event::EnterNode { + object: OBJECT, + class: C9, + trail: "00000001.00000011".into(), + classes: vec![C1, C2], + })); +} + +#[test] +fn stop_on_a_leaf_property_exits_nothing_for_it() { + fn stop_at_strings(event: &Event) -> Result { + Ok(match event { + Event::EnterProperty { + field: F_STRINGS, .. + } => Visit::Stop, + _ => Visit::Continue, + }) + } + let [(owned, outcome), _] = walk_both(&fixture(), || Recorder::new(stop_at_strings)); + assert_eq!(outcome.unwrap(), WalkOutcome::Stopped); + let at = owned + .events + .iter() + .position(|e| { + matches!( + e, + Event::EnterProperty { + field: F_STRINGS, + .. + } + ) + }) + .unwrap(); + assert_eq!( + &owned.events[at + 1..], + [Event::ExitNode { + class: C1, + trail: String::new(), + }] + ); +} + +#[test] +fn stop_and_abort_from_an_exit_end_the_walk() { + fn stop_at_exit_property(event: &Event) -> Result { + Ok(match event { + Event::ExitProperty { field: F_INNER, .. } => Visit::Stop, + _ => Visit::Continue, + }) + } + let [(owned, outcome), _] = walk_both(&fixture(), || Recorder::new(stop_at_exit_property)); + assert_eq!(outcome.unwrap(), WalkOutcome::Stopped); + let at = owned + .events + .iter() + .position(|e| matches!(e, Event::ExitProperty { field: F_INNER, .. })) + .unwrap(); + assert_eq!( + &owned.events[at + 1..], + [ + Event::ExitNode { + class: C2, + trail: "00000001".into(), + }, + Event::ExitProperty { + field: F_STRUCT, + trail: String::new(), + }, + Event::ExitNode { + class: C1, + trail: String::new(), + }, + ] + ); + + fn abort_at_exit_node(event: &Event) -> Result { + Ok(match event { + Event::ExitNode { class: C9, .. } => Visit::Abort, + _ => Visit::Continue, + }) + } + let [(owned, outcome), _] = walk_both(&fixture(), || Recorder::new(abort_at_exit_node)); + assert_eq!(outcome.unwrap(), WalkOutcome::Aborted); + assert!(matches!( + owned.events.last(), + Some(Event::ExitNode { class: C9, .. }) + )); +} + +/// A map keyed by every kind, to a node, for the hash form of each key. +fn keyed_fixture() -> Bin { + let mut object = BinObject::builder(OBJECT, C1).build(); + for kind in key_kinds() { + let map = values::Map::new( + kind, + Kind::Struct, + vec![(leaf_of(kind), node(C7, vec![]).into())], + ) + .unwrap(); + object + .properties + .insert(BinHash(F_KEYS + kind as u32), map.into()); + } + Bin::builder().object(object).build() +} + +#[test] +fn the_hash_form_renders_every_key_kind() { + let [owned, _] = record_both(&keyed_fixture(), always_continue); + let expected = [ + (Kind::None, "{}"), + (Kind::Bool, "{true}"), + (Kind::I8, "{-8}"), + (Kind::U8, "{8}"), + (Kind::I16, "{-16}"), + (Kind::U16, "{16}"), + (Kind::I32, "{-32}"), + (Kind::U32, "{32}"), + (Kind::I64, "{-64}"), + (Kind::U64, "{64}"), + (Kind::F32, "{1.5}"), + (Kind::Vector2, "{(1, 2)}"), + (Kind::Vector3, "{(1, 2, 3)}"), + (Kind::Vector4, "{(1, 2, 3, 4)}"), + ( + Kind::Matrix44, + "{(1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15, 4, 8, 12, 16)}", + ), + (Kind::Color, "{(1, 2, 3, 4)}"), + (Kind::String, "{\"weapon\"}"), + (Kind::Hash, "{1e6ba0c4}"), + (Kind::WadChunkLink, "{00c9fd8f1a2b3c4d}"), + ]; + let nodes = owned.nodes(); + for (kind, key) in expected { + let trail = format!("{:08x}{key}", F_KEYS + kind as u32); + assert!(nodes.contains(&(trail.clone(), C7)), "{kind:?}: {trail}"); + } + assert_eq!(nodes.len(), 1 + expected.len()); +} + +#[test] +fn a_string_key_is_a_json_string() { + let object = BinObject::builder(OBJECT, C1) + .property( + F_MAP_EMBED, + values::Map::new( + Kind::String, + Kind::Struct, + vec![( + values::String::from("a\"b\\c\n\u{1}").into(), + node(C7, vec![]).into(), + )], + ) + .unwrap(), + ) + .build(); + let [owned, _] = record_both(&Bin::builder().object(object).build(), always_continue); + assert_eq!(owned.nodes()[1].0, "0000000b{\"a\\\"b\\\\c\\n\\u0001\"}"); +} + +/// Records the trail's capacity at every node. +#[derive(Default)] +struct Capacities(Vec<(usize, usize)>); + +impl<'a, V: TreeValue<'a>> Visitor<'a, V> for Capacities { + type Error = Error; + + fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + let trail = node.trail(); + self.0 + .push((trail.steps.capacity(), trail.classes.capacity())); + Ok(Visit::Continue) + } +} + +#[test] +fn a_map_of_ten_thousand_entries_grows_the_trail_once() { + let entries: Vec<_> = (0..10_000u32) + .map(|i| { + ( + values::Hash::new(i).into(), + PropertyValueEnum::Struct(node(C7, vec![])), + ) + }) + .collect(); + let object = BinObject::builder(OBJECT, C1) + .property( + F_MAP_STRUCT, + values::Map::new(Kind::Hash, Kind::Struct, entries).unwrap(), + ) + .build(); + let bin = Bin::builder().object(object).build(); + + let [(owned, _), (viewed, _)] = walk_both(&bin, Capacities::default); + for visited in [owned, viewed] { + assert_eq!(visited.0.len(), 10_001); + // The root sees an empty trail; every entry after it sees the same two-step trail, at a + // capacity that never moves once it is set. + let first = visited.0[1]; + assert!(first.0 <= 4 && first.1 <= 4, "{first:?}"); + assert!(visited.0[1..].iter().all(|c| *c == first)); + } +} + +const UIBASE: &[u8] = include_bytes!("../../tests/bins/lolminimap_uibase.bin"); +const UIFLIPPED: &[u8] = include_bytes!("../../tests/bins/lolminimap_uiflipped.ptch.bin"); + +/// `(object, class, trail)` per node. +#[derive(Default)] +struct Roots(Vec<(u32, u32, String)>); + +impl<'a, V: TreeValue<'a>> Visitor<'a, V> for Roots { + type Error = Error; + + fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + self.0.push(( + node.object_hash().0, + node.class_hash().0, + node.trail().to_string(), + )); + Ok(Visit::Continue) + } +} + +#[test] +fn a_shipped_bin_walks_the_same_over_both_trees() { + let bin = Bin::from_reader(&mut io::Cursor::new(UIBASE)).unwrap(); + let mut owned = Roots::default(); + bin.walk(&mut owned).unwrap(); + + let mut stream = BinStream::mount(io::Cursor::new(UIBASE.to_vec())).unwrap(); + let mut viewed = Roots::default(); + stream.walk(&mut viewed).unwrap(); + + assert_eq!(owned.0, viewed.0); + let roots: Vec<_> = owned + .0 + .iter() + .filter(|(_, _, trail)| trail.is_empty()) + .map(|(object, _, _)| *object) + .collect(); + assert_eq!(roots.len(), 66); + assert_eq!( + roots, + bin.objects.keys().map(|h| h.0).collect::>(), + "roots in file order" + ); + assert!(owned.0.len() > roots.len(), "the bin has nested nodes"); +} + +#[test] +fn an_override_walks_its_embedded_objects_and_never_a_record() { + let patch = BinOverride::from_reader(&mut io::Cursor::new(UIFLIPPED)).unwrap(); + assert!(!patch.patches.is_empty()); + let mut visited = Roots::default(); + patch.walk(&mut visited).unwrap(); + + let roots: Vec<_> = visited + .0 + .iter() + .filter(|(_, _, trail)| trail.is_empty()) + .map(|(object, _, _)| *object) + .collect(); + assert_eq!(roots, patch.objects.keys().map(|h| h.0).collect::>()); + for (object, _, _) in &visited.0 { + assert!(patch.objects.contains_key(&BinHash(*object))); + } +} + +#[test] +fn a_mutable_reference_to_a_visitor_is_a_visitor() { + let bin = fixture(); + let mut recorder = Recorder::new(always_continue); + let by_ref: &mut dyn Visitor<'_, &PropertyValueEnum, Error = Error> = &mut recorder; + let mut by_ref = by_ref; + bin.walk(&mut by_ref).unwrap(); + assert_eq!(recorder.nodes().len(), EXPECTED_NODES.len()); +} diff --git a/crates/ltk_meta/src/walk/tree.rs b/crates/ltk_meta/src/walk/tree.rs new file mode 100644 index 00000000..f0046ab0 --- /dev/null +++ b/crates/ltk_meta/src/walk/tree.rs @@ -0,0 +1,296 @@ +//! The tree the walk sees: two sealed traits, a child step, and a decoded leaf. + +use std::fmt; + +use glam::{Mat4, Vec2, Vec3, Vec4}; +use ltk_hash::{BinHash, WadHash}; +use ltk_primitives::Color; + +use crate::{property::values, property::Kind, Error, PropertyValueEnum}; + +pub(crate) mod sealed { + pub trait Sealed {} +} + +/// Whether a [`Kind`] is a node kind. +/// +/// Sealed: implemented for [`Kind`] and by nothing else. A node kind is `Struct` or +/// `Embedded`. The walk asks nothing else of a kind. +/// +/// # Examples +/// +/// ``` +/// use ltk_meta::{walk::TreeKind, PropertyKind}; +/// +/// assert!(PropertyKind::Struct.is_node()); +/// assert!(PropertyKind::Embedded.is_node()); +/// assert!(!PropertyKind::ObjectLink.is_node()); +/// assert!(!PropertyKind::Container.is_node()); +/// ``` +pub trait TreeKind: Copy + sealed::Sealed { + /// Whether a value of this kind is a node: `Struct` or `Embedded`. + /// + /// [`Kind::is_primitive`] answers a different question, which kinds a value model treats + /// as leaves. The two are not complements: `ObjectLink` and `BitBool` are neither + /// primitive nor a node. + fn is_node(self) -> bool; +} + +impl sealed::Sealed for Kind {} +impl TreeKind for Kind { + #[inline] + fn is_node(self) -> bool { + matches!(self, Kind::Struct | Kind::Embedded) + } +} + +/// A value the walk can cross. +/// +/// Sealed: implemented for `&'a PropertyValueEnum` and for [`ValueView<'a, M>`], and by +/// nothing else. A visitor written against this trait runs over either tree. +/// +/// [`ValueView<'a, M>`]: crate::stream::ValueView +pub trait TreeValue<'a>: Copy + sealed::Sealed { + /// The node type this tree's `Struct` and `Embedded` values are. + type Node: TreeNode<'a, Value = Self>; + /// The values inside a container, optional or map, each with the child step reaching it. + type Children: Iterator, Self), Error>>; + + /// The kind this value is. + fn kind(&self) -> Kind; + + /// Whether entering this value can reach a node. + /// + /// True for a `Struct` or `Embedded` whose class hash is not 0, and for a container, + /// optional or map whose item kind [`TreeKind::is_node`]. An empty optional or container + /// of a node kind answers true. Entering it costs nothing. + /// + /// # Errors + /// + /// Over a view, a header that does not decode. The owned tree never fails. + fn holds_node(&self) -> Result; + + /// This value as a node, if it is a `Struct` or `Embedded` with a class hash that is not 0. + /// + /// # Errors + /// + /// Over a view, a header that does not decode. The owned tree never fails. + fn as_node(&self) -> Result, Error>; + + /// The values inside this one, with the step reaching each. Empty for a leaf and for a + /// node. A node's contents are its properties. + /// + /// # Errors + /// + /// Over a view, a header that does not decode. The owned tree never fails. + fn children(&self) -> Result; + + /// This value decoded, if it is a leaf kind. `None` for every complex kind. + /// + /// # Errors + /// + /// Over a view, a leaf that does not decode. The owned tree never fails. + fn leaf(&self) -> Result>, Error>; + + /// The whole value, owned. Allocates. A visitor reaches for it for a subtree, not for a + /// leaf. + /// + /// # Errors + /// + /// Over a view, whatever the eager reader raises for the same bytes. The owned tree never + /// fails. + fn to_value(&self) -> Result; +} + +/// A node the walk can visit: a class and properties. +/// +/// Sealed: implemented for the owned tree's node, [`OwnedNode`](super::OwnedNode), and for +/// [`StructView<'a, M>`]. An object's root is a `StructView` over the same bytes. +/// +/// [`StructView<'a, M>`]: crate::stream::StructView +pub trait TreeNode<'a>: Copy + sealed::Sealed { + /// The value type of this tree. + type Value: TreeValue<'a, Node = Self>; + /// The properties in file order. A view's kind byte can fail to decode. Items are + /// `Result`; the owned tree never fails. + type Properties: Iterator>; + + /// The class hash this node carries. + fn class_hash(&self) -> BinHash; + + /// The properties, in file order. + fn properties(&self) -> Self::Properties; + + /// One property by field hash. The owned tree looks it up by key; the view scans in + /// place. + /// + /// # Errors + /// + /// Over a view, a kind byte that does not decode before the property is reached. The + /// owned tree never fails. + fn property(&self, field: BinHash) -> Result, Error>; + + /// The whole node, owned, as a `Struct` carrying this class and every property. + /// Allocates. The object's path hash is not part of a `Struct`; a root's is + /// [`Node::object_hash`](super::Node::object_hash). + /// + /// # Errors + /// + /// Over a view, whatever the eager reader raises for the same bytes. The owned tree never + /// fails. + fn to_struct(&self) -> Result; +} + +/// The step from a container, optional or map to one value inside it. +#[derive(Clone, Copy, Debug, PartialEq)] +pub enum Child { + /// A container element by position, or the value of a present optional, which is 0. + Index(usize), + /// A map entry, by its key value. + Key(V), +} + +/// A leaf, decoded and borrowed. +/// +/// The client's names for the tags, not the wire enum's: `File` is [`Kind::WadChunkLink`], +/// `Link` is [`Kind::ObjectLink`], `Flag` is [`Kind::BitBool`]. +/// +/// Non-exhaustive: the set of leaf kinds is the game's. A match outside this crate carries a +/// wildcard arm. +#[derive(Clone, Copy, Debug, PartialEq)] +#[non_exhaustive] +pub enum Leaf<'a> { + /// [`Kind::None`]. + None, + /// [`Kind::Bool`]. + Bool(bool), + /// [`Kind::I8`]. + I8(i8), + /// [`Kind::U8`]. + U8(u8), + /// [`Kind::I16`]. + I16(i16), + /// [`Kind::U16`]. + U16(u16), + /// [`Kind::I32`]. + I32(i32), + /// [`Kind::U32`]. + U32(u32), + /// [`Kind::I64`]. + I64(i64), + /// [`Kind::U64`]. + U64(u64), + /// [`Kind::F32`]. + F32(f32), + /// [`Kind::Vector2`]. + Vector2(Vec2), + /// [`Kind::Vector3`]. + Vector3(Vec3), + /// [`Kind::Vector4`]. + Vector4(Vec4), + /// [`Kind::Matrix44`]. + Matrix44(Mat4), + /// [`Kind::Color`]. + Color(Color), + /// [`Kind::String`]. + String(&'a str), + /// [`Kind::Hash`]. + Hash(BinHash), + /// [`Kind::WadChunkLink`]. + File(WadHash), + /// [`Kind::ObjectLink`]. + Link(BinHash), + /// [`Kind::BitBool`]. + Flag(bool), +} + +impl Leaf<'_> { + /// The kind this leaf is. + #[must_use] + pub fn kind(&self) -> Kind { + match self { + Self::None => Kind::None, + Self::Bool(_) => Kind::Bool, + Self::I8(_) => Kind::I8, + Self::U8(_) => Kind::U8, + Self::I16(_) => Kind::I16, + Self::U16(_) => Kind::U16, + Self::I32(_) => Kind::I32, + Self::U32(_) => Kind::U32, + Self::I64(_) => Kind::I64, + Self::U64(_) => Kind::U64, + Self::F32(_) => Kind::F32, + Self::Vector2(_) => Kind::Vector2, + Self::Vector3(_) => Kind::Vector3, + Self::Vector4(_) => Kind::Vector4, + Self::Matrix44(_) => Kind::Matrix44, + Self::Color(_) => Kind::Color, + Self::String(_) => Kind::String, + Self::Hash(_) => Kind::Hash, + Self::File(_) => Kind::WadChunkLink, + Self::Link(_) => Kind::ObjectLink, + Self::Flag(_) => Kind::BitBool, + } + } + + /// Writes this leaf as the text inside a `{key}` step of the hash form. + /// + /// An integer in decimal, a bool as `true` or `false`, a float in its shortest + /// round-trip form, a string as a JSON string, a hash as lowercase zero-padded hex, a + /// vector, colour or matrix as its components in parentheses, and `None` as nothing. + pub(crate) fn write_key(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::None => Ok(()), + Self::Bool(v) | Self::Flag(v) => write!(f, "{v}"), + Self::I8(v) => write!(f, "{v}"), + Self::U8(v) => write!(f, "{v}"), + Self::I16(v) => write!(f, "{v}"), + Self::U16(v) => write!(f, "{v}"), + Self::I32(v) => write!(f, "{v}"), + Self::U32(v) => write!(f, "{v}"), + Self::I64(v) => write!(f, "{v}"), + Self::U64(v) => write!(f, "{v}"), + Self::F32(v) => write!(f, "{v}"), + Self::Vector2(v) => write_tuple(f, &v.to_array()), + Self::Vector3(v) => write_tuple(f, &v.to_array()), + Self::Vector4(v) => write_tuple(f, &v.to_array()), + Self::Matrix44(v) => write_tuple(f, &v.transpose().to_cols_array()), + Self::Color(c) => write_tuple(f, &[c.r, c.g, c.b, c.a]), + Self::String(s) => write_json_string(f, s), + Self::Hash(h) | Self::Link(h) => write!(f, "{h:08x}"), + Self::File(h) => write!(f, "{h:016x}"), + } + } +} + +/// `(a, b, c)`. +fn write_tuple(f: &mut fmt::Formatter<'_>, items: &[T]) -> fmt::Result { + f.write_str("(")?; + for (i, item) in items.iter().enumerate() { + if i > 0 { + f.write_str(", ")?; + } + write!(f, "{item}")?; + } + f.write_str(")") +} + +/// A JSON string literal. `"`, `\` and the control characters are escaped, as `serde_json` +/// writes them. +fn write_json_string(f: &mut fmt::Formatter<'_>, s: &str) -> fmt::Result { + f.write_str("\"")?; + for c in s.chars() { + match c { + '"' => f.write_str("\\\"")?, + '\\' => f.write_str("\\\\")?, + '\n' => f.write_str("\\n")?, + '\r' => f.write_str("\\r")?, + '\t' => f.write_str("\\t")?, + '\u{8}' => f.write_str("\\b")?, + '\u{c}' => f.write_str("\\f")?, + c if (c as u32) < 0x20 => write!(f, "\\u{:04x}", c as u32)?, + c => write!(f, "{c}")?, + } + } + f.write_str("\"") +} diff --git a/crates/ltk_meta/src/walk/view.rs b/crates/ltk_meta/src/walk/view.rs new file mode 100644 index 00000000..22346600 --- /dev/null +++ b/crates/ltk_meta/src/walk/view.rs @@ -0,0 +1,290 @@ +//! The streaming view as the walk sees it: [`ValueView`] and [`StructView`]. + +use std::fmt; + +use ltk_hash::BinHash; + +use super::{ + tree::{sealed::Sealed, Child, Leaf, TreeKind as _, TreeNode, TreeValue}, + Error, +}; +use crate::{ + property::{values, Kind, NoMeta}, + stream::{ContainerItems, MapEntries, Properties, StructView, ValueView}, + PropertyValueEnum, +}; + +impl Sealed for ValueView<'_, M> {} +impl Sealed for StructView<'_, M> {} + +/// The properties of a [`StructView`], in file order, each header decoded as it is reached. +#[must_use = "iterators are lazy and do nothing unless consumed"] +pub struct ViewProperties<'a, M = NoMeta> { + inner: Properties<'a, M>, +} + +impl<'a, M> Iterator for ViewProperties<'a, M> { + type Item = Result<(BinHash, ValueView<'a, M>), Error>; + + fn next(&mut self) -> Option { + let property = self.inner.next()?; + Some(property.and_then(|p| Ok((p.name_hash(), p.value_view()?)))) + } + + fn size_hint(&self) -> (usize, Option) { + self.inner.size_hint() + } +} + +impl std::iter::FusedIterator for ViewProperties<'_, M> {} + +impl fmt::Debug for ViewProperties<'_, M> { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("ViewProperties") + .field("inner", &self.inner) + .finish() + } +} + +impl<'a, M: Default> TreeNode<'a> for StructView<'a, M> { + type Value = ValueView<'a, M>; + type Properties = ViewProperties<'a, M>; + + fn class_hash(&self) -> BinHash { + StructView::class_hash(self) + } + + fn properties(&self) -> Self::Properties { + ViewProperties { + inner: StructView::properties(self), + } + } + + fn property(&self, field: BinHash) -> Result, Error> { + StructView::property(self, field)? + .map(|p| p.value_view()) + .transpose() + } + + fn to_struct(&self) -> Result { + Ok(values::Struct { + class_hash: StructView::class_hash(self), + properties: TreeNode::properties(self) + .map(|property| { + let (field, value) = property?; + Ok((field, value.to_value()?)) + }) + .collect::>()?, + meta: NoMeta, + }) + } +} + +/// The values inside a viewed container, optional or map, each decoded as it is reached. +#[must_use = "iterators are lazy and do nothing unless consumed"] +pub struct ViewChildren<'a, M = NoMeta> { + inner: ViewChildrenInner<'a, M>, +} + +enum ViewChildrenInner<'a, M> { + Items { + items: ContainerItems<'a, M>, + index: usize, + }, + Optional(Option>), + Entries(MapEntries<'a, M>), + Empty, +} + +impl<'a, M> Iterator for ViewChildren<'a, M> { + type Item = Result<(Child>, ValueView<'a, M>), Error>; + + fn next(&mut self) -> Option { + match &mut self.inner { + ViewChildrenInner::Items { items, index } => { + let item = items.next()?; + let step = Child::Index(*index); + *index += 1; + Some(item.map(|value| (step, value))) + } + ViewChildrenInner::Optional(value) => value.take().map(|v| Ok((Child::Index(0), v))), + ViewChildrenInner::Entries(entries) => { + let entry = entries.next()?; + Some(entry.map(|(key, value)| (Child::Key(key), value))) + } + ViewChildrenInner::Empty => None, + } + } + + fn size_hint(&self) -> (usize, Option) { + match &self.inner { + ViewChildrenInner::Items { items, .. } => items.size_hint(), + ViewChildrenInner::Optional(value) => { + let n = usize::from(value.is_some()); + (n, Some(n)) + } + ViewChildrenInner::Entries(entries) => entries.size_hint(), + ViewChildrenInner::Empty => (0, Some(0)), + } + } +} + +impl std::iter::FusedIterator for ViewChildren<'_, M> {} + +impl fmt::Debug for ViewChildren<'_, M> { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let (kind, remaining) = match &self.inner { + ViewChildrenInner::Items { items, .. } => ("items", items.size_hint().0), + ViewChildrenInner::Optional(value) => ("optional", usize::from(value.is_some())), + ViewChildrenInner::Entries(entries) => ("entries", entries.size_hint().0), + ViewChildrenInner::Empty => ("empty", 0), + }; + f.debug_struct("ViewChildren") + .field("kind", &kind) + .field("remaining", &remaining) + .finish() + } +} + +impl<'a, M: Default> TreeValue<'a> for ValueView<'a, M> { + type Node = StructView<'a, M>; + type Children = ViewChildren<'a, M>; + + fn kind(&self) -> Kind { + ValueView::kind(self) + } + + fn holds_node(&self) -> Result { + Ok(match self { + Self::Struct(s) | Self::Embedded(s) => *s.class_hash() != 0, + Self::Container(c) | Self::UnorderedContainer(c) => c.item_kind().is_node(), + Self::Optional(o) => o.item_kind().is_node(), + Self::Map(m) => m.value_kind().is_node(), + _ => false, + }) + } + + fn as_node(&self) -> Result, Error> { + Ok(match self { + Self::Struct(s) | Self::Embedded(s) if *s.class_hash() != 0 => Some(*s), + _ => None, + }) + } + + fn children(&self) -> Result { + let inner = match self { + Self::Container(c) | Self::UnorderedContainer(c) => ViewChildrenInner::Items { + items: c.iter(), + index: 0, + }, + Self::Optional(o) => ViewChildrenInner::Optional(o.get()?), + Self::Map(m) => ViewChildrenInner::Entries(m.iter()), + _ => ViewChildrenInner::Empty, + }; + Ok(ViewChildren { inner }) + } + + fn leaf(&self) -> Result>, Error> { + Ok(Some(match *self { + Self::None => Leaf::None, + Self::Bool(v) => Leaf::Bool(v), + Self::I8(v) => Leaf::I8(v), + Self::U8(v) => Leaf::U8(v), + Self::I16(v) => Leaf::I16(v), + Self::U16(v) => Leaf::U16(v), + Self::I32(v) => Leaf::I32(v), + Self::U32(v) => Leaf::U32(v), + Self::I64(v) => Leaf::I64(v), + Self::U64(v) => Leaf::U64(v), + Self::F32(v) => Leaf::F32(v), + Self::Vector2(v) => Leaf::Vector2(v), + Self::Vector3(v) => Leaf::Vector3(v), + Self::Vector4(v) => Leaf::Vector4(v), + Self::Matrix44(v) => Leaf::Matrix44(v), + Self::Color(v) => Leaf::Color(v), + Self::String(v) => Leaf::String(v), + Self::Hash(v) => Leaf::Hash(v), + Self::WadChunkLink(v) => Leaf::File(v), + Self::ObjectLink(v) => Leaf::Link(v), + Self::BitBool(v) => Leaf::Flag(v), + Self::Container(_) + | Self::UnorderedContainer(_) + | Self::Optional(_) + | Self::Map(_) + | Self::Struct(_) + | Self::Embedded(_) => return Ok(None), + })) + } + + fn to_value(&self) -> Result { + use PropertyValueEnum as P; + macro_rules! prim { + ($ty:ident, $v:expr) => { + P::$ty(values::$ty::new_with_meta($v, NoMeta)) + }; + } + Ok(match *self { + Self::None => P::None(values::None { meta: NoMeta }), + Self::Bool(v) => prim!(Bool, v), + Self::I8(v) => prim!(I8, v), + Self::U8(v) => prim!(U8, v), + Self::I16(v) => prim!(I16, v), + Self::U16(v) => prim!(U16, v), + Self::I32(v) => prim!(I32, v), + Self::U32(v) => prim!(U32, v), + Self::I64(v) => prim!(I64, v), + Self::U64(v) => prim!(U64, v), + Self::F32(v) => prim!(F32, v), + Self::Vector2(v) => prim!(Vector2, v), + Self::Vector3(v) => prim!(Vector3, v), + Self::Vector4(v) => prim!(Vector4, v), + Self::Matrix44(v) => prim!(Matrix44, v), + Self::Color(v) => prim!(Color, v), + Self::String(v) => prim!(String, v.to_owned()), + Self::Hash(v) => prim!(Hash, v), + Self::WadChunkLink(v) => prim!(WadChunkLink, v), + Self::ObjectLink(v) => prim!(ObjectLink, v), + Self::BitBool(v) => prim!(BitBool, v), + Self::Struct(s) => P::Struct(struct_of(s)?), + Self::Embedded(s) => P::Embedded(values::Embedded(struct_of(s)?)), + Self::Container(c) => P::Container(container_of(c.item_kind(), c.iter())?), + Self::UnorderedContainer(c) => P::UnorderedContainer(values::UnorderedContainer( + container_of(c.item_kind(), c.iter())?, + )), + Self::Optional(o) => P::Optional(values::Optional::new( + o.item_kind(), + o.get()?.map(|v| v.to_value()).transpose()?, + )?), + Self::Map(m) => P::Map(values::Map::new( + m.key_kind(), + m.value_kind(), + m.iter() + .map(|entry| { + let (k, v) = entry?; + Ok((k.to_value()?, v.to_value()?)) + }) + .collect::>()?, + )?), + }) + } +} + +/// A null pointer stays a null pointer: class 0 and no properties. +fn struct_of(view: StructView<'_, M>) -> Result { + if *view.class_hash() == 0 { + return Ok(values::Struct::default()); + } + view.to_struct() +} + +fn container_of( + item_kind: Kind, + items: ContainerItems<'_, M>, +) -> Result { + values::Container::new( + item_kind, + items + .map(|item| item?.to_value()) + .collect::>()?, + ) +} diff --git a/crates/ltk_meta/tests/corpus.rs b/crates/ltk_meta/tests/corpus.rs index d54649d7..1164e141 100644 --- a/crates/ltk_meta/tests/corpus.rs +++ b/crates/ltk_meta/tests/corpus.rs @@ -27,7 +27,8 @@ use ltk_meta::{ concrete::BinStream, path::{PatchError, ResolveErrorKind, ValueShape}, traits::PropertyExt as _, - Bin, BinKind, BinObject, BinOverride, + walk::{Node, TreeValue, Visit, Visitor}, + Bin, BinKind, BinObject, BinOverride, Error, PropertyValueEnum, }; use ltk_wad::Wad; @@ -523,3 +524,125 @@ fn every_shipped_prop_streams_the_same_object_set() { println!("{counts}"); assert!(counts.prop_chunks > 0, "no PROP chunks in {game_dir}"); } + +/// `(object, class, trail)` per node, in visit order. +#[derive(Default)] +struct Visits(Vec<(u32, u32, String)>); + +impl<'a, V: TreeValue<'a>> Visitor<'a, V> for Visits { + type Error = Error; + + fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + self.0.push(( + *node.object_hash(), + *node.class_hash(), + node.trail().to_string(), + )); + Ok(Visit::Continue) + } +} + +/// The nodes under `value`, by a recursion independent of the walk: every `Struct` and +/// `Embedded` with a non-zero class, wherever it sits. +fn count_nodes(value: &PropertyValueEnum) -> usize { + use PropertyValueEnum as P; + let count_struct = |s: <k_meta::property::values::Struct| { + if *s.class_hash == 0 { + 0 + } else { + 1 + s.properties.values().map(count_nodes).sum::() + } + }; + match value { + P::Struct(s) => count_struct(s), + P::Embedded(e) => count_struct(&e.0), + P::Container(c) => c.items().iter().map(count_nodes).sum(), + P::UnorderedContainer(c) => c.0.items().iter().map(count_nodes).sum(), + P::Optional(o) => o.value().map_or(0, count_nodes), + P::Map(m) => m.entries().iter().map(|(_, v)| count_nodes(v)).sum(), + _ => 0, + } +} + +#[derive(Default)] +struct WalkCounts { + wads: usize, + prop_chunks: usize, + objects: usize, + nodes: usize, +} + +impl fmt::Display for WalkCounts { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + writeln!(f, "{} wad archives", self.wads)?; + writeln!(f, "{} PROP chunks walked twice", self.prop_chunks)?; + writeln!(f, "{} objects, {} nodes", self.objects, self.nodes) + } +} + +/// The streaming walk against the owned walk and an independent count, for one `PROP` chunk. +fn check_walk_parity(wad_path: &Path, data: &[u8], counts: &mut WalkCounts) { + let context = || format!("{}", wad_path.display()); + + let eager = Bin::from_reader(&mut Cursor::new(data)) + .unwrap_or_else(|e| panic!("{}: the eager parse failed: {e}", context())); + let mut owned = Visits::default(); + eager + .walk(&mut owned) + .unwrap_or_else(|e| panic!("{}: the owned walk failed: {e}", context())); + + let mut stream = BinStream::mount(Cursor::new(data)) + .unwrap_or_else(|e| panic!("{}: the stream did not mount: {e}", context())); + let mut streamed = Visits::default(); + stream + .walk(&mut streamed) + .unwrap_or_else(|e: Error| panic!("{}: the streaming walk failed: {e}", context())); + + assert_eq!(owned.0, streamed.0, "{}", context()); + + let expected: usize = eager + .objects + .values() + .map(|object| 1 + object.properties.values().map(count_nodes).sum::()) + .sum(); + assert_eq!(owned.0.len(), expected, "{}", context()); + + counts.prop_chunks += 1; + counts.objects += eager.objects.len(); + counts.nodes += expected; +} + +#[test] +#[ignore = "needs an installed client; set LTK_LOL_GAME_DIR"] +fn every_shipped_prop_walks_the_same_over_both_trees() { + let Ok(game_dir) = std::env::var(GAME_DIR) else { + panic!("set {GAME_DIR} to the client's Game directory"); + }; + + let mut wad_files = Vec::new(); + wad_paths(Path::new(&game_dir), &mut wad_files); + wad_files.sort(); + assert!(!wad_files.is_empty(), "no .wad.client under {game_dir}"); + + let mut counts = WalkCounts::default(); + for wad_path in &wad_files { + counts.wads += 1; + + let source = File::open(wad_path).expect("the wad opens"); + let mut wad = Wad::mount(source).expect("the wad mounts"); + let chunks: Vec<_> = wad.chunks().as_slice().to_vec(); + + for chunk in &chunks { + let Ok(data) = wad.load_chunk_decompressed(chunk) else { + continue; + }; + if BinKind::identify_from_bytes(&data) != Some(BinKind::Prop) { + continue; + } + check_walk_parity(wad_path, &data, &mut counts); + } + } + + println!("{counts}"); + assert!(counts.prop_chunks > 0, "no PROP chunks in {game_dir}"); +} diff --git a/docs/LTK_GUIDE.md b/docs/LTK_GUIDE.md index a2c6f89e..6d76df0d 100644 --- a/docs/LTK_GUIDE.md +++ b/docs/LTK_GUIDE.md @@ -409,6 +409,73 @@ if report.is_clean() { `crates/ltk_meta/tests/corpus.rs` runs all of this over an installed client; it is `#[ignore]`d unless `LTK_LOL_GAME_DIR` is set. +**Walking a bin**: `ltk_meta::walk` is one read-only traversal over every node of an object, +driven by a `Visitor` that is generic over the tree. The walk visits every node of an object once, in pre-order and file order, and asks the visitor before entering each property. The visitor is generic over the tree: the same `Census` runs over an owned `Bin` and over a `BinStream`, where nothing is materialised. The design is +`docs/design/value-walk.md`. + +```rust +use ltk_hash::BinHash; +use ltk_meta::{ + walk::{Node, TreeValue, Visit, Visitor}, + Error, +}; + +/// Counts nodes and records the address of every `Struct` of one class. +#[derive(Default)] +struct Census { + nodes: usize, + hits: Vec<(BinHash, String)>, +} + +impl<'a, V: TreeValue<'a>> Visitor<'a, V> for Census { + type Error = Error; + + fn enter_node(&mut self, node: &Node<'_, 'a, V>) -> Result { + self.nodes += 1; + if *node.class_hash() == 0x1e6b_a0c4 { + self.hits.push((node.object_hash(), node.trail().to_string())); + } + Ok(Visit::Continue) + } +} + +let mut census = Census::default(); +bin.walk(&mut census)?; +``` + +**In parallel.** The walk over one object is sequential by contract: one visitor, pre-order, `Stop` and `Skip` as ordered decisions. Objects are independent of one another, and every view, node and trail type is `Send`. A sweep parallelises across objects with one visitor instance per worker and a reduce at the end. Nothing in the crate schedules this; the split is the caller's. + +```rust +let objects: Vec<_> = bin.objects.values().collect(); +let workers = std::thread::available_parallelism().map_or(1, |n| n.get()); +let per_worker = objects.len().div_ceil(workers).max(1); + +let counted = std::thread::scope(|scope| { + let workers: Vec<_> = objects + .chunks(per_worker) + .map(|chunk| { + scope.spawn(move || { + let mut census = Census::default(); + for object in chunk { + object.walk(&mut census)?; + } + Ok::<_, Error>(census) + }) + }) + .collect(); + + let mut all = Census::default(); + for worker in workers { + let census = worker.join().expect("a worker panicked")?; + all.nodes += census.nodes; + all.hits.extend(census.hits); + } + Ok::<_, Error>(all) +})?; +``` + +Across many files the same shape applies one level up: one task per file, each mounting its own `BinStream` and walking it sequentially. The per-object walk is microseconds; decompression and I/O are where a sweep spends its time. + **Path/Name Hashing**: Object paths and property names are stored as FNV-1a hashes. Use community hash databases or `ltk_hash::fnv1a::hash_lower()` to compute hashes. --- diff --git a/docs/design/value-walk.md b/docs/design/value-walk.md index af7caf9a..9df696a6 100644 --- a/docs/design/value-walk.md +++ b/docs/design/value-walk.md @@ -12,7 +12,8 @@ is the bug and gets edited. Two things it does not hold: - **Why an option was chosen over the alternatives it beat** - ADR-0005, ADR-0012, ADR-0013 and ADR-0014, cited from the rules in [section 8](#s8). -Designed and not yet built, tracked as #219 (`ValuePath`) and #225 (the walk). +`ValuePath`, `MapKey`, `FieldNames` and the three methods that produce them - +`TreeValue::map_key`, `Trail::to_value_path`, `Node::value_path` - are #219. ## 1. Summary @@ -94,21 +95,23 @@ Every term this document uses in a specific sense. ## 3. The tree the walk sees +The walk is written once, against two sealed traits, and the owned tree and the view each +implement them. A visitor is generic over the value type and never names either. A third +sealed trait carries the one question the walk asks of a `Kind`. All three live in +`ltk_meta::walk`; `Kind` and `PropertyValueEnum` carry no walk vocabulary of their own (W1). + ```rust -impl Kind { +/// The one question the walk asks of a `Kind`. Sealed: implemented for `Kind` only. +pub trait TreeKind: Copy + sealed::Sealed { /// Whether a value of this kind is a node: `Struct` or `Embedded`. /// /// The other question - which kinds a value model treats as leaves - is - /// [`Kind::is_primitive`], and the two are not complements: `ObjectLink` and `BitBool` + /// `Kind::is_primitive`, and the two are not complements: `ObjectLink` and `BitBool` /// are neither primitive nor a node. - pub fn is_node(self) -> bool; + fn is_node(self) -> bool; } -``` - -The walk is written once, against two sealed traits, and the owned tree and the view each -implement them. A visitor is generic over the value type and never names either. +impl TreeKind for Kind {} -```rust /// A value the walk can cross. Sealed: implemented for `&'a PropertyValueEnum` and for /// `ValueView<'a, M>`, and by nothing else. pub trait TreeValue<'a>: Copy + sealed::Sealed { @@ -122,8 +125,8 @@ pub trait TreeValue<'a>: Copy + sealed::Sealed { /// Whether entering this value can reach a node. /// /// True for a `Struct` or `Embedded` whose class hash is not 0, and for a container, - /// optional or map whose item kind [`Kind::is_node`]. An empty optional or container of a - /// node kind answers true: it *can* hold one, and entering it costs nothing. + /// optional or map whose item kind [`TreeKind::is_node`]. An empty optional or container + /// of a node kind answers true: it *can* hold one, and entering it costs nothing. /// /// # Errors /// @@ -153,11 +156,13 @@ pub trait TreeValue<'a>: Copy + sealed::Sealed { } /// A node the walk can visit: a class and properties. Sealed: implemented for the owned -/// tree's node, for `StructView<'a, M>`, and for `ObjectView<'a, M>` as a root. +/// tree's node and for `StructView<'a, M>`. An object's root is walked as a `StructView` +/// over the same bytes; `ObjectView` implements nothing, its `Value` would have to name it +/// as `Node` and `ValueView::Node` is `StructView`. pub trait TreeNode<'a>: Copy + sealed::Sealed { type Value: TreeValue<'a, Node = Self>; - /// The properties in file order. Items are `Result` because a view's kind byte can fail to - /// decode; the owned tree never fails. + /// The properties in file order. A view's kind byte can fail to decode. Items are + /// `Result`; the owned tree never fails. type Properties: Iterator>; fn class_hash(&self) -> BinHash; @@ -181,7 +186,9 @@ pub enum Child { /// A leaf, decoded and borrowed. The client's names for the tags, not the wire enum's: /// `File` is `Kind::WadChunkLink`, `Link` is `Kind::ObjectLink`, `Flag` is `Kind::BitBool`. +/// Non-exhaustive: the set of leaf kinds is the game's (W22). #[derive(Clone, Copy, Debug, PartialEq)] +#[non_exhaustive] pub enum Leaf<'a> { None, Bool(bool), @@ -196,20 +203,55 @@ pub enum Leaf<'a> { Flag(bool), } -impl<'a, M> TreeValue<'a> for &'a PropertyValueEnum { type Node = OwnedNode<'a, M>; /* ... */ } -impl<'a, M: Default> TreeValue<'a> for ValueView<'a, M> { type Node = StructView<'a, M>; /* ... */ } +impl Leaf<'_> { + pub fn kind(&self) -> Kind; +} + +impl<'a, M> TreeValue<'a> for &'a PropertyValueEnum { + type Node = OwnedNode<'a, M>; + type Children = OwnedChildren<'a, M>; + /* ... */ +} +impl<'a, M: Default> TreeValue<'a> for ValueView<'a, M> { + type Node = StructView<'a, M>; + type Children = ViewChildren<'a, M>; + /* ... */ +} /// The owned tree's node: a class hash and a borrowed property map. `BinObject` and -/// `values::Struct` both view as one. +/// `values::Struct` both view as one, through `From`. #[derive(Clone, Copy, Debug)] pub struct OwnedNode<'a, M = NoMeta> { /* class_hash, &'a IndexMap> */ } -impl<'a, M> TreeNode<'a> for OwnedNode<'a, M> { type Value = &'a PropertyValueEnum; /* ... */ } -impl<'a, M: Default> TreeNode<'a> for StructView<'a, M> { type Value = ValueView<'a, M>; /* ... */ } -impl<'a, M: Default> TreeNode<'a> for ObjectView<'a, M> { type Value = ValueView<'a, M>; /* ... */ } +impl<'a, M> OwnedNode<'a, M> { + pub fn new(class_hash: BinHash, properties: &'a IndexMap>) -> Self; +} +impl<'a, M> From<&'a BinObject> for OwnedNode<'a, M> {} +impl<'a, M> From<&'a values::Struct> for OwnedNode<'a, M> {} + +impl<'a, M> TreeNode<'a> for OwnedNode<'a, M> { + type Value = &'a PropertyValueEnum; + type Properties = OwnedProperties<'a, M>; + /* ... */ +} +impl<'a, M: Default> TreeNode<'a> for StructView<'a, M> { + type Value = ValueView<'a, M>; + type Properties = ViewProperties<'a, M>; + /* ... */ +} + +/// The iterators behind the associated types. Each is `FusedIterator`; the owned pair is +/// `ExactSizeIterator` too. +pub struct OwnedProperties<'a, M = NoMeta> { /* ... */ } +pub struct OwnedChildren<'a, M = NoMeta> { /* ... */ } +pub struct ViewProperties<'a, M = NoMeta> { /* ... */ } +pub struct ViewChildren<'a, M = NoMeta> { /* ... */ } ``` -`PropertyValueEnum::holds_node` is also exposed directly, infallible, and `Kind::is_primitive` -plays no part in any of this (W1). `Leaf` is the one place the crate names the tags as the +`to_value` and `to_struct` return a `NoMeta` tree whichever `M` the source carries: over the +owned tree every metadata slot is reset, over a view there was none. A null pointer stays a +`Struct` with class 0 under either. + +`Kind::is_primitive` plays no part in any of this (W1). `Leaf` is the one place the crate names the tags as the client does (W19): a visitor reads a texture path as `Leaf::File`, whatever `Kind` calls it. ## 4. `ValuePath` @@ -396,7 +438,8 @@ field step nowhere in the text. A `ValuePath` with no steps renders as the empty Hex is lowercase, zero-padded to the hash's width: eight digits for a `BinHash`, sixteen for a `WadHash`. A string key is written as a JSON string, escaped as `serde_json` would write it. An `F32` key is written in Rust's shortest round-trip form. A vector or colour key is its components -in parentheses, comma separated; it is text for a human and nothing parses it. +in parentheses, comma separated, and a matrix key is its sixteen components row by row, as the +wire holds them; it is text for a human and nothing parses it. `to_property_path` writes a `Hash` key as its raw value in decimal rather than as a name, even when the table has one: a plaintext hashes back to the same value, but it is the value that is @@ -522,8 +565,9 @@ pub trait Visitor<'a, V: TreeValue<'a>> { -> Result { Ok(Visit::Continue) } - /// Called once for every property that was descended - after its nodes, after a `Skip`, - /// and while unwinding for a `Stop`. Not called for a leaf. Never after an `Abort`. + /// Called once for every property that holds a node and was entered - after its nodes, + /// after a `Skip`, and while unwinding for a `Stop`. Not called for a leaf. Never after + /// an `Abort`. fn exit_property(&mut self, field: BinHash, value: V, node: &Node<'_, 'a, V>) -> Result { Ok(Visit::Continue) @@ -624,17 +668,21 @@ impl<'a, M: Default> ObjectView<'a, M> { } impl ObjectStream<'_, R, M> { /// `view()?` then `walk`. - pub fn walk(&mut self, visitor: &mut W) -> Result - where W: for<'a> Visitor<'a, ValueView<'a, M>>; + pub fn walk(&mut self, visitor: &mut W) -> Result + where E: From, W: for<'a> Visitor<'a, ValueView<'a, M>, Error = E>; } impl BinStream { /// Walks every object in file order, one buffered object at a time: `objects()` and /// `walk` on each. Holds one object's bytes at any moment and nothing of the tree. - pub fn walk(&mut self, visitor: &mut W) -> Result - where W: for<'a> Visitor<'a, ValueView<'a, M>>; + pub fn walk(&mut self, visitor: &mut W) -> Result + where E: From, W: for<'a> Visitor<'a, ValueView<'a, M>, Error = E>; } ``` +The stream entry points name the visitor's error as their own type parameter `E`. A bound +quantified over every `'a` has no single `W::Error` to project; `Error = E` pins it, and a +caller never spells it. + A visitor written against `TreeValue` and `TreeNode` runs over either tree unchanged. The manager's rules are written once and run over `BinStream::walk` in the pass and over `BinObject::walk` when a repair verifies the tree it just edited in memory @@ -694,7 +742,8 @@ For one object: `Visit::Skip` from `enter_node` runs rule 5 without rule 2; from `exit_property` it runs rule 5 without the node's remaining properties; from `exit_node` it ends the enclosing property's remaining items and runs rule 4. `Stop` runs every pending rule 4 and rule 5 innermost first, -then returns `Stopped`; `Abort` returns `Aborted` at once. +then returns `Stopped`; a `Stop` from `enter_property` on a value that holds a node counts that +property as pending, and a `Stop` on a leaf does not. `Abort` returns `Aborted` at once. So a visitor sees nodes in pre-order, in file order, each exactly once, and every push is popped before the walk returns. With every callback at its default the walk visits every node in the @@ -784,7 +833,7 @@ rules append. | ID | Rule | Instead of | Why | Spec | | -- | ---- | ---------- | --- | ---- | -| W1 | The walk's prune is `holds_node`, built on `Kind::is_node` (`Struct`, `Embedded`), asked of the tree before the visitor. `Kind::is_primitive` plays no part. | Entering everything `is_primitive` does not cover. | `ObjectLink` and `BitBool` are neither primitive nor a node, so the complement of `is_primitive` enters containers that hold nothing; and a consumer should not have to know which set `is_primitive` is. | [section 3](#s3), [section 5.1](#s5.1) | +| W1 | The walk's prune is `TreeValue::holds_node`, built on `TreeKind::is_node` (`Struct`, `Embedded`), asked of the tree before the visitor. Both are traits in `walk`; `Kind` and `PropertyValueEnum` carry no inherent walk predicate. `Kind::is_primitive` plays no part. | Inherent `Kind::is_node` and `PropertyValueEnum::holds_node`; or entering everything `is_primitive` does not cover. | "Node" is the walk's vocabulary, defined in this document, and a method on `Kind` shows the word to every reader of the crate with nothing beside it to say what it means. `ObjectLink` and `BitBool` are neither primitive nor a node, so the complement of `is_primitive` enters containers that hold nothing. | [section 3](#s3), [section 5.1](#s5.1) | | W2 | A `Struct` or `Embedded` with class 0 is not a node and is not entered. | Visiting it as a node with class 0. | It is the client's null pointer, has no properties, and the resolver already treats it as one (`NullPointer`). A visitor keyed on class would otherwise see a class no meta class dump has. | [section 5.1](#s5.1) | | W3 | `ltk_meta` owns one single-visitor walk with a trail; scheduling several visitors over one walk, and what each does with a node, is the consumer's. | A multi-visitor walk with per-visitor pruning in the crate; or only a predicate and a step enum. | The single-visitor descent is identical for every consumer and is what merge and diff need; the active-set policy is one consumer's and would pin its shape under semver. | [section 5](#s5); ADR-0013 | | W4 | A `ValuePath` keeps a class context beside its steps - the class of the node each field was read on - and `Step::Field` carries the field hash alone. | `Field { class, field }`, or no class anywhere. | Naming a field takes the class it is on, and every table a consumer holds is keyed by class; keeping it beside the steps leaves `Step` the address and the context free to grow. | [section 4.1](#s4.1); ADR-0012 | @@ -805,3 +854,4 @@ rules append. | W19 | `Leaf` and `MapKey` name the tags as the client does: `File`, `Link`, `Flag`. `Kind` keeps `WadChunkLink`, `ObjectLink`, `BitBool`. | Reusing `Kind`'s names in the new types. | The new surface is what a consumer writes against and should carry the vocabulary the reversing notes and the meta class dumps use; renaming `Kind` is a break for every existing caller and is its own decision. | [section 3](#s3) | | W20 | The walk runs over two sealed traits, `TreeNode` and `TreeValue`, implemented by the owned tree and by the views; a visitor is generic over the value type. | A walk over `PropertyValueEnum` only, with `read()` per streamed object; or a walk over the views only. | One traversal, one visitor, both sources; the stream pass materialises nothing and the repair's in-memory check uses the same rule. Sealed, because a third tree would have to be this crate's. | [section 3](#s3), [section 5](#s5); ADR-0014 | | W21 | The visitor has `ltk_ritobin`'s CST visitor shape: symmetric enter and exit, a `Visit` answer of `Abort`, `Stop`, `Skip` or `Continue`, a `WalkOutcome`. `Skip` from `enter_property` prunes that value, where the CST's token `Skip` prunes the rest of the node. | A `bool` prune and no early exit. | One visitor idiom across the workspace; and a property, unlike a token, has a subtree of its own to prune. | [section 5](#s5) | +| W22 | `Leaf` is `#[non_exhaustive]`; `Visit`, `WalkOutcome`, `Child`, `TrailStep` and `Step` are exhaustive. | Marking every new public enum, or none. | The leaf kinds are the game's to extend, and `WadChunkLink` was added once; a consumer's wildcard arm is the price of a minor release carrying the next one. The other enums are this crate's own, and a consumer matching a new `Visit` answer or step kind is told by the compiler. | [section 3](#s3) |