Shared data structures and algorithms crate by vineethk · Pull Request #19388 · aptos-labs/aptos-core

vineethk · 2026-04-10T01:08:29Z

Description

We implement and expose a fast, non-cryptographic, non-hash-DoS-resistant hashmap and hashset, without exposing the arbitrary iteration over keys/key-value pairs. This is what we want in a number of places in our codebase, but we end up using the slower default hashmap, which is non-deterministic, or various other hashmap implementations.

Currently, this implementation wraps rustc_hash's FxHash[Map|Set] , used by the rust compiler.

How Has This Been Tested?

added some simple tests

Type of Change

New feature

Note

Medium Risk
Introduces new collection types backed by non-DoS-resistant hashing; if adopted in code paths that accept untrusted keys, it could create performance-attack risk despite being unused in this PR.

Overview
Adds a new workspace crate, third_party/move/shared-dsa, exposing UnorderedMap and UnorderedSet wrappers around rustc_hash’s FxHashMap/FxHashSet while intentionally not exposing iteration APIs to avoid reliance on non-deterministic ordering.

Wires the crate into the workspace (Cargo.toml) and lockfile, adds a workspace dependency on rustc-hash, and includes unit tests covering basic map/set operations and trait behavior.

^{Reviewed by Cursor Bugbot for commit 13019da. Bugbot is set up for automated code reviews on this repo. Configure here.}

vineethk · 2026-04-10T01:08:42Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

Copilot

Pull request overview

Adds a new shared-dsa crate to the Move workspace that provides fast, non-cryptographic hash-based collections (UnorderedMap / UnorderedSet) intended to discourage reliance on nondeterministic iteration order.

Changes:

Introduces UnorderedMap and UnorderedSet wrappers around rustc_hash::FxHashMap/FxHashSet, exposing common map/set operations while avoiding iteration APIs.
Adds basic unit tests for the new collections.
Registers the new crate in the workspace and adds the rustc-hash workspace dependency.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
third_party/move/shared-dsa/src/unordered_set.rs	New `UnorderedSet` wrapper and tests
third_party/move/shared-dsa/src/unordered_map.rs	New `UnorderedMap` wrapper (incl. `entry`) and tests
third_party/move/shared-dsa/src/lib.rs	Crate module wiring + public re-exports
third_party/move/shared-dsa/Cargo.toml	New crate manifest
Cargo.toml	Adds crate to workspace + adds `rustc-hash` workspace dependency
Cargo.lock	Locks new `shared-dsa` crate dependency graph

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

calintat

Nice!

…r use in other crates.

github-actions · 2026-04-13T22:59:59Z

✅ Forge suite `realistic_env_max_load` success on `13019daf01024ad4491ecc300413b5b1193aa032`

two traffics test: inner traffic : committed: 14392.02 txn/s, latency: 1251.10 ms, (p50: 1200 ms, p70: 1300, p90: 1500 ms, p99: 1900 ms), latency samples: 5367820
two traffics test : committed: 100.00 txn/s, latency: 657.27 ms, (p50: 600 ms, p70: 700, p90: 800 ms, p99: 1200 ms), latency samples: 1700
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 0.519, avg: 0.456", "ConsensusProposalToOrdered: max: 0.116, avg: 0.110", "ConsensusOrderedToCommit: max: 0.191, avg: 0.183", "ConsensusProposalToCommit: max: 0.306, avg: 0.293"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.95s no progress at version 11242 (avg 0.06s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.31s no progress at version 2599347 (avg 0.31s) [limit 16].
Test Ok

github-actions · 2026-04-13T23:00:53Z

✅ Forge suite `compat` success on `1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb` ==> `13019daf01024ad4491ecc300413b5b1193aa032`

Compatibility test results for 1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb ==> 13019daf01024ad4491ecc300413b5b1193aa032 (PR)
1. Check liveness of validators at old version: 1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb
compatibility::simple-validator-upgrade::liveness-check : committed: 13255.93 txn/s, latency: 2617.48 ms, (p50: 2700 ms, p70: 2900, p90: 3300 ms, p99: 3700 ms), latency samples: 434760
2. Upgrading first Validator to new version: 13019daf01024ad4491ecc300413b5b1193aa032
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6234.69 txn/s, latency: 5413.94 ms, (p50: 5900 ms, p70: 6000, p90: 6100 ms, p99: 6300 ms), latency samples: 218940
3. Upgrading rest of first batch to new version: 13019daf01024ad4491ecc300413b5b1193aa032
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6331.90 txn/s, latency: 5373.17 ms, (p50: 5900 ms, p70: 5900, p90: 6200 ms, p99: 6300 ms), latency samples: 220440
4. upgrading second batch to new version: 13019daf01024ad4491ecc300413b5b1193aa032
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 10576.48 txn/s, latency: 3099.74 ms, (p50: 2900 ms, p70: 3800, p90: 4300 ms, p99: 4600 ms), latency samples: 351440
5. check swarm health
Compatibility test for 1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb ==> 13019daf01024ad4491ecc300413b5b1193aa032 passed
Test Ok

github-actions · 2026-04-13T23:04:40Z

✅ Forge suite `framework_upgrade` success on `1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb` ==> `13019daf01024ad4491ecc300413b5b1193aa032`

Compatibility test results for 1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb ==> 13019daf01024ad4491ecc300413b5b1193aa032 (PR)
Upgrade the nodes to version: 13019daf01024ad4491ecc300413b5b1193aa032
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 2434.67 txn/s, submitted: 2442.76 txn/s, failed submission: 8.10 txn/s, expired: 8.10 txn/s, latency: 1192.68 ms, (p50: 1200 ms, p70: 1200, p90: 1500 ms, p99: 2100 ms), latency samples: 222520
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 2255.88 txn/s, submitted: 2263.80 txn/s, failed submission: 7.92 txn/s, expired: 7.92 txn/s, latency: 1278.26 ms, (p50: 1200 ms, p70: 1200, p90: 1800 ms, p99: 2700 ms), latency samples: 205200
5. check swarm health
Compatibility test for 1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb ==> 13019daf01024ad4491ecc300413b5b1193aa032 passed
Upgrade the remaining nodes to version: 13019daf01024ad4491ecc300413b5b1193aa032
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 2042.32 txn/s, submitted: 2050.71 txn/s, failed submission: 8.39 txn/s, expired: 8.39 txn/s, latency: 1565.29 ms, (p50: 1200 ms, p70: 1200, p90: 2100 ms, p99: 11100 ms), latency samples: 170360
Test Ok

vineethk changed the title ~~Implement a crate containing shared data structures and algorithms for use in other crates.~~ Shared data structures and algorithms crate Apr 10, 2026

vineethk force-pushed the vk/shared-dsa branch from d9d7068 to 79a4dee Compare April 10, 2026 01:28

vineethk requested a review from Copilot April 10, 2026 01:28

vineethk marked this pull request as ready for review April 10, 2026 01:28

Copilot started reviewing on behalf of vineethk April 10, 2026 01:29 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

Comment thread third_party/move/shared-dsa/src/unordered_set.rs

Comment thread third_party/move/shared-dsa/src/unordered_map.rs

vineethk mentioned this pull request Apr 13, 2026

[mono-vm] Representation and perf refactoring within specializer #19427

Merged

2 tasks

vineethk requested review from calintat, georgemitenkov and vgao1996 April 13, 2026 14:59

georgemitenkov approved these changes Apr 13, 2026

View reviewed changes

vineethk force-pushed the vk/shared-dsa branch from 79a4dee to b221056 Compare April 13, 2026 15:06

vineethk enabled auto-merge (squash) April 13, 2026 15:42

calintat approved these changes Apr 13, 2026

View reviewed changes

Comment thread third_party/move/shared-dsa/src/unordered_map.rs

Comment thread third_party/move/shared-dsa/src/unordered_map.rs Outdated

This comment has been minimized.

Sign in to view

vineethk mentioned this pull request Apr 13, 2026

[mono-vm][specializer] Move a fusing operation from lowering to destacking phase #19431

Merged

1 task

Vineeth Kashyap and others added 3 commits April 13, 2026 17:04

Implement a crate containing shared data structures and algorithms fo…

ca2e4ac

…r use in other crates.

clippy fix

da2437b

address review comments

13019da

vineethk force-pushed the vk/shared-dsa branch from b221056 to 13019da Compare April 13, 2026 22:21

This comment has been minimized.

Sign in to view

vineethk merged commit 0631630 into main Apr 14, 2026
57 of 59 checks passed

vineethk deleted the vk/shared-dsa branch April 14, 2026 00:14

This was referenced Apr 15, 2026

[mono-vm] Factor specializer pipeline out of the executable and into the specialize crate. #19454

Merged

[mono-vm][specializer] Remove explicit types from simulated operand stack #19466

Merged

vineethk mentioned this pull request Apr 22, 2026

[mono-move] Migrate from using the old type representation to the new one #19530

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Shared data structures and algorithms crate#19388

Shared data structures and algorithms crate#19388
vineethk merged 3 commits intomainfrom
vk/shared-dsa

vineethk commented Apr 10, 2026 •

edited by cursor Bot

Loading

Uh oh!

vineethk commented Apr 10, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

calintat left a comment

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions Bot commented Apr 13, 2026

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

vineethk commented Apr 10, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How Has This Been Tested?

Type of Change

Uh oh!

vineethk commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

calintat left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions Bot commented Apr 13, 2026

✅ Forge suite realistic_env_max_load success on 13019daf01024ad4491ecc300413b5b1193aa032

Uh oh!

github-actions Bot commented Apr 13, 2026

✅ Forge suite compat success on 1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb ==> 13019daf01024ad4491ecc300413b5b1193aa032

Uh oh!

github-actions Bot commented Apr 13, 2026

✅ Forge suite framework_upgrade success on 1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb ==> 13019daf01024ad4491ecc300413b5b1193aa032

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vineethk commented Apr 10, 2026 •

edited by cursor Bot

Loading

vineethk commented Apr 10, 2026 •

edited

Loading

✅ Forge suite `realistic_env_max_load` success on `13019daf01024ad4491ecc300413b5b1193aa032`

✅ Forge suite `compat` success on `1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb` ==> `13019daf01024ad4491ecc300413b5b1193aa032`

✅ Forge suite `framework_upgrade` success on `1ccf655f27971b762e4c90fc1606e3a8ba4ed7fb` ==> `13019daf01024ad4491ecc300413b5b1193aa032`