vm: parallelize BagOfCells::deserialize by dependency depth

[Lordron]

What

vm::BagOfCells::deserialize builds the cell tree on a single thread. On mainnet masterchain state (121 M cells, 6.8 GB serialized), this takes ~47.5 s on a 16-core host — 15 cores idle. This PR replaces the serial loop with a level-parallel pipeline, taking the same deserialize down to ~21 s.

This path is hot on node cold start: initial sync, restart with wiped CellDB, state re-import from disk. It's not on the warm block-application path (that uses lazy cells via CellDb), so the PR benefits sync catch-up, not steady-state throughput.

How

Three phases, respecting BOC's ref_idx > idx invariant:

1. Phase 1a — parallel. Parse each cell's CellSerializationInfo, store its ref indices in a flat std::vector<CellRefs>. Pure read of BOC bytes, no ordering dependency.
2. Phase 1b — sequential, fast. Walk high-idx to low-idx. Compute depth[idx] = 1 + max(depth[ref_idx]) by table lookup (no re-parsing).
3. Phase 2 — sequential, fast. Bucket cells by depth. Cells at the same depth never reference each other.
4. Phase 3 — parallel per level. For each depth level bottom-up, worker threads atomically pull cells and call create_data_cell. cell_list writes target distinct indices, no locks.

Arena allocation (existing DataCell::use_arena) is enabled for the duration — 1 MB thread-local bump batches instead of 121 M individual operator new. That alone is a 15 % win on the serial path and compounds with parallelism by reducing memory-controller contention. Arena cells are not individually freed, matching how InMemoryBagOfCellsDb::parallel_scan_cells already uses this path.

Fallback: BOCs < 64 MB keep the original serial path (thread-spawn overhead not worth it).

Results

Test host: AMD EPYC-Rome, 16 cores, 30 GiB RAM, Ubuntu 24.04 LTS VPS. Build: -O3 -march=native, jemalloc 5.3.

Same input (mainnet state 60660942, 121,421,299 cells, 6,839 MB):

Configuration	Time	cells/sec	MB/sec	Speedup
Serial, no arena (baseline)	47,573 ms	2.55 M	143	1.00×
Serial + arena	40,289 ms	3.01 M	169	1.18×
Parallel + arena (this PR)	21,008 ms	5.78 M	325	2.26×

Phase breakdown of the 21 s run: ph1a=6.5 s, ph1b=0.3 s, ph3=10.2 s (Phase 3 is the remaining bottleneck).

Correctness

Same BOC deserialized serial and parallel — identical Merkle root hash 1202B7FB528AA618…. A Merkle root is an SHA-256 digest over every cell's contents and every link, so matching roots imply byte-equivalent trees.

A commented-out BOC_FORCE_SERIAL env-var switch is left in the code for reviewers to re-run this A/B test. Existing test-cells and test-db unit tests pass unchanged.

Trade-offs

• Transient memory: +2.4 GB during Phase 1a (20 B × 121 M cells, freed after Phase 1b, lives ~6.5 s). Struct can be packed to 16 B if a reviewer is fine adding a max-cell-count assertion; kept at 20 B to avoid an observable cap.
• Remaining ceiling (Phase 3, 10.2 s): atomic refcount traffic on Ref<Cell> copies for widely-shared children, plus per-cell SHA-256. Both are worth a follow-up PR — Ref<Cell> handoff could bump the child refcount once per parent instead of twice, and multi-buffer SHA-NI could amortize per-level. Out of scope here.

Files

crypto/vm/boc.cpp: +286 / −11. No API changes; BagOfCells::deserialize signature unchanged; all callers unaffected.

[Lordron]

Quick follow-up on the "atomic refcount contention" hypothesis I called out in the
"remaining ceiling" section:

I prototyped a move-variant of create_data_cell that passes refs into DataCell via
std::move instead of copy, saving one acquire_shared / dec pair per ref (~484M atomic
ops removed across the run on our 121M-cell mainnet state).

A/B on the same host (EPYC-Rome, 16 cores, 30 GiB, arena on):

parallel + arena (this PR): 21,008 ms
parallel + arena + refmove variant: 21,119 ms

Within noise — refcount traffic is not the bottleneck here.

Leaving this here in case a reviewer was curious about the refcount angle — it looked
tempting on paper but didn't move the needle.

[DanShaders]

Thanks! I will look into this in the near future.

[Lordron]

Thanks! Some follow-up findings and a couple of ready-to-go optimisations we'd propose on top of this PR.

First, the low-hanging win: bump the arena batch + madvise huge pages

allocate_in_arena currently hands out from thread-local 1 MB batches (crypto/vm/cells/DataCell.cpp). We'd propose bumping to 16 MB and hinting the kernel right after the backing allocation:

char* allocate_in_arena(size_t size) {
  constexpr size_t batch_size = 16 * (1ULL << 20);   // was 1 << 20
  thread_local td::MutableSlice batch;
  auto aligned_size = (size + 7) / 8 * 8;
  if (batch.size() < aligned_size) {
    char* ptr = new char[batch_size];
#if defined(__unix__) || defined(__APPLE__)
    ::madvise(ptr, batch_size, MADV_HUGEPAGE);
#endif
    batch = td::MutableSlice(ptr, batch_size);
  }
  auto res = batch.begin();
  batch.remove_prefix(aligned_size);
  return res;
}

Two things in one small diff:

1. 16 MB batch — a clean multiple of the 2 MB transparent-huge-page size so the kernel can back each batch with 8 huge pages instead of 256 × 4 KB pages. Also amortises allocation overhead: ~80K cells per batch at 200 B avg, so a full mainnet state needs ~1.5K batches instead of 24K.
2. MADV_HUGEPAGE — surgical (applies only to arena regions, not to unrelated process memory). Best-effort: if THP is disabled system-wide or madvise-only, this is a no-op and we fall back to 4 KB pages. On hosts where the kernel's THP setting is madvise (default on many distros), operators don't need to change THP=always.

No API change, ~5 lines.

Why we think it's worth the change

We ran perf stat on the validator PID during a deserialize window:

perf stat -e cycles,instructions,cache-references,cache-misses,\
          L1-dcache-loads,L1-dcache-load-misses,\
          dTLB-loads,dTLB-load-misses \
          -p $PID -- sleep 120

Results (EPYC-Rome, 16-core, 30 GiB, Ubuntu 24.04):

cycles             881,080,950,901
instructions       422,585,127,886   (0.48 IPC)
cache-references    22,659,106,491
cache-misses         7,932,521,192   (35.0%)
L1-dcache-loads    209,482,780,214
L1-dcache-misses     6,530,015,926   ( 3.1%)
dTLB-loads           1,099,339,548
dTLB-load-misses       377,680,142   (34.4%)

Zen2 tops out near 5 IPC, so 0.48 is ~10% of peak — CPU is heavily stalled, and the 35% cache miss + 34% dTLB miss point at memory as the dominant source.

The 34% dTLB rate stands out. During a mainnet state deserialize the DataCell working set reaches ~18 GB; at 4 KB pages that's ~4.5M pages against a 2,048-entry L2 dTLB on Zen2 — working set vs. TLB capacity ratio ~2,200×. Effectively every third field access does a page-walk. Switching the arena backing to 2 MB pages collapses that page count by 512×.

A further, more invasive idea — tentative, not recommended yet

Separately, we noticed that during parent-cell creation each child access straddles two cache lines: cnt_ (for the Ref<Cell> bookkeeping) lives in the first 64 B, while the hash field in the trailer_ flexible array starts at offset 56 and extends into the second cache line. Layout:

class CntObject {
  virtual ~CntObject();           // vtable ptr  @ [0..8]
  std::atomic<int> cnt_;          // refcount    @ [8..12]    HOT
};
class CellTraits : public CntObject {};
class Cell       : public CellTraits {};

class DataCell final : public Cell {
  unsigned bit_length_         : 11;   // packed bitfield @ [16..20]
  unsigned refs_cnt_           : 3;
  unsigned type_               : 3;
  unsigned level_              : 3;
  unsigned level_mask_         : 3;
  unsigned allocated_in_arena_ : 1;
  unsigned virtualized_        : 1;

  std::array<Ref<Cell>, 4> refs_;      //             @ [24..56]

  alignas(detail::LevelInfo) char trailer_[];
  //   holds LevelInfo[level+1] (hash+depth) starting @ offset 56,
  //   then cell data bytes.
};

struct detail::LevelInfo {
  CellHash hash;      // 32 B
  td::uint16 depth;   //  2 B  (padded to 40 B overall)
};

sizeof(DataCell) == 56 B. trailer_[0] starts at offset 56 — in CL0 — but the first LevelInfo's hash spans bytes 56..88, so it straddles the CL0/CL1 boundary: 8 B in CL0, 24 B in CL1. Every hash read pulls both cache lines.

trailer_ has to be last (flexible array, variable-length per cell depending on level + 1). The fix we'd sketch is an inline hash[0] / depth[0] slot in the fixed part of the struct, with get_hash() / get_depth() taking a fast path when level_ == 0. Level ≥ 1 cells still read from trailer.

Caveat: we haven't confirmed this is worth the surgery

The 120M-cell figure comes from cold-start masterchain-state deserialization. In the warm block-application loop, per-block cell counts are probably two orders of magnitude smaller — maybe ~1M cells at peak. At that scale the cache-line-straddle cost may be lost in noise and not justify adding a branch to every get_hash() / get_depth() call.

Before we'd open a PR for the layout change, we'd want to measure runtime DataCell::create volume on a synced node and see if it actually shows up in perf stat during block application. We haven't done that yet, so treating this as observation + prototype sketch, not a proposal.

The arena + madvise change above is different — it's zero-API-change and zero-branch-overhead, so even if it only helps cold start it's worth having.

[DanShaders]

I don't exactly like the idea to allocate cells inside of that global arena, these cells then stay in memory indefinitely IIRC. What I wanted to do for quite some time now is to have separate arenas for specific operations we do with BoCs (like "validation of candidate X" or, as you poined out "initial sync mc state deserialization") and also somehow track provenance of cells in trees. DB then becomes an ultimate way to clear provenance. If we treat cells this way, we can also stop counting their refcounts in most of the cases since the provenance lifetime becomes an ultimate test if the cell should be alive (and we spend lot of time on ARC). This should also fix some long-standing issues with memory leaks in the node. If you want, I can share with you my discussion with Claude about this, but such design change spanning the whole node might not be the best for first-time outside contribution.

The insight about cache lines is.. unexpected (Which model do you use? When people tried to do perf work with models before what they got is an immense amounts of slop) If you decide to contribute it, I will ask for benchmarks.

[Lordron]

Thanks — that's really useful context, and the per-operation arena + provenance direction makes a lot of sense. Scoping lifetime to "validate candidate X" or "initial sync mc state" is a much cleaner invariant than the current global+ARC mix, and it also naturally kills the indefinite-retention problem with the current arena (which is exactly why we were hesitant to push the 16 MB bump harder — it's a tactical improvement on a design you're already planning to replace). I'd very much appreciate seeing your Claude discussion on this — even just to know which direction not to patch over. You're right that it's not a good first-time contribution though; we'd rather leave the arena structural work to you.

On the model question — it's a team of two: me driving and Claude Opus 4.7 (1M context) doing the heavy lifting on layout analysis and profiling setup. We agree with your prior on AI perf slop; the way we tried to keep it honest was to never let the model's claims stand on their own — every proposal had to be backed by a number from the actual binary (perf stat counters, pahole output for the struct layout, bench timings on a real state file). When something couldn't be backed up, we cut it. The cache-line observation survived that filter; the refcount-move variant did not (measured within noise, so we dropped it from the proposal even though it looked plausible on paper).

On benchmarks for the hash[0] inline idea — honest caveat: we only have a 16-core / 32 GB VPS available, so what we can actually measure cleanly is cold-start masterchain-state deserialization. That's where the straddle cost compounds (120M+ cells), and we can produce before/after perf stat plus wall-clock on a fixed state file. What we can't realistically produce is runtime block-application numbers at any meaningful scale — that would need a fully synced node replaying real block streams with representative workload, and we don't have the hardware or the sync time for that. If cold-start-only benchmarks are acceptable as evidence (with the explicit caveat in the PR that we couldn't validate runtime), we're happy to put the PR together. If you'd want runtime numbers before considering it, we'd rather wait until someone with the infra picks it up than submit a PR that's known to be under-validated.

Either way, thanks again for the detailed response — it's rare to get this level of engagement on an outside contribution.

[DanShaders]

I'd very much appreciate seeing your Claude discussion on this — even just to know which direction not to patch over.

https://gist.github.com/DanShaders/8e7a1c29ed69367e0510733068182fc4

I don't exactly agree with everything there but you can see general direction.

[Lordron]

Sorry for the long answer — I had all sorts of real-life difficulties and caught a common cold. I checked the cache hypothesis, and while it is a real problem during initial synchronization, it does not occur inside DataCell. I tried various tweaks and optimizations, but it accounted for only around 0.5–1% of all cache misses we had IIRC. The real cause of the misses was the tree-building process itself IIRC

I asked my Claude to create a complete overview of our discussion to share with you. It’s much more detailed and technical: https://gist.github.com/Lordron/f6d4f06a48c2dfe3d3273905109e8e29
. Maybe someone will find it useful. @DanShaders he really liked your approach. And find it more suitiable than anything we discussed with him

I hope I can get back to this problem in a couple of weeks once I have access to a 32/64 GB VPS. Does anyone know if that would be sufficient to run as a liteserver? I remember I was unable to get past the initial synchronization even with large swap on a 32 GB machine.