offline tx retrieval tool/library WIP

[mkiesel]

This is (mostly LLM'd) code for an offline/disk TX retrieval tool.
It is comprised of a CLI tool as well as a Python wrapper.
It can pull individual transactions as well as all transactions associated with a specified mc_seqno directly from TON node on disk files.

The idea is to be able to run a "cheap" archive node:
Run a normal TON node, rclone/rsync to some cheap storage (without deleting files on the destination ever), access those files using this tool.

This is by no means finished (needs more testing, performance improvements, cleanup, memory usage checking, etc.), but I'd be interested in feedback concerning improvements and comments on whether this is a candidate to include in the TON monorepo to start with.
If so, I'll try to make things more presentable, and convert to a proper (non draft) PR.

[DanShaders]

(Assuming this was coordinated with the core team beforehand which it seems it is)

A couple comments from the maintenance standpoint:

• It would make sense to move database layout definitions closer to the place where we write to the database (validator/) to reduce the risk of desync between the reading/writing code in node and this tool. Ideally, node and the tool would use the same classes to access the database. At the same time, building the whole node to use just the database reading will be suboptimal, so there is some refactoring of db code needed to be done.
• Since this (tries) to expose an FFI interface (and then forgets to actually use it), I would prefer if the main functionality were exposed through tonlibjson (no need to force it go through the awful ClientJson implementation, just a separate header). Then, on the Python side, you will automatically get the full machinery of the tontester.
• As written, the Python code won't pass lint, please run basedpyright and uv against "your" code.

[mkiesel]

Thanks for the feedback! I'm not aware of a documented contribution process but I'll investigate.

I'll see how to improve the database layout integration; that'd probably be another PR. One goal here was to not touch the core code (which is also why some things are a bit weird).

FFI and linting: Thanks, I'll fix that.

[DanShaders]

I'm not aware of a documented contribution process but I'll investigate.

I don't think we have any (I guess because we rarely get non-spam external PRs and even then they are usually from teams we are already in touch with).

I'll see how to improve the database layout integration; that'd probably be another PR. One goal here was to not touch the core code (which is also why some things are a bit weird).

If the goal is to not touch node's code, then the tool can live in a fork. Upstreaming means that you intend to offload tool's maintenance to regular contributors and in this case you need to make the code as easy to maintain as possible and its breakage as discoverable as possible. In this particular case, this means you need to create a single source of truth of db layout somewhere, so that when we change db and forget about the tool, we get red CI and are forced to fix it.