Skip to content

Commit fcc2a1a

Browse files
committed
Add cancellable runtime-neutral async XLSX queries
1 parent b166b70 commit fcc2a1a

14 files changed

Lines changed: 491 additions & 16 deletions

‎.github/workflows/rust.yml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,37 @@ jobs:
194194
--filter "FullyQualifiedName~MiniExcelCsvTests"
195195
--verbosity minimal
196196
197+
async-query-parity:
198+
strategy:
199+
matrix:
200+
os: [ubuntu-latest, windows-latest]
201+
runs-on: ${{ matrix.os }}
202+
steps:
203+
- uses: actions/checkout@v4
204+
- name: Checkout MiniExcel .NET
205+
uses: actions/checkout@v4
206+
with:
207+
repository: mini-software/MiniExcel
208+
ref: b9a76d7af62142e0e38545b6905b01a06e8d160e
209+
path: dotnet-source
210+
- name: Install Rust 1.85.0
211+
uses: dtolnay/rust-toolchain@1.85.0
212+
- uses: Swatinem/rust-cache@v2
213+
with:
214+
key: async-query-parity-${{ matrix.os }}
215+
- name: Setup .NET
216+
uses: actions/setup-dotnet@v4
217+
with:
218+
dotnet-version: 10.0.x
219+
- name: Run Rust async query tests
220+
run: cargo +1.85.0 test -p miniexcel --test query_async --features async --locked
221+
- name: Run MiniExcel .NET async query baseline tests
222+
run: >-
223+
dotnet test dotnet-source/tests/MiniExcel.OpenXml.Tests/MiniExcel.OpenXml.Tests.csproj
224+
--framework net10.0
225+
--filter "FullyQualifiedName~MiniExcelOpenXmlImporterAsyncTests"
226+
--verbosity minimal
227+
197228
msrv:
198229
runs-on: ubuntu-latest
199230
steps:

‎README.md‎

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,33 @@ for record in MiniExcel::query_as::<Record>("book.xlsx")? {
9898

9999
`MiniExcel::query()` and `query_as()` accept paths because a worker owns the ZIP archive while the iterator is alive.
100100

101+
### Async Query
102+
103+
Enable the optional `async` feature to consume dynamic or Serde-typed path queries as a
104+
runtime-neutral stream:
105+
106+
```rust
107+
use futures_util::StreamExt;
108+
use miniexcel::{CancellationToken, MiniExcel, ReadOptions};
109+
110+
let cancellation = CancellationToken::new();
111+
let mut rows = MiniExcel::query_async_with_options_and_cancellation(
112+
"book.xlsx",
113+
&ReadOptions::new(),
114+
cancellation.clone(),
115+
)?;
116+
117+
while let Some(row) = rows.next().await {
118+
println!("{:?}", row?);
119+
}
120+
```
121+
122+
`query_async*()` and `query_as_async*()` use bounded channels and keep blocking ZIP/XML work off
123+
the async executor. They do not turn filesystem access into async I/O and do not require Tokio.
124+
Explicit cancellation yields an error recognized by `Error::is_cancelled()`; dropping the stream
125+
requests cancellation without blocking the executor. Parser initialization or the current row may
126+
finish before background cleanup completes.
127+
101128
## Borrowed Readers And Writers
102129

103130
Use visitor APIs for caller-owned `Read + Seek` sources without materializing all rows or transferring ownership:
@@ -489,7 +516,7 @@ Version 1 does not implement `@group`, `@if`, parametrized sheet cloning, `$=` f
489516
- `MiniExcel::query()` and `query_as()` strictly stream worksheet XML from paths.
490517
- Grouped analytics retain state proportional to distinct groups and stop at `max_groups`.
491518
- RAG exports never recalculate formulas and reject hidden sheets unless explicitly allowed.
492-
- Streaming queries are synchronous and use one worker thread per active query. The optional async Insert API makes row production asynchronous, not ZIP I/O.
519+
- Synchronous streaming queries use one worker thread per active query. Optional async query and Insert APIs use bounded channels around blocking XLSX workers; ZIP/XML/filesystem work is not async I/O.
493520
- Save creates new workbooks and refuses existing target paths by default. `MiniExcel::insert*()` atomically appends or strictly replaces a worksheet in an existing `.xlsx` path, or creates a workbook when the path is missing.
494521

495522
## Not Supported

‎README.zh-CN.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,31 @@ for record in MiniExcel::query_as::<Record>("book.xlsx")? {
9898

9999
`MiniExcel::query()` 和 `query_as()` 接收路径,因为迭代器存活期间由 worker 持有 ZIP archive。
100100

101+
### Async Query
102+
103+
启用可选 `async` feature 后,可以 runtime-neutral stream 消费动态或 Serde 类型化 path query:
104+
105+
```rust
106+
use futures_util::StreamExt;
107+
use miniexcel::{CancellationToken, MiniExcel, ReadOptions};
108+
109+
let cancellation = CancellationToken::new();
110+
let mut rows = MiniExcel::query_async_with_options_and_cancellation(
111+
"book.xlsx",
112+
&ReadOptions::new(),
113+
cancellation.clone(),
114+
)?;
115+
116+
while let Some(row) = rows.next().await {
117+
println!("{:?}", row?);
118+
}
119+
```
120+
121+
`query_async*()` 与 `query_as_async*()` 使用 bounded channel,将 blocking ZIP/XML 工作移出
122+
async executor。它们不会把 filesystem access 变成 async I/O,也不依赖 Tokio。显式取消会
123+
返回可由 `Error::is_cancelled()` 识别的错误;丢弃 stream 会请求取消,但不会阻塞 executor。
124+
Parser 初始化或当前 row 可能会在后台清理完成前继续执行。
125+
101126
## 借用 Reader 与 Writer
102127

103128
对调用方持有的 `Read + Seek` source,可使用 visitor API 流式处理,不会物化全部 row 或转移所有权:
@@ -484,7 +509,7 @@ MiniExcel::save_as_template(
484509
- `MiniExcel::query()` 和 `query_as()` 会从路径严格流式解析 worksheet XML。
485510
- 分组分析保留与不同 group 数量成比例的状态,并在 `max_groups` 停止。
486511
- RAG 导出不会重新计算公式,hidden sheet 未显式允许时会拒绝处理。
487-
- 流式 query 是同步接口,每个活动 query 使用一个 worker thread。可选 async Insert API 只让 row production 异步,不是 async ZIP I/O。
512+
- 同步流式 query 每个活动 query 使用一个 worker thread。可选 async query/Insert API 通过 bounded channel 包装 blocking XLSX worker;ZIP/XML/filesystem 工作并不是 async I/O。
488513
- Save 会创建新工作簿,并默认拒绝已有目标路径。`MiniExcel::insert*()` 会向现有 `.xlsx` path 原子 append 或严格 replace worksheet;路径不存在时则创建 workbook。
489514

490515
## 暂不支持

‎docs/compatibility.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ The Rust MVP implements the smallest useful MiniExcel-style XLSX read/write surf
1111
| Dependency | Locked API line | Role | License | MSRV note |
1212
| --- | --- | --- | --- | --- |
1313
| `atomicwrites` | 0.4 | Safe cross-platform atomic replacement for Windows path inserts | MIT | Windows-only dependency; checked with Rust 1.85 |
14-
| `async-channel` / `event-listener` / `futures-*` | 2.5 / 5.4 / 0.3 | Optional runtime-neutral async Insert producer and cancellation | MIT OR Apache-2.0 | Enabled only by the `async` feature; checked with Rust 1.85 |
14+
| `async-channel` / `event-listener` / `futures-*` | 2.5 / 5.4 / 0.3 | Optional runtime-neutral async query/Insert and cancellation | MIT OR Apache-2.0 | Enabled only by the `async` feature; checked with Rust 1.85 |
1515
| `fs2` | 0.4 | Cross-platform advisory locking for path inserts | MIT OR Apache-2.0 | Checked with Rust 1.85 |
1616
| `calamine` | 0.35 | XLSX parsing and Serde row deserialization | MIT | 0.35 declares Rust 1.83 |
1717
| `clap` | 4.6 | Local CLI argument parsing | MIT OR Apache-2.0 | 4.6 declares Rust 1.85 |
@@ -54,6 +54,7 @@ The latest `calamine 0.36` and `rust_xlsxwriter 0.97` require Rust 1.88. The MVP
5454
| Multi-sheet export | `save_as_sheets()` / `save_as_serialized_sheets()` | Preserves input sheet order and returns data-row counts |
5555
| `InsertSheet` append/replace | `insert()` / `insert_with_schema()` / `insert_serialized()` / borrowed reader-to-writer variants | Path APIs are atomic; separate borrowed streams require an empty sink and preserve package behavior without atomic commit |
5656
| Async Insert producer | `insert_with_schema_async*()` | Optional `async` feature; bounded producer channel with blocking XLSX work on a dedicated thread |
57+
| Async path query | `query_async*()` / `query_as_async*()` | Optional `async` feature; bounded dynamic/Serde streams, cooperative cancellation, blocking XLSX workers |
5758
| Per-sheet visibility | `WriteOptions::with_sheet_visibility()` | Visible, hidden, and very hidden; first visible sheet is active |
5859
| `overwriteFile` | `WriteOptions::with_overwrite_file()` | Defaults to `false`; existing paths require explicit opt-in |
5960
| `FreezeRowCount` / `FreezeColumnCount` | `WriteOptions::with_freeze_row_count()` / `with_freeze_column_count()` | Defaults to one frozen row and zero frozen columns |
@@ -162,7 +163,7 @@ dotnet test ../MiniExcel/tests/MiniExcel.OpenXml.Tests/MiniExcel.OpenXml.Tests.c
162163

163164
The Rust workflow runs the Rust contract on Linux and Windows. Its .NET parity job checks out the MiniExcel repository, copies this revision's contract into that checkout, and runs the .NET adapter on Linux. A compatibility change is complete only when the shared contract is updated deliberately and both adapters pass it.
164165

165-
The contract covers only the current common surface: dynamic/typed path queries, inclusive range queries, column-name discovery, header behavior, sheet selection/order, A1 starts, empty/style-only rows, inferred cell references, scalar/date/duration mapping, trimmed typed headers, and conversion-error row/value context. Structured provenance is a Rust research extension and is not a .NET parity claim. Async APIs, DataReader, templates, and writing parity remain outside version 1 and must not be described as equivalent yet.
166+
The contract covers only the current common surface: dynamic/typed path queries, inclusive range queries, column-name discovery, header behavior, sheet selection/order, A1 starts, empty/style-only rows, inferred cell references, scalar/date/duration mapping, trimmed typed headers, and conversion-error row/value context. Structured provenance is a Rust research extension and is not a .NET parity claim. Async APIs are tested against the same fixtures but are not part of the shared contract; DataReader, templates, and writing parity also remain outside version 1.
166167

167168
## .NET Coverage Boundary
168169

@@ -182,6 +183,7 @@ The contract covers only the current common surface: dynamic/typed path queries,
182183
| Versioned grouped analytics | Rust research extension | No |
183184
| Addressed JSONL/Markdown/manifest RAG export | Rust research extension | No |
184185
| Async Insert producer | Implemented behind optional feature | Rust cancellation tests; not shared internals |
186+
| Async dynamic/typed path query | Implemented behind optional feature | Rust parity, cancellation, error, and cleanup tests |
185187
| DataReader and broader stream ownership | Deferred | No |
186188
| Append worksheet to existing `.xlsx` workbook | Implemented and atomically committed | Rust tests; shared parity contract not yet extended |
187189
| Strict worksheet replacement | Implemented for plain targets and supported target-owned closures | Rust tests; stale calcChain removed and full recalculation requested |
@@ -192,4 +194,4 @@ This matrix is the coverage claim: Rust does not yet provide complete API parity
192194

193195
## Deferred Work
194196

195-
SQL text parsing, `HAVING`, `ORDER BY`, joins, windows, pivots, disk-spill aggregation, vector indexing, model calls, old Excel formats, advanced template directives and sheet cloning, image authoring, merged-cell APIs, formula calculation/dependency expansion, formula authoring, general styling, async query/export/template I/O, and borrowed XLSX lazy readers require separate design and acceptance milestones. CSV DataReader/DataTable adapters are intentionally replaced by Rust iterators, and a one-call CSV/XLSX converter is not exposed; callers compose query and save APIs. See the [Insert migration guide](insert-v1-migration.md) for supported workflows and deliberate differences.
197+
SQL text parsing, `HAVING`, `ORDER BY`, joins, windows, pivots, disk-spill aggregation, vector indexing, model calls, old Excel formats, advanced template directives and sheet cloning, image authoring, merged-cell APIs, formula calculation/dependency expansion, formula authoring, general styling, async export/template I/O, async borrowed readers, and borrowed XLSX lazy readers require separate design and acceptance milestones. CSV DataReader/DataTable adapters are intentionally replaced by Rust iterators, and a one-call CSV/XLSX converter is not exposed; callers compose query and save APIs. See the [Insert migration guide](insert-v1-migration.md) for supported workflows and deliberate differences.

‎docs/compatibility.zh-CN.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Rust MVP 在统一的 `MiniExcel` facade 后实现最小但实用的 MiniExcel
1111
| 依赖 | 锁定 API 版本线 | 用途 | 许可证 | MSRV 说明 |
1212
| --- | --- | --- | --- | --- |
1313
| `atomicwrites` | 0.4 | Windows path insert 的安全跨平台原子替换 | MIT | 仅 Windows 依赖;已使用 Rust 1.85 检查 |
14-
| `async-channel` / `event-listener` / `futures-*` | 2.5 / 5.4 / 0.3 | 可选 runtime-neutral async Insert producer 与 cancellation | MIT OR Apache-2.0 | 仅由 `async` feature 启用;已使用 Rust 1.85 检查 |
14+
| `async-channel` / `event-listener` / `futures-*` | 2.5 / 5.4 / 0.3 | 可选 runtime-neutral async query/Insert 与 cancellation | MIT OR Apache-2.0 | 仅由 `async` feature 启用;已使用 Rust 1.85 检查 |
1515
| `fs2` | 0.4 | Path insert 的跨平台 advisory lock | MIT OR Apache-2.0 | 已使用 Rust 1.85 检查 |
1616
| `calamine` | 0.35 | XLSX 解析和 Serde row 反序列化 | MIT | 0.35 声明 Rust 1.83 |
1717
| `clap` | 4.6 | 本地 CLI 参数解析 | MIT OR Apache-2.0 | 4.6 声明 Rust 1.85 |
@@ -54,6 +54,7 @@ Rust MVP 在统一的 `MiniExcel` facade 后实现最小但实用的 MiniExcel
5454
| 多工作表导出 | `save_as_sheets()` / `save_as_serialized_sheets()` | 保留输入工作表顺序并返回数据行数 |
5555
| `InsertSheet` append/replace | `insert()` / `insert_with_schema()` / `insert_serialized()` / borrowed reader-to-writer variants | Path API 为原子操作;独立 borrowed stream 要求空 sink,并在无原子 commit 的情况下保持相同 package 行为 |
5656
| Async Insert producer | `insert_with_schema_async*()` | 可选 `async` feature;bounded producer channel,XLSX 工作在专用 blocking thread |
57+
| Async path query | `query_async*()` / `query_as_async*()` | 可选 `async` feature;bounded 动态/Serde stream、协作式 cancellation、blocking XLSX worker |
5758
| 每表 visibility | `WriteOptions::with_sheet_visibility()` | visible、hidden、very hidden;第一个 visible sheet 为 active |
5859
| `overwriteFile` | `WriteOptions::with_overwrite_file()` | 默认 `false`;已有路径需要显式允许覆盖 |
5960
| `FreezeRowCount` / `FreezeColumnCount` | `WriteOptions::with_freeze_row_count()` / `with_freeze_column_count()` | 默认冻结一行、零列 |
@@ -162,7 +163,7 @@ dotnet test ../MiniExcel/tests/MiniExcel.OpenXml.Tests/MiniExcel.OpenXml.Tests.c
162163

163164
Rust workflow 会在 Linux 和 Windows 上运行 Rust 契约。其 .NET parity job 会 checkout MiniExcel 仓库,将当前 revision 的契约复制到该 checkout,并在 Linux 上运行 .NET adapter。只有在共享契约被有意识地更新且两个 adapter 都通过后,兼容性修改才算完成。
164165

165-
契约只覆盖当前公共交集:动态/类型化路径 query、包含端点的 range query、column name 发现、header 行为、sheet 选择/顺序、A1 起点、空 row/仅 style row、推断 cell reference、scalar/date/duration 映射、trim 后的类型化 header,以及转换 error 的 row/value 上下文。Structured provenance 是 Rust 研究扩展,不构成 .NET 等价声明。Async API、DataReader、template 和写入等价行为仍不属于版本 1,当前不得描述为等价。
166+
契约只覆盖当前公共交集:动态/类型化路径 query、包含端点的 range query、column name 发现、header 行为、sheet 选择/顺序、A1 起点、空 row/仅 style row、推断 cell reference、scalar/date/duration 映射、trim 后的类型化 header,以及转换 error 的 row/value 上下文。Structured provenance 是 Rust 研究扩展,不构成 .NET 等价声明。Async API 使用相同 fixture 测试,但尚未纳入共享契约;DataReader、template 和写入等价行为也仍不属于版本 1。
166167

167168
## .NET 覆盖边界
168169

@@ -182,6 +183,7 @@ Rust workflow 会在 Linux 和 Windows 上运行 Rust 契约。其 .NET parity j
182183
| 版本化分组分析 | Rust 研究扩展 | 否 |
183184
| 带地址 JSONL/Markdown/manifest RAG 导出 | Rust 研究扩展 | 否 |
184185
| Async Insert producer | 已通过可选 feature 实现 | Rust cancellation test;不共享内部行为 |
186+
| Async 动态/类型化 path query | 已通过可选 feature 实现 | Rust parity、cancellation、error 与 cleanup 测试 |
185187
| DataReader 与更广泛的 stream ownership | 延后 | 否 |
186188
| 向现有 `.xlsx` workbook 追加 worksheet | 已实现并原子提交 | Rust 测试;共享 parity contract 尚未扩展 |
187189
| 严格 worksheet replacement | 已支持 plain target 与受支持的 target-owned closure | Rust 测试;删除 stale calcChain 并要求 full recalculation |
@@ -192,4 +194,4 @@ Rust workflow 会在 Linux 和 Windows 上运行 Rust 契约。其 .NET parity j
192194

193195
## 延后工作
194196

195-
SQL 文本解析、`HAVING`、`ORDER BY`、join、window、pivot、磁盘 spill 聚合、向量索引、模型调用、旧 Excel 格式、高级 template 指令与 sheet 克隆、image authoring、merged-cell API、公式计算/依赖展开、公式编写、通用 style、async query/export/template I/O,以及 borrowed XLSX lazy reader,都需要独立的设计与验收里程碑。CSV DataReader/DataTable adapter 已有意替换为 Rust iterator;当前不提供一步式 CSV/XLSX converter,调用方可组合 query 与 save API。支持工作流与有意保留的差异见 [Insert 迁移说明](insert-v1-migration.zh-CN.md)。
197+
SQL 文本解析、`HAVING`、`ORDER BY`、join、window、pivot、磁盘 spill 聚合、向量索引、模型调用、旧 Excel 格式、高级 template 指令与 sheet 克隆、image authoring、merged-cell API、公式计算/依赖展开、公式编写、通用 style、async export/template I/O、async borrowed reader,以及 borrowed XLSX lazy reader,都需要独立的设计与验收里程碑。CSV DataReader/DataTable adapter 已有意替换为 Rust iterator;当前不提供一步式 CSV/XLSX converter,调用方可组合 query 与 save API。支持工作流与有意保留的差异见 [Insert 迁移说明](insert-v1-migration.zh-CN.md)。

0 commit comments

Comments
 (0)