Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 23 additions & 11 deletions docs/developers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,10 @@ The public Ninja process helpers are re-exported from `netsuke::runner`.
`CommandEnv::inherit()` leaves the parent environment in place,
`with_var` overrides one variable, and `with_path` replaces the child's
`PATH`. The parent process is never mutated. `NinjaBuildRequest` and
`NinjaToolRequest` borrow the program, CLI settings, generated build file,
target list or tool name, and `CommandEnv` needed for one invocation.
`NinjaToolRequest` borrow the program, `NinjaProcessOptions` (`working_dir` and
`jobs`), generated build file, target list or tool name, and `CommandEnv`
needed for one invocation. The process boundary is parser-independent; callers
without CLI state construct `NinjaProcessOptions` directly.

The legacy `run_ninja` and `run_ninja_tool` helpers retain their existing
signatures and inherit the parent environment. Callers that need an isolated
Expand Down Expand Up @@ -3431,6 +3433,13 @@ selected source at debug level. Process construction uses the resolved path
exported by this module and must not interpret the environment override
independently.

`src/runner/ninja_process_adapter.rs` owns the one-way translation from `Cli`
to `NinjaProcessOptions` and the public CLI-facing wrappers. It converts
`Cli::directory` to the options' UTF-8 `working_dir`, returning
`io::ErrorKind::InvalidData` for a non-UTF-8 path. The process module remains
parser-independent; callers without CLI state construct `NinjaProcessOptions`
directly.

### Module: `runner::process::command_logging`

`src/runner/process/command_logging.rs` owns the structured logging contract
Expand All @@ -3447,7 +3456,9 @@ high-cardinality payload to a debug companion event.

All command events share these structured fields:

- `operation`: caller-provided operation label such as `"build"` or tool name.
- `operation`: `run_ninja_build_internal` supplies the fixed label `"build"`
before command configuration, while `run_ninja_tool_internal` supplies the
label from `NinjaToolRequest::tool`.
- `ninja_program`: command program after UTF-8 normalization.
- `suppress_stderr`: bool derived from the `StderrMode` policy via
`stderr_mode.is_suppress()`, true when the policy suppresses direct
Expand All @@ -3469,12 +3480,13 @@ exits, which lets downstream filtering distinguish spawn failures from
exit-status failures.

`run_ninja_internal` is the shared execution pattern used by build and tool
paths:
paths. It takes a `NinjaInternalRequest`, a clock, and a configuration closure;
the request groups the execution fields:

1. Create `Command` with `Command::new(request.program)`.
2. Pass it into a closure that applies operation-specific configuration.
3. Call `run_command_and_stream_with_context` with optional status observer,
the request's `stderr_mode` policy, and the chosen `operation`.
3. Call `run_command_and_stream_with_context` with the request's optional
status observer and execution context.
4. Let `run_command_and_stream_with_context` handle span creation, execution
logging, failure logging, and exit-status enforcement via context helpers.

Expand Down Expand Up @@ -3587,11 +3599,11 @@ governs is the environment Ninja's own child commands see when it shells out.

The explicit request APIs compose on top of `CommandEnv`: `NinjaBuildRequest`/
`NinjaToolRequest` carry `env: &CommandEnv` and `stderr_mode: StderrMode`
fields alongside the program, CLI settings, and build file, and are consumed
by `run_ninja_with`/`run_ninja_tool_with`. The convenience wrappers
`run_ninja`/`run_ninja_tool` live at the runner boundary
(`src/runner/mod.rs`), call these with `CommandEnv::inherit()`, and derive the
`stderr_mode` policy from the CLI via
fields alongside the program, `NinjaProcessOptions`, and build file, and are
consumed by `run_ninja_with`/`run_ninja_tool_with`. The convenience wrappers
`run_ninja`/`run_ninja_tool` live in `src/runner/ninja_process_adapter.rs`,
call these with `CommandEnv::inherit()`, and derive the `stderr_mode` policy
from the CLI via
`StderrMode::from_json_enabled(cli.json)`, reproducing production behaviour;
tests reach for `run_ninja_with`/`run_ninja_tool_with` directly to supply a
`CommandEnv` built with `with_path` instead. Section 6.1 of the
Expand Down
24 changes: 17 additions & 7 deletions docs/netsuke-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -2231,14 +2231,24 @@ child process's execution environment.

Every invocation is described by a borrowed request bundle rather than a long
parameter list: `NinjaBuildRequest` for a build and `NinjaToolRequest` for
`ninja -t <tool>`. Each names the resolved program, the parsed CLI settings,
the generated build file, the targets or tool, a `&CommandEnv` describing the
child's environment, and the `stderr_mode: StderrMode` policy field.
`ninja -t <tool>`. Each names the resolved program, `NinjaProcessOptions`
(an optional UTF-8 working directory and job count), the generated build file,
the targets or tool, a `&CommandEnv` describing the child's environment, and the
`stderr_mode: StderrMode` policy field.
`run_ninja_with` and `run_ninja_tool_with` consume these; the convenience
wrappers `run_ninja` and `run_ninja_tool` live at the runner boundary, call
them with `CommandEnv::inherit()`, and derive the `stderr_mode` policy from the
CLI via `StderrMode::from_json_enabled(cli.json)`, which is production
behaviour.
wrappers `run_ninja` and `run_ninja_tool` live in
`runner::ninja_process_adapter`, translate `Cli` state at the runner boundary,
call them with `CommandEnv::inherit()`, and derive the `stderr_mode` policy
from the CLI via `StderrMode::from_json_enabled(cli.json)`, which is production
behaviour. Process requests never import `Cli`; callers without parser state
construct `NinjaProcessOptions` directly.
The adapter converts `Cli::directory` to the UTF-8 path at this boundary and
returns `io::ErrorKind::InvalidData` if the CLI path is not valid UTF-8.

The private `run_ninja_internal` helper takes a `NinjaInternalRequest`, a
clock, and a `configure` closure. The request groups the resolved program,
stderr policy, optional status observer, operation label, and failure-output
capture setting before the closure configures the command.

The command construction follows this pattern:

Expand Down
23 changes: 12 additions & 11 deletions docs/users-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -701,21 +701,21 @@ stay isolated from the injected `PATH`.

The request itself is a named type: `netsuke::runner::NinjaBuildRequest` for a
build and `netsuke::runner::NinjaToolRequest` for `ninja -t <tool>`. Both
borrow their fields, so one `CommandEnv` and one `Cli` can serve several
borrow their fields, so one `CommandEnv` and one `NinjaProcessOptions` can
serve several
invocations. The [v0.1.0 migration guide](v0-1-0-migration-guide.md)
summarizes these additions and confirms the wrappers are unchanged.

<!-- tested-example: guide-ninja-request-snippet -->

```rust
use netsuke::cli::Cli;
use netsuke::runner::{
BuildTargets, CommandEnv, NinjaBuildRequest, NinjaToolRequest, StderrMode, run_ninja_tool_with,
run_ninja_with,
BuildTargets, CommandEnv, NinjaBuildRequest, NinjaProcessOptions, NinjaToolRequest,
StderrMode, run_ninja_tool_with, run_ninja_with,
};
use std::path::Path;

let cli = Cli::default();
let options = NinjaProcessOptions::default();
let targets = BuildTargets::default();
// `with_path` replaces the child's `PATH` outright, so compose the whole
// value first. The calling process is never modified.
Expand All @@ -727,21 +727,21 @@ let env = CommandEnv::inherit()

let build = NinjaBuildRequest {
program: Path::new("/usr/bin/ninja"),
cli: &cli,
options: &options,
build_file: Path::new("build.ninja"),
targets: &targets,
env: &env,
// `Suppress` in JSON diagnostics mode keeps the child's output out of
// the machine-readable streams; `Forward` relays it to the caller.
stderr_mode: StderrMode::from_json_enabled(cli.json),
stderr_mode: StderrMode::Forward,
};
let clean = NinjaToolRequest {
program: Path::new("/usr/bin/ninja"),
cli: &cli,
options: &options,
build_file: Path::new("build.ninja"),
tool: "clean",
env: &env,
stderr_mode: StderrMode::from_json_enabled(cli.json),
stderr_mode: StderrMode::Forward,
};

if std::env::var_os("NETSUKE_GUIDE_RUN").is_some() {
Expand All @@ -752,8 +752,9 @@ if std::env::var_os("NETSUKE_GUIDE_RUN").is_some() {

The convenience wrappers `run_ninja` and `run_ninja_tool` keep their existing
signatures and behaviour, so a caller that uses them needs no change; the
request bundles gained the required `stderr_mode` field, so a caller that
constructs `NinjaBuildRequest`/`NinjaToolRequest` directly must supply it.
request bundles use `options: &options` instead of `cli: &cli` and gained the
required `stderr_mode` field, so a caller that constructs
`NinjaBuildRequest`/`NinjaToolRequest` directly must supply both.
Each release records such additions in [`CHANGELOG.md`](../CHANGELOG.md),
which is where Netsuke signposts Rust API changes — with no stability promise
attached to them ahead of 1.0.
Expand Down
31 changes: 18 additions & 13 deletions docs/v0-1-0-migration-guide.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
# Migrating to v0.1.0

This guide signposts the v0.1.0 beta additions: the injectable child
environment (`CommandEnv`), the named Ninja request types, and target/action
discovery through `description` and `netsuke help targets`. Existing manifests
remain compatible, and callers of the unchanged convenience wrappers compile
unchanged.
Rust callers that construct `Target` with a struct literal must add the new
environment (`CommandEnv`), the named Ninja request types, narrow process
options (`NinjaProcessOptions`), and target/action discovery through
`description` and `netsuke help targets`. Existing manifests remain compatible,
and callers of the unchanged convenience wrappers compile unchanged. Rust
callers that construct `Target` with a struct literal must add the new
`description` field (set it to `None` or `Some(...)`); deserialized manifests
remain compatible, and every other addition is opt-in.
remain compatible. Callers constructing `NinjaBuildRequest` or
`NinjaToolRequest` must replace `cli: &cli` with `options: &options`; every
other addition is opt-in.

## Netsuke is a build tool, not a library

Expand All @@ -27,7 +29,7 @@ Table: documented v0.1.0 additions, including `netsuke help targets`, and their
| --- | --- | --- |
| Convenience wrappers | Unchanged. `run_ninja` and `run_ninja_tool` behave exactly as before, inheriting the process environment. | [Users' guide](users-guide.md) |
| Child environment | New opt-in `netsuke::runner::CommandEnv` carries additive variable overrides and an injected `PATH` for Ninja child processes. | [Users' guide](users-guide.md) |
| Request types | New `netsuke::runner::NinjaBuildRequest` and `netsuke::runner::NinjaToolRequest` name the program, build file, targets or tool, a child environment, and a required `stderr_mode: StderrMode` policy for the `*_with` run functions. | [Users' guide](users-guide.md) |
| Request types | New `netsuke::runner::NinjaBuildRequest` and `netsuke::runner::NinjaToolRequest` name the program, `NinjaProcessOptions`, build file, targets or tool, a child environment, and a required `stderr_mode: StderrMode` policy for the `*_with` run functions. | [Users' guide](users-guide.md) |
| Glob expansion | Parent-relative patterns such as `glob('../shared/*.h')` now expand. Metadata checks use a capability rooted at the pattern's longest literal directory prefix; missing or non-directory prefixes return no matches, and unresolvable symlink matches are skipped. | [Users' guide](users-guide.md) and [ADR-010](adr-010-scope-glob-capability-to-literal-prefix.md) |
| Command recipes | Existing scalar `command` recipes are unchanged. New YAML command lists are opt-in and run in declaration order with fail-fast semantics. | [Rules and recipes](users-guide.md#rules-and-recipes) |
| Manifest discovery | Optional target/action `description` values are shown by the new `netsuke help targets` command. Manifests without them and existing build output are unchanged. | [Users' guide](users-guide.md) |
Expand All @@ -38,10 +40,10 @@ Table: documented v0.1.0 additions, including `netsuke help targets`, and their

The convenience wrappers keep their signatures and their behaviour: the
child inherits the calling process's environment, and Ninja is resolved
exactly as before. Callers of `run_ninja` or `run_ninja_tool` need no change;
a caller that constructs `NinjaBuildRequest`/`NinjaToolRequest` directly must
now supply the required `stderr_mode: StderrMode` field, derived from the CLI
with `StderrMode::from_json_enabled(cli.json)`.
exactly as before. No caller using these wrappers needs to change to adopt this
release. A caller that constructs `NinjaBuildRequest`/`NinjaToolRequest`
directly must pass `options: &options` and supply the required
`stderr_mode: StderrMode` field.

## Opting into ordered command lists

Expand All @@ -66,8 +68,11 @@ names remain valid and resolve through that child `PATH`; supply an
absolute or otherwise resolved `program` only when executable selection
must stay isolated from the injected `PATH`.

Both request types borrow their fields, so one `CommandEnv` and one `Cli`
can serve several invocations. Worked examples live in the users' guide's
Both request types borrow their fields, so one `CommandEnv` and one
`NinjaProcessOptions` can serve several invocations. Direct request callers
now pass `options: &options` in place of `cli: &cli`; the adapter converts a
CLI working directory to UTF-8 and returns `io::ErrorKind::InvalidData` when
it cannot. Worked examples live in the users' guide's
"Drive Ninja with an explicit environment" section.

## Reusing cached configuration discovery
Expand Down
68 changes: 10 additions & 58 deletions src/runner/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,6 @@ mod dyndep_generation_telemetry;
mod dyndep_publication;
mod error;
mod reporter;

pub use error::RunnerError;

use crate::cli::{BuildArgs, Cli, Commands};
use crate::localization::{self, keys};
use crate::output_mode;
Expand All @@ -19,8 +16,9 @@ use crate::status::{LocalizationKey, PipelineStage, StatusReporter, report_pipel
use crate::{ir::BuildGraph, manifest, ninja_gen};
use anyhow::{Context, Result};
use camino::Utf8PathBuf;
pub use error::RunnerError;
use std::borrow::Cow;
use std::io::{self, IsTerminal};
use std::io::IsTerminal;
use std::path::Path;
use tracing::{debug, info};

Expand All @@ -42,14 +40,16 @@ pub const NINJA_ENV: &str = "NETSUKE_NINJA";
mod graph;
mod help;
mod ninja_content;
mod ninja_process_adapter;
mod path_helpers;
mod process;
pub use ninja_content::NinjaContent;
pub use ninja_process_adapter::{run_ninja, run_ninja_tool};
#[cfg(doctest)]
pub use process::doc;
pub use process::{
CommandEnv, MAX_RETAINED_DYNDEP_FILES, NinjaBuildRequest, NinjaToolRequest, StderrMode,
run_ninja_tool_with, run_ninja_with,
CommandEnv, MAX_RETAINED_DYNDEP_FILES, NinjaBuildRequest, NinjaJobCount, NinjaProcessOptions,
NinjaToolRequest, StderrMode, run_ninja_tool_with, run_ninja_with,
};

use dyndep_publication::{materialize_dyndep_bundle, prune_dyndep_bundle};
Expand Down Expand Up @@ -144,56 +144,6 @@ fn run_with_ninja_program_resolver(
dispatch::execute(cli, command, &context)
}

/// Invoke the Ninja executable with the provided CLI settings.
///
/// Forwards the job count and working directory and specifies the temporary
/// build file. Child output follows the `stderr_mode` policy derived from the
/// CLI's JSON diagnostic setting: `StderrMode::Suppress` drains both child
/// streams to a sink so JSON output stays machine-readable, while
/// `StderrMode::Forward` relays them to the user.
///
/// # Errors
///
/// Returns an [`io::Error`] if the Ninja process fails to spawn, the standard
/// streams are unavailable, or when Ninja reports a non-zero exit status.
pub fn run_ninja(
program: &Path,
cli: &Cli,
build_file: &Path,
targets: &BuildTargets<'_>,
) -> io::Result<()> {
run_ninja_with(&NinjaBuildRequest {
program,
cli,
build_file,
targets,
env: &CommandEnv::inherit(),
stderr_mode: StderrMode::from_json_enabled(cli.json),
})
}

/// Invoke a Ninja tool (e.g., `ninja -t clean`) with the provided CLI settings.
///
/// Forwards the job count and working directory and specifies the build file.
/// Child output follows the `stderr_mode` policy derived from the CLI's JSON
/// diagnostic setting: `StderrMode::Suppress` drains both child streams to a
/// sink, while `StderrMode::Forward` relays them to the user.
///
/// # Errors
///
/// Returns an [`io::Error`] if the Ninja process fails to spawn, the standard
/// streams are unavailable, or when Ninja reports a non-zero exit status.
pub fn run_ninja_tool(program: &Path, cli: &Cli, build_file: &Path, tool: &str) -> io::Result<()> {
run_ninja_tool_with(&NinjaToolRequest {
program,
cli,
build_file,
tool,
env: &CommandEnv::inherit(),
stderr_mode: StderrMode::from_json_enabled(cli.json),
})
}

fn on_task_progress_callback(reporter: &dyn StatusReporter) -> impl FnMut(u32, u32, &str) + '_ {
move |current: u32, total: u32, description: &str| {
reporter.report_task_progress(current, total, description);
Expand Down Expand Up @@ -227,11 +177,12 @@ fn handle_build(cli: &Cli, args: &BuildArgs, context: &ExecutionContext<'_>) ->
)
};
if context.progress_enabled {
let options = ninja_process_adapter::ninja_process_options(cli)?;
let mut on_task_progress = on_task_progress_callback(context.reporter);
process::run_ninja_with_status(
process::NinjaBuildRequest {
program: context.ninja_program,
cli,
options: &options,
build_file: build_path,
targets: &targets,
env: &CommandEnv::inherit(),
Expand Down Expand Up @@ -290,11 +241,12 @@ fn handle_ninja_tool(
)
};
if context.progress_enabled {
let options = ninja_process_adapter::ninja_process_options(cli)?;
let mut on_task_progress = on_task_progress_callback(context.reporter);
process::run_ninja_tool_with_status(
process::NinjaToolRequest {
program: context.ninja_program,
cli,
options: &options,
build_file: build_path,
tool: tool.name,
env: &CommandEnv::inherit(),
Expand Down
Loading
Loading