Sinex

Local-first event capture for your machine. Query your digital history like a database.

Quick Start · Architecture · Security · Deployment & Operations

Project overview · Roadmap and work graph

---

What Is Sinex?

Sinex captures local activity as typed, timestamped events and stores them in an append-only PostgreSQL event log. The runtime is built around Rust services, NATS JetStream transport, and a NixOS deployment surface.

The system is meant for grounded local analytics, replayable derived state, and operator-visible automation, not opaque cloud processing.

Example payoff:

-- What was I researching when that build failed?
SELECT v.url
FROM commands c
JOIN visits v ON v.ts BETWEEN c.ts AND c.ts + interval '5 min'
WHERE c.command LIKE 'cargo test%'
  AND c.exit_code = 1
  AND v.domain = 'stackoverflow.com';

Current Surfaces

Surface	Current Owner
Capture	source contracts over staged materials, input-shape adapters, and parsers under crate/sinexd/src/sources/
Query / control	sinexd::api + sinexctl
Persistence	sinexd::event_engine + PostgreSQL
Derived state	sinexd::automata and replay-aware stream runtime
Deployment	NixOS modules + systemd
Runtime extension	inline sinexd runtime support and source/automaton traits

Architecture

The capture layer uses a staged-source parser substrate: source material is registered, an input-shape adapter enumerates records or bytes, and a parser emits material-provenance events. See crate/sinexd/docs/sources/staged_source_parser_substrate.md. The diagram below shows the deployed runtime shape. NATS is the event transport for live producers and derived services; staged material parsing is hosted by sinexd and still enters canonical persistence through the event engine.

sinexd::sources    sinexd::automata      Clients
  fs, terminal,      analytics,           CLI, browser
  desktop, system,   automata        extension
  browser, exports
       │                 │                    │
       ▼                 ▼                    │
  ┌────────────────────────────────────────┐  │
  │         NATS JetStream                 │  │
  │       (event transport)                │  │
  └──────────────┬─────────────────────────┘  │
                 │                            │
                 ▼                            │
        ┌──────────────────┐                  │
        │ sinexd           │                  │
        │ ::event_engine   │ validate, persist │
        └───────┬──────────┘                  │
                │                             │
                ▼                             │
        ┌────────────────┐                    │
        │   PostgreSQL   │ TimescaleDB,       │
        │   + extensions │ pgvector, schemas  │
        └───────┬────────┘                    │
                │                             │
                ▼                             │
        ┌──────────────────┐                  │
        │ sinexd           │◄─────────────────┘
        │ ::api            │ auth, rate limits
        │   JSON-RPC       │
        └──────────────────┘

Core Invariants

• canonical persistence flows through sinexd::event_engine
• core.events is append-only; corrections become new events with provenance
• derived events carry source, temporal, and replay metadata
• UUIDv7 IDs provide ordering; ts_orig and ts_coided are distinct and load-bearing
• blobs are content-addressed and referenced stably
• long-running replay/lifecycle work is recorded in operations_log

Operating Model

• services run as separate systemd units with NixOS-managed configuration
• observability is journald-first; service logs are part of the event universe
• runtime modules and derived automata recover through checkpoints and replay
• replay, archive, and restore are explicit control-plane operations
• direct DB access is diagnostic; the normal control/query boundary is sinexd::api

Stack

• Rust
• PostgreSQL 18 + TimescaleDB + pgvector + pg_jsonschema
• NATS JetStream
• NixOS modules + systemd hardening

Quick Start

git clone https://github.com/sinity/sinex.git
cd sinex
direnv allow  # loads the flake devShell and puts xtask on PATH

xtask infra start
xtask run core --logs
xtask run list
sinexctl events recent -n 10

Development

Start with the canonical repo workflow docs:

• contributing workflow: CONTRIBUTING.md
• testing workflow: TESTING.md
• local runtime loop and devshell smoke: xtask/docs/devshell-runtime.md
• xtask/tooling reference: xtask/docs/README.md
• sandbox harness details: xtask/docs/sandbox/README.md

Deployment & Operations

The canonical deployment surface is the NixOS module tree under services.sinex. Stable operational guidance lives here and in nixos/modules/README.md. There is no separate top-level operations runbook anymore.

Hardening defaults that are already part of the repo:

• API RPC is TLS-only and non-loopback binds require mTLS policy
• managed long-running units and helper/maintenance oneshots use systemd sandboxing
• managed local NATS now has typed server TLS under services.sinex.nats.tls.*
• managed local NATS now has typed subject-level authz for the current shared runtime identity under services.sinex.nats.authorization.sharedClient.*
• shared client transport still lives under services.sinex.runtime.nats.{servers,tls,auth} and is exported to all managed services automatically

Conventional secret names that the module now resolves automatically through agenix:

• API admin token: sinex-api-admin-token
• local NATS server TLS: sinex-nats-server-cert, sinex-nats-server-key, sinex-nats-client-ca
• shared NATS client TLS/auth: sinex-nats-ca, sinex-nats-client-cert, sinex-nats-client-key, sinex-nats-client-creds, sinex-nats-client-nkey, sinex-nats-token
• compatibility aliases are also accepted for the NATS client path: nats-ca, nats-client-cert, nats-client-key, nats-client-creds, nats-client-nkey, nats-token

Common operator entrypoints:

xtask doctor
xtask status --summary
xtask infra status
journalctl -u sinexd -f

Deployment/host readiness proof (systemd units, schema, source-config validators) is owned by sinexd startup preflight and sinexctl/NixOS, not by xtask — see runtime-target boundaries.

Documentation

The documentation map groups the current architecture, capture, operator, deployment, and contributor contracts.

I want to...	Start here
Browse the documentation by concern	docs/README.md
Understand the system shape	README.md#architecture
Deploy and harden the common NixOS path	README.md#deployment--operations
Deploy on NixOS	nixos/README.md
Build a source or derived service	crate/sinexd/docs/sources/README.md
Understand event schemas	crate/sinex-db/docs/schema/event-taxonomy.md
Separate notes, typed records, graph, and artifacts	crate/sinex-primitives/docs/knowledge_boundaries.md
Define current-state projections for event-native domains	crate/sinex-primitives/docs/domain_reducers.md
Promote generated suggestions through human or policy authority	crate/sinex-primitives/docs/curation_authority.md
Expose read-only evidence to coding agents	crate/sinexctl/docs/mcp_readonly_server.md
Reason about replay evidence and source snapshots	crate/sinexd/docs/sources/evidence_lanes.md
Reason about large aggregate provenance	crate/sinexd/docs/automata/high_fan_in_lineage.md
Reason about runtime backpressure and loss policy	crate/sinexd/docs/runtime_qos.md
Suppress live capture through private mode	crate/sinexctl/docs/private_mode.md
Add a staged personal-export parser	crate/sinexd/docs/sources/adding_staged_export_parser.md
Drain and recover source material cleanly	crate/sinexd/docs/sources/source_drain.md
Snapshot or restore runtime state	crate/sinexctl/docs/state_snapshot.md
Configure PostgreSQL backup/restore	crate/sinex-db/docs/backup_restore.md
Decide which surface owns a runtime or data concern	.github/authority-surfaces.md
Integrate an external tool or sibling project	crate/sinexd/docs/sources/integration_authority.md
Work on repo workflow or verification	CONTRIBUTING.md, TESTING.md
Work on the CLI/tooling loop	xtask/docs/README.md

Active and proposed work belongs to the committed Beads graph, not historical GitHub Issues. Browse it on the web board or query it locally with bd ready and bd list.

Security

Threat model shorthand:

• trusted single-user local host
• runtime modules submit over NATS; the sinexd API is the hardened external boundary
• canonical persistence stays single-writer through sinexd::event_engine
• host full-disk encryption and capture-time privacy controls are the intended baseline

Current controls:

• typed payload validation with schema checks
• TLS-only API RPC; non-loopback binds require stronger transport policy
• bearer-token auth with constant-time comparison
• per-token rate limiting
• structured request access audit logs on RPC, SSE, and native-messaging dispatch paths
• systemd hardening from the NixOS deployment layer, including helper/maintenance units
• typed managed-NATS TLS and subject-level authorization surfaces in the NixOS module

---

_{Built for personal use. Not yet production-ready for general deployment.}