Skip to content

feat: add NDJSON file sink - #29

Merged
PabloPardoGarcia merged 3 commits into
mainfrom
feat/ndjson-sink
Mar 27, 2026
Merged

PabloPardoGarcia merged 3 commits into
mainfrom
feat/ndjson-sink

Conversation

@PabloPardoGarcia

Copy link
Copy Markdown
Member

Summary

  • Adds NDJSONSink that writes generated events as newline-delimited JSON (one record per line)
  • Registered as "ndjson" in SinkFactory — usable via "type": "ndjson" in config
  • Preserves nested structures as-is (no flattening, unlike the CSV sink)
  • Includes docs page and tests with 100% coverage on the new sink

Usage

{
  "sink": {
    "type": "ndjson",
    "params": { "path": "output.ndjson" }
  }
}

Test plan

  • test_ndjson_sink_publish — single record written as one JSON line
  • test_ndjson_sink_bulk_publish — multiple records each on their own line
  • test_ndjson_sink_preserves_nested_structure — nested objects and arrays are not flattened
  • test_sink_factory — SinkFactory.create("ndjson", ...) returns NDJSONSink

🤖 Generated with Claude Code

Adds NDJSONSink that writes generated events as newline-delimited JSON
(one record per line), preserving nested structures without flattening.

- glassgen/sinks/ndjson_sink.py: new sink implementation
- glassgen/sinks/__init__.py: register "ndjson" in SinkFactory
- tests/test_sinks.py: tests for publish, bulk publish, nested structure
- tests/conftest.py: temp_ndjson_file fixture
- docs/app/sinks/ndjson/page.mdx: sink documentation page
- docs/app/sinks/page.mdx: add NDJSON to available sinks list

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@vercel

vercel Bot commented Mar 27, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
glassgen Ready Ready Preview, Comment Mar 27, 2026 3:19pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

Coverage

Test Coverage Report
FileStmtsMissCoverMissing
__init__.py60100% 
config.py791186%104–106, 108–109, 111–113, 115, 125–126
interface.py601181%14–16, 40–44, 54, 65–66
generator
   __init__.py50100% 
   batch_controller.py39392%16, 21–22
   duplication.py641281%34, 38–39, 41–43, 45–46, 48–50, 52
   generator.py67494%41, 54, 91, 108
   generators.py1032476%61, 118, 147–149, 151–154, 156–157, 159–161, 163, 170, 172–173, 176, 188, 191, 205, 258, 263
schema
   __init__.py40100% 
   base.py70100% 
   schema.py92792%50, 66, 76–77, 106–107, 132
   user_schema.py16193%30
sinks
   __init__.py140100% 
   base.py90100% 
   csv_sink.py47197%41
   kafka_sink.py281353%20–23, 27–28, 33, 36–38, 43–44, 47
   ndjson_sink.py250100% 
   webhook_sink.py260100% 
   yield_sink.py100100% 
TOTAL7018787% 

@github-actions

Copy link
Copy Markdown
Contributor

Coverage

Test Coverage Report
FileStmtsMissCoverMissing
__init__.py60100% 
config.py791186%104–106, 108–109, 111–113, 115, 125–126
interface.py601181%14–16, 40–44, 54, 65–66
generator
   __init__.py50100% 
   batch_controller.py39392%16, 21–22
   duplication.py641281%34, 38–39, 41–43, 45–46, 48–50, 52
   generator.py67494%41, 54, 91, 108
   generators.py1032476%61, 118, 147–149, 151–154, 156–157, 159–161, 163, 170, 172–173, 176, 188, 191, 205, 258, 263
schema
   __init__.py40100% 
   base.py70100% 
   schema.py92792%50, 66, 76–77, 106–107, 132
   user_schema.py16193%30
sinks
   __init__.py140100% 
   base.py90100% 
   csv_sink.py47197%41
   kafka_sink.py281353%20–23, 27–28, 33, 36–38, 43–44, 47
   ndjson_sink.py250100% 
   webhook_sink.py260100% 
   yield_sink.py100100% 
TOTAL7018787% 

@PabloPardoGarcia
PabloPardoGarcia merged commit 9ec0ec4 into main Mar 27, 2026
7 checks passed

This branch was successfully deployed

1 active deployment
Preview — 9618c86d Deployed Mar 27, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants