Skip to content

Array generator enhancements - #22

Merged
PabloPardoGarcia merged 7 commits into
glassflow:mainfrom
aditypan:fix/array_generator_schema
May 29, 2026
Merged

PabloPardoGarcia merged 7 commits into
glassflow:mainfrom
aditypan:fix/array_generator_schema

Conversation

@aditypan

@aditypan aditypan commented Mar 23, 2026 •

Copy link
Copy Markdown
Contributor

Enhanced the array generator to support data generation using other existing generators without failures. I used regex-based parsing to extract the parameter name, additional parameters, and count from expressions like "choice(red,blue,green), 2".

Previously, the array generator was expected to be used in the following format:

{
    "field" : "$array(choice, 2, red,blue,green)",
     "prices": "$array(price, 3, 1.2, 2.3, 3)"
}

However, the earlier implementation would fail for cases like the price generator shown above. The implementation has now been enhanced to correctly handle such edge cases.

Current Array generator format updated to:

{
       "choices": "$array(choice(red,blue,green), 4)",
        "prices": "$array(price(1.2, 2.3, 3), 2)",
        "nums": "$array(intrange(0,100), 5)",
}

Fixes : #20

@vercel

vercel Bot commented Mar 23, 2026

Copy link
Copy Markdown

@aditypan is attempting to deploy a commit to the GlassFlow Team on Vercel.

A member of the Team first needs to authorize it.

@PabloPardoGarcia
PabloPardoGarcia self-requested a review March 27, 2026 14:33

@PabloPardoGarcia PabloPardoGarcia left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution @aditypan !!

The fix is correct and the regex approach is a clear improvement over the old comma-split. The recursive reuse of _schema_dict_to_fields to parse nested generator params is elegant. However, the change silently breaks existing configurations that use the old flat syntax, and the test suite removes coverage for that syntax without replacement. These need to be addressed before merging.

Comment thread glassgen/schema/schema.py Resolved
Comment thread glassgen/schema/schema.py Resolved
Comment thread tests/test_schema.py Resolved
@PabloPardoGarcia PabloPardoGarcia added bug Something isn't working generator labels Mar 27, 2026
@PabloPardoGarcia
PabloPardoGarcia self-requested a review March 31, 2026 13:19
@PabloPardoGarcia

Copy link
Copy Markdown
Member

@aditypan could you fix the format issues flagged by the failing test?

Once that green I will merge the PR, thank you!

@aditypan

aditypan commented May 1, 2026

Copy link
Copy Markdown
Contributor Author

@PabloPardoGarcia I have fixed the Format Issues.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage

Test Coverage Report
FileStmtsMissCoverMissing
__init__.py60100% 
config.py791186%104–106, 108–109, 111–113, 115, 125–126
interface.py601181%14–16, 40–44, 54, 65–66
generator
   __init__.py50100% 
   batch_controller.py39392%16, 21–22
   duplication.py641281%34, 38–39, 41–43, 45–46, 48–50, 52
   generator.py67494%41, 54, 91, 108
   generators.py1032476%61, 118, 147–149, 151–154, 156–157, 159–161, 163, 170, 172–173, 176, 188, 191, 205, 258, 263
schema
   __init__.py40100% 
   base.py70100% 
   schema.py92792%50, 68, 74–75, 115–116, 141
   user_schema.py16193%30
sinks
   __init__.py140100% 
   base.py90100% 
   csv_sink.py47197%41
   kafka_sink.py281353%20–23, 27–28, 33, 36–38, 43–44, 47
   ndjson_sink.py250100% 
   webhook_sink.py260100% 
   yield_sink.py100100% 
TOTAL7018787% 

@vercel

vercel Bot commented May 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
glassgen Ready Ready Preview, Comment May 29, 2026 9:17am

Request Review

@PabloPardoGarcia
PabloPardoGarcia merged commit 42645ee into glassflow:main May 29, 2026
7 checks passed

This branch was successfully deployed

1 active deployment
Preview — 86bfae42 Deployed May 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working generator

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Array generator failing with sub generator's like price

2 participants