Fix pipeline prefill short-prompt wedge + dashboard kv-backend display - #104

Merged

ttupper92618 merged 32 commits into

mainfrom

fix-pipeline-prefill-short-prompt-wedge

Apr 9, 2026

Commits on Apr 8, 2026

Fix pipeline_parallel_prefill short-prompt wedge + dashboard kv-backend display

Thomas Tupper
and
claude
committed
3259322
Temporarily bypass LLM warmup for debugging
Thomas Tupper
committed
edf7145
Add pipeline wrapper hang instrumentation
Thomas Tupper
committed
70e4192
Handle closed worker info event stream
Thomas Tupper
committed
430462e
Allow warmup planning to tolerate ready peers
Thomas Tupper
committed
cc0c033
Make Gemma warmup more representative
Thomas Tupper
committed
b91a99b
Test Gemma 4 prompts without empty thought suffix
Thomas Tupper
committed
4c6000f
Trace warmup and first live request shapes
Thomas Tupper
committed
2e4fdf8
Use hello warmup prompt for MLX isolation
Thomas Tupper
committed
66392df
Restore sampler settings for warmup bisect
Thomas Tupper
committed
e252ccc
Restore warmup instructions for bisect
Thomas Tupper
committed
1b4d3ee
Revert "Restore warmup instructions for bisect"
Thomas Tupper
committed
f55a421
Disable pipeline prefill mode for short warmup path
Thomas Tupper
committed
b206f87
Retry warmup instructions after prefill fix
Thomas Tupper
committed
a7be93f
Retry longer warmup content after prefill fix
Thomas Tupper
committed
dab0b00
Revert "Retry longer warmup content after prefill fix"
Thomas Tupper
committed
12e3cdd
Try neutral padded warmup content
Thomas Tupper
committed
16d5a1c
Clean up stale runners on node timeout
Thomas Tupper
committed
1fd6824
Raise warmup output budget for bisect
Thomas Tupper
committed
bc0c7ae
Add runner lifecycle debug logging
Thomas Tupper
committed
3d54e87
Make pipeline warmup prompt length configurable
Thomas Tupper
committed
a22effd
Make warmup instructions opt-in for debugging
Thomas Tupper
committed
4520e61
Force minimal warmup for pipeline models
Thomas Tupper
committed
f1df109
Document pipeline warmup policy and debug env vars
Thomas Tupper
committed
097685e
Address high-severity PR review findings
Thomas Tupper
committed
6ae7078
Tighten warmup skip and add review regression tests
Thomas Tupper
committed
431559c
Bound warmup decode to the first token
Thomas Tupper
committed
2302c41

Commits on Apr 9, 2026

Tighten warmup planner and debug helpers
Thomas Tupper
committed
e7a1d59
Tighten env precedence for warmup and debug config
Thomas Tupper
committed
9163d8d
Normalize invalid KV backend values in config
Thomas Tupper
committed
33f8c0f
Keep warmup cancel checks from becoming too frequent
Thomas Tupper
committed
3312c29
Deduplicate env precedence and KV backend validation
Thomas Tupper
committed
2cb8e00

Navigation Menu

Uh oh!

Fix pipeline prefill short-prompt wedge + dashboard kv-backend display - #104

Fix pipeline prefill short-prompt wedge + dashboard kv-backend display#104
ttupper92618 merged 32 commits into
mainfrom
fix-pipeline-prefill-short-prompt-wedge

Commits on Apr 8, 2026

Fix pipeline_parallel_prefill short-prompt wedge + dashboard kv-backend display

Temporarily bypass LLM warmup for debugging

Add pipeline wrapper hang instrumentation

Handle closed worker info event stream

Allow warmup planning to tolerate ready peers

Make Gemma warmup more representative

Test Gemma 4 prompts without empty thought suffix

Trace warmup and first live request shapes

Use hello warmup prompt for MLX isolation

Restore sampler settings for warmup bisect

Restore warmup instructions for bisect

Revert "Restore warmup instructions for bisect"

Disable pipeline prefill mode for short warmup path

Retry warmup instructions after prefill fix

Retry longer warmup content after prefill fix

Revert "Retry longer warmup content after prefill fix"

Try neutral padded warmup content

Clean up stale runners on node timeout

Raise warmup output budget for bisect

Add runner lifecycle debug logging

Make pipeline warmup prompt length configurable

Make warmup instructions opt-in for debugging

Force minimal warmup for pipeline models

Document pipeline warmup policy and debug env vars

Address high-severity PR review findings

Tighten warmup skip and add review regression tests

Bound warmup decode to the first token

Commits on Apr 9, 2026

Tighten warmup planner and debug helpers

Tighten env precedence for warmup and debug config

Normalize invalid KV backend values in config

Keep warmup cancel checks from becoming too frequent

Deduplicate env precedence and KV backend validation