Add auto-improvement classification blog draft#3084
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Lotte-Verheyden
left a comment
Lotte-Verheyden
left a comment
There was a problem hiding this comment.
I mainly left comments on writing style and detailed content, but in general I think the story/framing could be stronger.
Currently it reads as "we tried this setup, but didn't put enough time in looking at the dataset before starting, but we still want to discuss the results, even though they aren't very good (because of the dataset)". My main thought as a reader would be "you should have done it again with a better dataset before writing about it".
--> there is still a lot of interesting stuff in here, but because the story/framing isn't there, it doesn't feel intentional, and it's not clear what the reader should take from this.
==> see my comment at the end of the post, I actually think your paragraph at the end makes a very good case for why sharing this information does make sense, but I'd frame it like that from the beginning: "we just wrote about how lack of human judgement in the AI engineering loop leads to poor agents, here is the living proof of that, even with the newest, smartest models" (also, even though the dataset looked good at first glance, digging deeper manually revealed some shortcomings). Readers will go into the article with a different mindset, which makes more sense imo
| --- | ||
| title: "97% on train, 82% on test: auto-improvement loops have a validation gap" | ||
| date: 2026/06/10 | ||
| description: "Claude Fable 5 had just come out, so we used it to run an auto-improvement loop end to end. It improved train accuracy fast. What it really taught us was how much a validation split matters." |
There was a problem hiding this comment.
the word "matters" has become an AI word for me
There was a problem hiding this comment.
could say "the importance of a good validation split"
|
|
||
| <BlogHeader | ||
| title="97% on train, 82% on test: auto-improvement loops have a validation gap" | ||
| description="Claude Fable 5 had just come out, so we used it to run an auto-improvement loop end to end. It improved train accuracy fast. What it really taught us was how much a validation split matters." |
There was a problem hiding this comment.
I would address that steipete is talking about personal coding agents, and Langfuse generally is used for agents that you deploy for users
|  | ||
| </Frame> | ||
|
|
||
| We designed a loop and gave Claude Fable 5 a classification task: a train/test split in [Langfuse Datasets](/docs/evaluation/experiments/datasets), a prompt in [Prompt Management](/docs/prompt-management/get-started), and a goal: hit 95% accuracy or stop at 15 runs. Train accuracy went from 78% to 97% in four runs. Test performance barely moved. The 11 remaining errors on test were shared across every prompt variant. |
There was a problem hiding this comment.
I'd not mention too many specifics here, and instead mention this in the "the task and setup" section. This paragraph could become
We designed a loop and gave Claude Fable 5 a classification task. The loop did its job, but it surfaced a dataset problem: while train accuracy improved, test performance barely moved.
| - a train split with 200 labeled examples and a held-out test split with 100 in [**Langfuse Datasets**](/docs/evaluation/experiments/datasets) | ||
| - a prompt in [**Prompt Management**](/docs/prompt-management/get-started) | ||
| - a small runner built with [**Langfuse Experiments via the SDK**](/docs/evaluation/experiments/experiments-via-sdk) | ||
| - `gpt-4o-mini` as the task model<sup id="fnref-model-choice"><a href="#fn-model-choice">1</a></sup> |
There was a problem hiding this comment.
I'd explain the improvement loop before this, and explicitly mention that there is a task model, used for the classification task, and an optimizer model, that runs experiments to improve that classification task. If not explained people might be confused because you mentioned using Claude Fable 5 in the intro, and here it mentions gpt-4o-mini
There was a problem hiding this comment.
also, the o renders weird, not sure why though?
| _Suggested platform screenshot: the Langfuse workbench for this loop, showing the dataset, prompt, and experiment setup together._ | ||
|
|
||
| <Frame fullWidth> | ||
|  |
There was a problem hiding this comment.
would make the formatting better here
- "revise and run again" overlaps with TRAIN LOOP box edge
- "pass ->" too close to the revise prompt box
There was a problem hiding this comment.
I'd also try to remove the bottom line of text in each box. It feels a bit noisy now
|
|
||
| **1. A real validation split.** We had train for fitting and test for the final check, but nothing in between. So the loop selected prompt versions on train accuracy alone. | ||
|
|
||
| **2. More repeated edge cases.** The hard errors clustered around a few label boundaries, especially Information Retrieval vs. Databases, Human-Computer Interaction vs. Computers and Society, and subject vs. representation for audio papers. A stronger benchmark would have forced new rules to prove themselves across multiple similar cases. |
There was a problem hiding this comment.
"The hard errors clustered around a few label boundaries, especially Information Retrieval vs. Databases, Human-Computer Interaction vs. Computers and Society, and subject vs. representation for audio papers." --> The section feels like a conclusion section to me, I don't expect to read new information here. I'd discuss these specific errors earlier and then reference them here
There was a problem hiding this comment.
benchmark --> dataset (we call it dataset earlier)
There was a problem hiding this comment.
"forced new rules to prove themselves across multiple similar cases." --> this sounds contradictory, what exactly does this mean?
|
|
||
| **2. More repeated edge cases.** The hard errors clustered around a few label boundaries, especially Information Retrieval vs. Databases, Human-Computer Interaction vs. Computers and Society, and subject vs. representation for audio papers. A stronger benchmark would have forced new rules to prove themselves across multiple similar cases. | ||
|
|
||
| **3. Clearer policy for ambiguous papers.** Some of the shared errors look genuinely arguable. If the benchmark wants one exact label, it needs sharper tie-break rules, better canonical examples, and maybe even an `unsure` or multi-label policy. |
There was a problem hiding this comment.
I feel like this point and point 2 are about the same thing, maybe cut one of them?
|
|
||
| **3. Clearer policy for ambiguous papers.** Some of the shared errors look genuinely arguable. If the benchmark wants one exact label, it needs sharper tie-break rules, better canonical examples, and maybe even an `unsure` or multi-label policy. | ||
|
|
||
| That is the real lesson: the loop did its job. It surfaced, quickly, that the next bottleneck was not another prompt tweak. It was the dataset. |
|
|
||
| ## Where this is actually useful today | ||
|
|
||
| None of this means "do not automate the loop." It means: automate the inner loop, own the outer one. |
There was a problem hiding this comment.
"automate the inner loop, not the outer one": what do you mean by that? In my head the 2 loops are
- the auto-improvement loop that Fable runs (outer)
- classification loop on the dataset by gpt 40 (inner)
(but that wouldn't make sense given this post is about auto improvement)
There was a problem hiding this comment.
Ah reading further I see what you mean with each. I wonder if the human-owned part is actually a loop though (the post also didn't describe it as a loop), would actually cut the inner vs outer loop here, I think it would confuse people
| - **Agent-owned:** running experiments, scoring, per-error annotation, drafting hypothesis-driven prompt revisions, diffing errors across runs, flagging plateaus | ||
| - **Human-owned:** the target function, including the validation and held-out test data nobody optimizes against, dataset composition, when to restart with different constraints, and when to stop | ||
|
|
||
| As we argued in [AI is eating the AI engineering loop](/blog/2026-06-09-ai-is-eating-ai-engineering), the mechanics aren't the hard part anymore. This experiment shows what the hard part actually is: the target function, the dataset, and the judgment calls nobody automates against. |
There was a problem hiding this comment.
Reading this, I would actually frame this post as an illustration of how lack of human judgement in parts of the AI engineering loop result in suboptimal agents.
2 participants
Summary