A 10,000-line vibe-coded project, fully refactored in 1h45 with a single prompt
How a fifteen-minute prompt put an untested project under the strictest quality control: a goal the agent checks itself, coverage first, a ratchet against shortcuts.
Translated from French with AI. Read the original.
I have a vibe-coded Java command-line project: 14 files, 9,580 lines, no tests, one 2,065-line file and about 870 static-analysis violations. I gave it a prompt that took me fifteen minutes to write. The agent worked alone for 1h45, from 00:42 to 02:27. The project now has 156 files, the largest at 379 lines, 680 tests, 99% of lines covered, and it passes a single command, ./verify, that chains eight automated checks.
Before starting the agent, I was certain of two things: nothing would be broken, and the result would go where I wanted. This article explains where that certainty came from.
The starting point: 14 files, 9,580 lines, no tests
The project audits GitLab repository activity using SonarQube data. It works, but everything sits in a few large files, including GitlabActivityAudit at 2,065 lines, and nothing checks that a change doesn’t damage something else. Static analysis reported about 870 violations.
What I asked for: one command that must turn green
The prompt describes, above all, a final state: the ./verify command must pass entirely. It chains Spotless for formatting, Checkstyle, Error Prone with NullAway, PMD, the tests with a coverage threshold of 90% of lines and 85% of branches, and a ratchet described below. The --full option adds CPD for duplicated code and PIT, a mutation test, with an 85% threshold.
Here is the prompt, shortened for reading (the original is longer):
- Cover the project with tests that allow a major refactoring.
- Use mutation testing to check that coverage.
- Run
./verifyto bring out every violation.- Fix them in whatever order you judge most relevant.
- Install
./verifyas git hooks and validate that they work.Save to
QUALITY_LOG.mdany questionable choice you have to make to pass the checks. Provide proof: coverage before, coverage after, any diff on the rules, the./verifyreport.
The order matters: coverage first, structure second. The QUALITY_LOG.md log lets the agent flag its questionable choices to me, more on that below. As for the proof requested at the end of the prompt, it fits on one page and is enough for me to review: the diff on the rules shows immediately whether the agent loosened a check.
What came out of the night
| Before | After | |
|---|---|---|
| Files | 14 | 156 (largest: 379 lines) |
| Tests | 0 | 680 |
| Lines covered | 0% | 99% (97% of branches) |
| Mutants killed (PIT) | not measured | 99% (3,392 of 3,433) |
| Static-analysis violations | about 870 | 0 |
| Code Health (CodeScene) | 6.0 | 9.8 |
The agent also found about twenty behaviours that look like bugs. I didn’t want them fixed during a refactoring. It documented them and pinned them with named tests, the behaviour stays unchanged, and I decide later.
Why I was sure nothing would break
Refactoring starts only once the behaviour is protected. The tests written first are characterization tests: they run the tool end to end against fake GitLab and SonarQube servers and compare its output with snapshots (using the Selfie library). If the refactoring changes one line of what the tool produces, a snapshot breaks.
99% coverage doesn’t prove those tests check anything: an executed line is not a verified line. That is the job of mutation testing. PIT modifies the code in small ways (a > becomes >=, a return is removed) and checks that at least one test fails. Out of 3,433 mutations, 3,392 were detected. The safety net was therefore tested before the first refactoring.
This is the method I call 3P and put into a skill: Protect the existing behaviour with quick and dirty tests, check the net by deliberately breaking something small, Prepare the ground with a refactoring that changes no behaviour, then Produce the feature with TDD. Here I only used the first two steps. But nothing stops you from using it to add a feature to any legacy code in the same way: protection first, novelty second.
Why the agent went where I wanted
The goal can be evaluated by the agent itself. It doesn’t need me to know where it stands: it runs ./verify, reads what fails, fixes it, and starts again. I described no target structure. The split into 156 files, with packages such as matching, sonar, gitlab, csv, console, naming, parallel or exclusions, came from the agent. What I fixed were the thresholds: Checkstyle’s and PMD’s, coverage, mutation.
Each iteration brings it closer to the final state or makes a test fail, and the agent chooses the path freely.
The ratchet: existing problems may stay, no new ones may enter
A measurable goal has a loophole: the agent can reach green by loosening the measure, by lowering a coverage threshold, adding a @SuppressWarnings, or disabling a test. The ratchet closes that door. The name comes from the ratchet wrench, which turns in one direction only. The rule fits in one sentence: existing problems may stay, no new ones are allowed, and nobody loosens a check quietly.
Concretely, a small tool compares the working tree with a git reference. A minimum threshold cannot go down, a maximum threshold cannot go up, a check cannot disappear. The number of suppression markers (@SuppressWarnings, NOPMD, CHECKSTYLE:OFF, @Disabled) cannot grow. An Error Prone rule switched off or downgraded to a warning is detected. A commit carrying a Ratchet-Override: line turns the failure into a warning: the exception stays possible, but it is a human decision, visible in the history.
During the night, the ratchet refused three new @SuppressWarnings that the agent had added.
The quality log: the agent flags its questionable choices
An agent working alone for 1h45 makes many decisions I will never see. The QUALITY_LOG.md section of the prompt asks it to record those that are questionable: a compromise, a shortcut, a place where it wasn’t sure. I read that log instead of 156 files.
The log served a second time. I ran an automated review on the result (CodeScene, a “clean coder” agent, an “architect” agent and a behaviour-change experiment) and compared its findings with the log. The gap between the two gave me new design rules: an ARCHITECTURE.md file and ArchUnit tests that enforce layers, isolation between slices of the code, no cycles between packages, and a single public class, the command. They are tests, so they run in ./verify like everything else. That second wave deserves its own article.
What this certainty doesn’t cover
The tests pin the behaviour as my fake servers simulate it. Of 3,433 mutants, 41 survived. And the twenty or so suspected bugs are still there, on purpose. The framework guarantees that the behaviour the tests observe doesn’t change. It doesn’t guarantee that the program is right.
They apply the same recipe
I’m not the only one seeing this. These published reports follow the same recipe: a goal the agent checks itself, and rules it cannot get around.
- Anthropic: a 100,000-line C compiler written by 16 agents, guided by existing test suites. The verifier must be almost perfect, otherwise the agent solves the wrong problem.
- OpenAI: a million lines in five months by three engineers, bounded by mechanical rules.
- Rakuten: seven hours of autonomy on vLLM, 99.9% numerical accuracy against the reference.
- Stripe (via InfoQ): more than 1,300 pull requests a week, with deterministic lint and tests and at most two rounds of continuous integration.
- Airbnb (via InfoQ): 3,500 test files migrated in six weeks instead of 18 months.
- Meta: 49% of tests generated from mutants add no line coverage, hence the value of mutation testing.
Which piece of work becomes reasonable now?
Turning 10,000 untested lines into tested, split and controlled code used to take a team weeks, so we didn’t do it. It took fifteen minutes of prompt and 1h45 of waiting. The physical laws of software development have changed, and our idea of what is worth attempting hasn’t caught up.
In your backlog, look for the jobs whose final state can be described in a verifiable way: every test of the new version green, a module covered at 90%, no violations left. Migrations, version upgrades, covering a legacy module and bringing old code into compliance often fall into that category. Tomorrow, what could you do with your backlog using this approach?
Going further
- The legacy-3p skill: Protect, Prepare, Produce.
- In the same series: Reviewing AI-written PRs sucks, Why we don’t delegate code review and How to delegate code review to agents.
Johan Martinsson
