For most of a year, I helped build a large data platform. AI agents wrote most of the code. We told them what to build and checked what they made.
My boss designed the system. My job was to turn that design into something that runs. This is what I learned.
- 4,900 commits in 90 days
- 630 merged pull requests in 90 days
- 11,000 automated tests
Agents are fast. They are also wrong, with confidence.
An agent will tell you the work is done and the tests pass. Sometimes that is true. Sometimes the tests were skipped, or the test could never fail. Once an agent told me 21 checks passed. All 21 had been skipped.
So the real work moved. I stopped being the person who writes code. I became the person who builds the checks that catch a wrong answer.
What worked
- A test must fail when the feature is missing. If a test still passes with the feature removed, it tests nothing. We made this a rule and checked it by machine.
- One merge at a time. Many agents work in parallel, but only one change may merge at once. This ended most of the breakage between changes.
- Cheap checks first, full checks last. A draft gets a fast, small test run. Only a finished change gets the full run.
- Write down every mistake. After each failure, I wrote one short rule so it could not happen the same way again. I now have a few hundred of them. The agents read them at the start of each session.
- One source of truth. One file holds the name of every item the system can report. Tests fail the build if the code and that file differ.
What did not work
- Review is the limit. Agents can open ten pull requests in an hour. A person must still read them. We got stuck with a long queue more than once.
- Most of the work is fixing. In my last 90 days, 44% of my commits were fixes and only about 12% were new features. Speed at the start becomes repair work later.
- The process becomes a product. The checks, the queue and the rules need care too. Nobody sees that work when it goes well.
- Agents do not get tired. I do. They can run all night, and it is easy to follow them. I did, and my health paid for it.
Every part of the project still needs an owner.
What I would do again
- Build the checks before you add more agents. More agents without checks only make mistakes faster.
- Count review time, not code time. Plan the work by how much a person can read and judge in a day.
- Keep the scope small. Agents make it feel possible to build everything. You still need people to own everything you build.
- Set a stop time for the humans. Let the agents work at night. Read the results in the morning.
- Share knowledge across the project so every area has backup. The project should not depend on anyone staying well.
What I am now
I used to ask if I was still a real developer, because an AI writes most of my code. Someone has to decide what gets built, prove that it works, and answer for it when it breaks. That is the job now, and it is a real one.