The bill is what catches it. One of my products is spending money it should not be spending, a steady drip, every single day.
I trace it back to a background job. A piece of the product that runs on its own, around the clock, checking in and calling the AI once an hour. It never sleeps and it never asks. Roughly ten dollars a day, whether I am using the product or not.
Here is the part that gets me. I turned that job off. I remember doing it. I open my copy of the code and there it is, switched off, exactly like I left it.
So why is it still running.
I point the agents at the gap. They find it. The version actually running is an older one. My off switch lives in the code on my desk. It never made it to the copy that is live. The app spending my money is not the app I edited.
One command and it stops. The drip ends that afternoon.
Then I do the part that matters more than the fix. I have the agents write a rule, so a job that can spend money on its own can never again run without a working off switch on the live system. Not the one in my editor. The one that is actually billing me.
Learnings
Editing a file is not the same as shipping it. I knew that in the abstract. The bill taught me it for real. The off switch I was so sure about was sitting in a copy nobody was running, while the live product kept happily spending.
The lesson I took: anything that can spend money on its own needs an off switch that works on the live system, and you prove it works before it ships, not after the bill shows up. A fix you cannot see running is a fix you have not made yet.