My AI Agent Took Down My Production Site — Then Found the Bug in Its Own Chat History

Bad gateway. My own site — gone. Two words on a phone screen, doing something that should not have been possible to a Friday evening. Let me back up.

I run my site almost entirely through conversations with an AI coding agent. It writes the features, it writes the blog posts, it spins up a staging environment and runs smoke tests, it deploys to production when I say the word. Most days that's a genuinely good arrangement. This is the story of the day it wasn't — and, more interestingly, the story of what happened after: a real, step-by-step case study in AI debugging and what agentic AI troubleshooting actually looks like once something has gone wrong.

The message I wasn't expecting

I'd asked for something small: a new blog post introducing the speed-test comparison feature we'd shipped a few days earlier. Routine work. Here's the link to the actual speed test. The agent wrote it, opened a pull request, and I merged it — quickly, pulled over on the side of the road on my way home from the office. My site is set up so a blog post doesn't need a redeploy; merging the PR is the deploy. I didn't open the site to check. I just merged and kept driving.

It was a while before I got home. When I finally opened the browser to see how the new post actually looked, what I got instead was a bad gateway error.

A confident answer, delivered a little too fast

I asked the obvious question: is the container dead? It checked, and came back with a full explanation. The container wasn't just stopped, it said — it was completely gone, not even listed as stopped. It found that the host machine had rebooted two days earlier, and reasoned that the container's restart policy must never have reattached after that reboot. Clean story, plausible mechanism, delivered with total confidence.

It was a fluently written falsehood.

It also happened to be wrong. Not maliciously, not carelessly exactly — it was a real, common failure mode, and the evidence it had gathered (container gone, host rebooted two days prior) fit that story well enough to stop looking. It just hadn't checked the one source of truth that would have told it immediately: how long real visitors had actually been getting errors.

The question that reset the clock

I didn't debate the theory. I just asked Claude Code to check the nginx logs and tell me when traffic had actually stopped being served. That's it — not "you're wrong," just "show me the evidence."

The access logs didn't agree with the two-day story at all. The last clean, successful request landed minutes before the first 502, and the first 502 landed that same afternoon, not two days earlier. Real visitors — a returning reader loading a blog post, a crawler indexing a page — had been served correctly right up until one specific minute, and broken from that minute on. The reboot two days prior had nothing to do with it. Whatever happened, happened during the session we were already in.

That's a meaningfully different incident than the one it had described five minutes earlier. Not "something rotted quietly for two days." Something specific happened, at a specific minute, and I was very possibly not the only one in the room when it did.

I'm not reprinting the actual log lines here — access logs carry real visitor IPs and exact paths, and those aren't mine to publish. But the shape of it looked something like this (reconstructed, with placeholder addresses):

A clean run of 200s, then a wall of 502s starting at one precise second, then clean 200s again the moment the real fix landed. That shape is what made the "quietly broken for two days" theory fall apart on sight.

The question I actually wanted answered

So I asked the question I'd been circling: does this chat have any record of tearing production down in that window?

That's the part of this story I still find genuinely interesting. I wasn't asking it to search logs this time. I was asking it to audit itself — to go back through its own record of every command it had run in our conversation and check whether one of them explained the timestamps we'd just pulled from nginx.

It could, and it did. A few minutes into the same session, while standing up a staging environment to test that blog post, it had run a routine deploy command — the same command it had run successfully plenty of times before. This time it had dropped one flag that scopes staging to its own isolated project name. Without that flag, staging and production both fell back to the same default project name, and both compose files happened to define a service with the identical name. The staging command didn't fail loudly. There was no explosion, no siren, no red text screaming across the terminal — just one clean, silent headshot: it recreated my live production container in place with staging's configuration, and nothing in the room so much as flinched. A little later, the getaway car: a perfectly routine "tear down staging now that we're done testing" command walked back in and finished the job, removing that same container for good, under the same mistaken identity. No error at any point. No smoking gun in the output. Just a project name colliding with itself, once understood, standing in plain sight the whole time — the quietest hit I've ever watched happen in real time.

It laid all of this out with receipts: the exact minute from the logs, the exact command from its own history, and the exact line of output at the time that, in hindsight, had been the tell — a message saying a container was being "recreated," printed and scrolled past without anyone, human or AI, clocking what it meant in the moment.

What actually broke, in plain terms

This wasn't a hack, a crash, or bad code shipping to production. It was a naming collision, caused by the non-deterministic, probabilistic design of LLMs. Two deploy configurations, one for the live site and one for a disposable test copy, both lived in the same folder. Neither one had been told what to call itself, so both quietly fell back to the folder's own name as their identity, and both happened to describe a piece of the infrastructure with the same generic label. Run the test one without stating its name out loud, and it can't tell itself apart from the real one. That's the whole bug. No exploit, no edge case in the AI's reasoning — a small human-written convention, documented in a comment at the top of the very file that got run, that got skipped once and cost thirty-seven minutes of downtime.

What I actually take away from this

Not "don't let AI touch production." I still do, most days, deliberately. What changed is what I now expect from myself while doing AI debugging, and what good agentic AI troubleshooting actually requires from the human in the loop:

• Its first explanation is a hypothesis, not a verdict. The reboot story was coherent and confidently delivered, and it was still wrong. Confidence and correctness are two different signals; ask for the second one.
• Logs beat narratives, every time. One request — "show me the actual evidence" — did more to fix the diagnosis than any amount of back-and-forth reasoning would have.
• An AI agent can put itself on trial, if you ask it to. The most useful question I asked wasn't about the server at all — it was "what did you do." Most engineers doing a postmortem don't have a verbatim transcript of every command that ran; here, that transcript already existed, and just needed to be interrogated.
• Naming conventions are load-bearing, not decoration. This is true whether a human or an AI is at the keyboard. A staging environment that isn't unmistakably, forcibly separated from production by name will eventually collide with it — the only question is who's driving when it happens.
• Outward-facing, hard-to-reverse actions deserve a second look before they run, not just a good explanation after. The fix here is cheap: a required project flag, checked and stated out loud before any command that touches a shared container name.

Frequently asked questions

Did the AI agent intentionally take the site down?

No. It was a side effect of a routine staging deploy, not anything adversarial or deliberate. Two Docker Compose files in the same repo resolved to the same default project name, so a normal staging command silently operated on the production container instead.

Why was the AI's first explanation for the outage wrong?

It pattern-matched: the container was gone, the host had rebooted two days earlier, and "restart policy didn't reattach after a reboot" is a common, plausible failure mode. It was a reasonable guess made without first checking the evidence that would confirm or rule it out — the actual access logs. In plain terms, it was a fluently written falsehood.

How did checking nginx logs fix the diagnosis?

The access logs are ground truth for when real traffic actually succeeded or failed. They showed the last successful request and the first 502 minutes apart, today — not two days apart — which immediately ruled out the reboot theory and pointed at something that happened during that specific session.

What is a Docker Compose project name collision?

Docker Compose groups containers under a project name, defaulting to the current folder name unless a docker-compose.yml sets one or -p is passed on the command line. If two compose files in the same folder both use the default and both define a service with the same name, commands run against one can recreate or delete the container that belongs to the other.

Should I let an AI agent run commands against production?

Treat it the way you would a capable but literal new hire: useful for real work, but outward-facing and hard-to-reverse actions deserve a second pair of eyes, and its first explanation for anything is a hypothesis to verify against logs, not a verdict to accept.

What is agentic AI troubleshooting?

It's the practice of diagnosing a real incident together with an AI agent that can run commands and read logs itself, rather than only asking it questions. The skill isn't delegating the whole investigation — it's knowing when to redirect the agent toward primary evidence (like access logs) and when to ask it to audit its own actions, the way a human incident review would.

What I've actually changed, to avoid a repeat

Takeaways are cheap. Here's what's actually different in how this site gets deployed now:

• Give every environment its own name, in the file, not just in a flag. A project name that only exists because someone remembered to type -p will eventually be the time someone didn't. It belongs in the compose file itself, so the safe behaviour is the default one, not an opt-in.
• Make a collision loud, not silent. The scariest part of this incident wasn't the mistake, it was that nothing complained. A staging command quietly recreating a production container should not look identical to a staging command doing exactly what it's supposed to.
• Don't rely on "I'll happen to check." I found this outage because I happened to open the site later that evening. That's not monitoring, that's luck. A simple uptime check that pings the homepage every few minutes and pings me back on a failure would have caught this in under five minutes instead of however long it actually ran.
• Treat a teardown command with the same respect as a delete command. Because that's what it is. down is not the gentle half of up and down — under the wrong project name it's the one that actually removes something.
• A fast merge from the side of the road is still a deploy. "It's just a blog post" is exactly the moment attention drops. The size of the change has nothing to do with the size of the blast radius if something unrelated happens to be collided with at the same time.
• Wire a health check into Claude Code itself, as a hook. A PostToolUse hook that fires after any command touching deploy or container state, runs a small shell script that curls the homepage and checks for a real 200, and pages me immediately over a real channel — Telegram, in my case — the moment it isn't. Had that existed already, this outage would have been caught within seconds of the command that caused it, not however long it took me to open a browser after work.
• Add a pre-flight gate, not just a post-mortem one. A PreToolUse hook that inspects any Docker Compose invocation before it runs, and blocks it outright if the resolved project name matches production's, turns "remembered to add a flag" from a discipline problem into something the tooling itself enforces. The safest fix here isn't a smarter agent — it's a dumber gate that doesn't care how it got called.
• Add a cron-backed watchdog that doesn't care who or what broke it. The hooks above only fire when Claude Code itself takes an action. A plain shell script on a schedule — curling every site this box serves and checking for a real 200 — catches downtime from any cause: a manual command gone wrong, a cert expiring, disk filling up, someone else entirely touching the box. One box, one box worth of sites, one script.
• But run the checker off the box, and only use the phone for the alert. A cron job on the same box that just went down can't tell anyone the box itself is down — it's a blind spot, not a complete fix. And a phone is a bad place to run the actual polling loop: iOS and Android both kill background processes to save battery, so a check that's supposed to run every few minutes will get throttled or silently skipped. The right split is an external check (a hosted uptime service, or a script on a second box) pushing a notification to the phone — Telegram, since that channel already exists here, or a purpose-built alert app like Pushover or ntfy.sh for something separated from personal chats.

The site's back up. The monkey's still holding the saw. I just don't hand it the branch I'm sitting on anymore without checking whose name is on it first.

One more thing

Everything above — the staging/prod discipline, the Docker Compose internals, reading nginx logs for ground truth instead of guessing, working an incident to an actual root cause instead of a plausible-sounding one — is the day-to-day of the work I do. I'm currently open to remote work: SDET / test automation, QA leadership, or DevRel-shaped roles, engaged via Upwork, Deel, or directly. If this post was useful to you and you're hiring, or you just want to talk shop: SDET / automation CV, DevRel CV, or LinkedIn.

If your own AI agent has ever confidently told you the wrong thing, I'd genuinely like to hear about it — and if this saved you from your own 3 a.m. project-name collision, send it to the one person on your team who still thinks "the AI wouldn't do that."

Related reading: The speed test comparison feature this session was actually about · How Sally the IT Intern Saved Her Employer From a Friday Disaster · Tools: Internet Speed Test · Network Health · HTTP Header Checker.