Five requests
took it down.
Never again, by design.

Everything was green. Nothing worked.

This platform was ours to begin with: a system I designed and built to replace a decade-old business system that ran on the company's own hardware and that its maker no longer supported. Warranty certificates, claims, dealer and distributor networks, escrow accounting, the sync with their core business software: all of it rebuilt from the ground up. Millions of certificates and a 136 GB database later, it still ran entirely on one hand-managed server: the website, the background work, the search, the scheduling, and every file, all on a single machine that hadn't been restarted in 283 days. One afternoon the platform went dark. The dashboard said everything was running, every light green. Customers got timeouts.

The diagnosis, delivered the same day: the platform could only work on five requests at a time, and all five were frozen, stuck waiting on a search function that had stopped answering. Five stuck requests (on a machine averaging 1.4% CPU, practically idle) equaled total outage. It was never a capacity problem. It was a design problem.

Fix what can be fixed. Design away the rest.

• Root cause, with receipts: traced the outage to its exact point of failure, identified the frozen search request that started the pile-up and the 5.8-million-row database scan hiding behind it, and stated plainly which parts of the evidence could not be confirmed. No hand-waving.
• Live detective work under pressure: examined the 136 GB live database piece by piece in the middle of the emergency, turning up a forgotten 1.8-million-row backup copy sitting in the live system, a database left open to a pile of individual laptops that should never have had access, and a hidden misconfiguration that could have silently stopped reports forever.
• Immediate cleanup, executed and independently re-verified: cloud cost and housekeeping fixes done, not just recommended. Oversized services trimmed to what's actually needed, abandoned pieces deleted, and a full set of automatic alerts put in place where there had been none.
• A judgment call that took discipline: ruled, against the obvious instinct, that this platform and its sister platform should not be merged, keeping it an independent product whose costs stay cleanly separated by client. The right merger is sometimes the one you refuse.
• The move, designed and routed properly: wrote the full plan for moving the platform: two candidate designs built separately and scored against each other, a phased plan where a practice environment that costs almost nothing when idle doubles as the dress-rehearsal stage for the move, a nine-step checklist for switchover day, and a way to switch back to the old system in minutes, with the old system preserved untouched as the source of truth.
• Then put it through the client's approval process instead of just doing it: the plan is moving through the client's own review and sign-off process, because a high-risk change to a live system that skips review is how you get the next outage.

The telling detail: the first thing the plan builds isn't the new live system. It's a practice environment that costs almost nothing when idle and exists so that switchover day is the second time everything runs, not the first.

In plain terms