Post

After six months of watching our engineering team deep in the AI trenches, I’m more convinced than ever: AI isn't just a nice-to-have in incident management—it’s a necessity.

As AI enables faster software development and systems become harder to fully understand, we’re left with a growing problem: how do we safely operate what we no longer fully grok?

My take: the only viable solution is to use AI to support the people operating these systems. It’s not unlike modern aircraft—too complex for a human alone, so we rely on automation to augment the pilot, not replace them.

But there’s a catch. Lisanne Bainbridge highlighted it back in 1983 in her paper "Ironies of Automation": the more we automate, the more we deskill.

People get good at watching systems, not running them. Then something breaks, and we expect them to swoop in and save the day, with little context and limited practice.

At incident, we’re trying to build in the opposite direction. Instead of replacing people, we’re designing AI to show up like your best engineer. Not the brilliant jerk who fixes the thing while you're grabbing a coffee, but the one that helps you understand, decide, and act when things are on fire.