Building with AI

Who Owns AI's Mistakes? Most Companies Can't Say.

October 21, 2026

Building with AI is Kevin Sandahl's weekly look at the latest in AI, for tech leaders who build with people in mind.

Welcome to the first edition of Building with AI. There's far more AI news every week than any of us has time to read, so each week I'm going to pick out the handful of stories I think actually matter for people running technology teams, and leave the rest alone.

This week one question kept coming up for me: when AI does the work and something goes wrong, who owns it? We talk a lot at Covalience about thinking and acting like owners, and a new Wharton survey suggests most companies haven't sorted this out yet. One team has, though, and the way they went about it is worth a look.

The Story

Who owns AI's mistakes? Most companies can't say.

On October 13, Wharton and the research firm GBK Collective published their annual survey of more than 1,000 managers and executives at large US companies. For most of them, using AI is now just part of the job. But when they were asked who is held responsible for a serious AI mistake, the answers were all over the place.

When AI working on its own makes a serious mistake, who is held most responsible?

28%

a governance or risk team

26%

the manager of the employee who deployed it

25%

the employee who deployed it

the vendor

1 in 5

managers and directors say nobody is clearly accountable, or don’t know

Source: Wharton Human-AI Research and GBK Collective, 2026 AI Adoption Report (managers and above)

No single answer gets to 30%. And the uncertainty is greatest in the middle of the organization, where most of the day-to-day work happens. One in five managers and directors said nobody is clearly accountable or didn't know, compared with 3% of vice presidents and above. Senior leaders may think this has been settled further down when it really hasn't.

Ownership matters even when the AI is doing a good job. This week the PostgreSQL project, which looks after one of the world's most widely used databases, pushed back its next major release from early October to around November 19 and pulled 8 to 10 significant features. One of its leading developers, Peter Eisentraut, explained that AI-assisted code reviews were turning up problems faster than people could work through them. Every one of those findings still needs someone to look at it and decide what to do.

The clearest example of a team that has thought this through came from Anthropic. On October 15, the people behind Claude Code wrote up how they now review code. Their rule is simple enough: whoever asked the agent to make a change owns it, just as if they had typed it themselves, and an agent's approval never counts as a person's. By their own numbers, the changes they merged more than doubled between June and September, while the share they had to undo fell by more than 40%.

What stands out to me is the order they did things in. They decided who owns the outcome first, and only then scaled up. AI can take on more and more of the work, but the responsibility still sits with people, and from what I can see, the teams that are clear about that end up moving faster, not slower.

_Sources:_{Wharton and GBK Collective}_·_{The Register}_·_{Peter Eisentraut}_·_Anthropic

The Rundown

Close to zero

At a Fortune event on October 15, Accenture's global CTO put the number of large organizations truly rebuilt around AI at "basically close to zero," and said these transformations are more about internal politics than technology. Mastercard's Greg Ulrich described a trap a lot of us will recognize: speed up one step in an 80-step process, and the next step just builds a backlog. The same day, 404 Media reported that Accenture has capped many employees' Claude spending at $29 a month. Redesigning the work, and paying for it sensibly, seems to be where most companies get stuck.

_Sources:_Fortune_·_{404 Media}

Open models are now doing most of the work

Vercel's October index of production AI traffic found open-weight models handled 71.5% of tokens but only 23% of spend, and almost one in three teams changed their main AI provider in September. On October 15, Meta released Muse Spark Open, an 816-billion-parameter open model that Apple showed running on four Mac Studios (read the license first if you're looking at it). It's getting harder to justify locking yourself into one provider.

_Sources:_Vercel_·_Meta_·_Apple

A startup building safe places for AI agents to work

Modal raised $750 million at a $15.75 billion valuation this week and said it has passed $1 billion in annualized revenue, a good part of it from sandboxes, sealed-off environments where AI-written code can run without touching anything else. That matters more than it might sound. Security experts told CSO Online the recent "rogue agent" incidents at AI labs came down to basic containment failures, and OpenAI disclosed four more on October 16.

_Sources:_Modal_·_{CSO Online}_·_Fortune

Most of the work people do isn't being tested

The startup Mercor compared 29 leading AI benchmarks with the 18,797 job tasks tracked by the US Department of Labor, and its Benchmark Map estimates that 90% of American working hours go to tasks the benchmarks barely cover. When a vendor says their model beats a benchmark, it's fair to ask whether that test looks anything like your work.

_Source:_Mercor

The Human Side

When the machine solves your life's work

Over the past few months OpenAI has published hundreds of math results, many on problems that had stumped mathematicians for decades. In an essay on October 15, Derek Thompson described how the field is taking it. Some of its leading figures say they have wept.

Cornell mathematician Steven Strogatz told him the grief isn't only about status. For him, mathematics is a conversation across centuries. Archimedes wrote about future discoverers building on what he left behind, and when a machine finishes the work, it can feel like that conversation has been cut off.

The same day, the Simons Foundation announced $15 million, with Anthropic and OpenAI, to help mathematicians evaluate AI-assisted results and set their own standards, and its president said the community, not any one company, should drive those decisions. On Sunday, mathematician Francesco Maggi wrote on Terence Tao's blog about what he sees as the real risk: knowledge growing faster than anyone can understand it.

I doubt mathematicians will be the last to feel this. Most of us tie some of our sense of worth to what we're good at, and I'd expect some people on our teams (and probably yours) to feel a version of it. I believe pretty strongly that AI can generate, but people create. We can imagine things that don't exist yet and question the rules. Part of our job as leaders is to be honest about the loss when it comes, and to help people see where their judgment and creativity are still needed.

_Sources:_{Derek Thompson}_·_{Simons Foundation}_·_{Terence Tao's blog}

Try This Week

Pick one team that's using AI tools and ask them for a one-page note: what counts as low, medium and high risk in their work, which work always needs a person to sign off, and who owns something an agent produced. Anthropic's team keeps notes like this right next to their code so their AI reviewer knows what failure looks like there. It takes an afternoon, and it answers the question one in five managers in the Wharton survey couldn't.

One for your IT team

OpenCode, a popular open-source AI coding tool, disclosed a breach on October 17 affecting 1.4% of users of its hosted Console, where some stored AI provider keys may have been exposed. If your developers use it, have them rotate any keys they saved there.

_Source:_OpenCode

That's it for the first edition. I'd really like to hear how your organization is handling this. When AI gets something wrong, who owns it? Hit reply and let me know, and tell me if there's something you think I should be covering.

References

About the Author

Kevin Sandahl

Chief Executive Officer

Covalience

Kevin Sandahl co-founded Covalience in 2008 and now stewards it as CEO. He grew up in Latin America and has spent over half his life living and working cross-culturally. He trained as an electrical engineer at Wheaton College and Illinois Institute of Technology and earned an MBA in international business from DePaul University. Before Covalience, he spent 14 years in technology and international management roles at Andersen Consulting (now Accenture), IBM, Orbitz, and Panduit, where he built and supported global services across North America, Europe, Asia, and Latin America. Today he splits his time between the US, India, and Malaysia, working alongside the teams he helped build.