Summary

Cathy O'Neil, founder of an algorithmic auditing firm, argues that frontier AI models are fundamentally unauditable for two structural reasons. First, meaningful auditing requires a narrow use case to define stakeholders and harms, but frontier AI is so general-purpose that no coherent 'cockpit' of monitors can be designed. Second, frontier AI systems behave inconsistently and unpredictably in ways that traditional scoring-based ML never did, making it impossible to honestly certify safety. She warns that companies claiming to audit these systems are actually just patching known holes while ignoring the thousands of undiscovered ones. The result is a dangerous public illusion of safety created by benchmark theater rather than genuine accountability.

Key Insight

Frontier AI cannot be genuinely audited because its generality and unpredictability make it impossible to define what 'safe' even means, leaving benchmark-patching as a dangerous substitute for real accountability.

Spicy Quotes (click to share)

• 5

  The frontier AI models cannot be audited, for two reasons.

• 6

  Just as we wouldn't want to fly in an airplane that has no pilots or cockpit, we shouldn't cede control of very important economic systems to opaque algorithms that might not work, or might only work for some people.

• 7

  We know how to audit a scoring system, but we cannot even predict the actions of an AI agent swarm.

• 8

  Like the proverbial little Dutch boy, they are sticking their fingers in all the known holes in the dike, and calling the dike safe.

• 7

  There are other holes, thousands of them, and there's not even a requirement for the frontier AI companies to report the holes that keep emerging.

• 6

  Auditing isn't just plugging up holes. That's necessary but not sufficient; when we say we've audited an algorithm for safety and it passed, it should mean the product is safe.

• 8

  What we have with frontier AI models is not safe, and the public should not fall for the hype.

Tone

critical, authoritative, warning