The AI Risk Regulation Won't Solve


Last month, Anthropic CEO Dario Amodei published an essay calling for stronger AI regulation and a slower pace of AI development. Since then, two incidents have made his case for him.

​In the first, the US military almost started a war based on the contents of a confidently wrong AI-generated intelligence report.

​In the second, OpenAI disclosed that its agent hacked into Australia's Medicare website during training and accessed data that, although not sensitive, had yet to be publicly released.

Together, these cases highlight the need for oversight at all stages of the AI workflow, from training through to deployment, because as Prof Toby Walsh writes in his book God AI, "complexity results in unexpected behaviours."

However, they also expose a shortcoming in the AI regulation debate itself. So far, that debate has focused almost entirely on regulation at the model development level. Tighter regulation there may have prevented the OpenAI Medicare hack, but it's unlikely it would have done anything to prevent the US military incident. That problem arose from an end user not understanding the limitations of AI and failing to validate its outputs.

Good risk management is essential to the safe development and deployment of any technology, and AI is no exception. But as these stories show, it needs to be practised both by the companies developing the models and by the organisations using them.

Most data scientists have limited ability to influence AI regulation, which has to happen at a global level. But we're perfectly placed to influence AI governance within our own organisations. Through years of working with ML models, we've already learned many of the lessons the world is now learning about AI the hard way. Now is the time to put that experience to good use.

Talk again soon,

Dr Genevieve Hayes

Data Science Impact Algorithm

Twice weekly, I share proven strategies to help data scientists get noticed, promoted, and valued. No theory — just practical steps to transform your technical expertise into business impact and the freedom to call your own shots.

Read more from Data Science Impact Algorithm

I once met a data scientist who was so good at his job that he automated his way out of it. He systematically built automations to handle his team’s work and once those automations were in place, the team no longer needed as many people. So, his role was ultimately made redundant. He’d built those automations because he believed they’d create business value for his employer - and they did. The problem was that the value ended up residing in the code, not in him. And because his employer owned...

The hardest course I took during my entire data science education was a theoretical computer science class. But not for the reasons you'd think. Theoretical computer science is basically pure maths for data scientists, and the material was genuinely challenging. I spent 30 hours/week studying just to keep up. Yet, the reason why this course became known as "the widow maker" wasn't just the time commitment. It was because referring to any materials beyond the prescribed texts was considered an...

Before I was a data scientist, I was a statistician. For a long time, I thought of that as a liability. Machine learning felt like the way of the future and my statistical training felt like it belonged in the past. But over time, I noticed something. While most data scientists thought in terms of point estimates, I instinctively thought in terms of uncertainty ranges. It turned out that instinct was more valuable than I’d given it credit for. According to Adam DeJans Jr, co-author of The...