When AI Gets It Wrong and No One Notices


Recently, I’ve been approached on more than one occasion by consulting clients asking me to review statistical methodologies developed with AI assistance.

In all cases, my clients came from non-technical backgrounds. And what they’ve managed to produce is impressive: detailed statistical specifications that would otherwise have required years of study to develop. Their work is living proof of just how far AI has come.

But on reviewing the work, one thing I invariably notice is that while on the surface what the AI has produced seems polished and professional, dig deeper and subtle methodological issues emerge - issues that are invisible without specialist expertise.

In these situations, this is exactly why I’ve been brought in. But conducting these reviews has made me wonder about the situations where no one in the room has the expertise to validate AI outputs - including situations where I’m the one producing AI outputs in areas beyond my own expertise.

Depending on the circumstances, the results could range from embarrassing to career-limiting or even catastrophic.

AI has the potential to amplify human capabilities. But it also makes us vulnerable in ways we’ve never had to face before.

In the latest episode of Value Driven Data Science, Derek Gibson, decision scientist and author of the upcoming Data, AI, and the Noise, joins me to share practical strategies for identifying unreliable AI outputs and building the defences necessary to keep AI-generated misinformation from reaching your stakeholders.

You’ll discover:

  1. Why AI is not a truth tool and what that means for how you use it [03:21]
  2. The red flags that signal an AI output shouldn’t be trusted [12:21]
  3. A simple prompting habit you can develop to reduce AI mistakes [16:13]
  4. Why the skill of verifying AI outputs is one you need to build yourself [24:25]

Don’t wait for AI to get better at telling the truth. Build your defences now.

Listen now on Apple Podcasts or Spotify, or click the link below:

Episode 111: Building Your Defences Against AI Misinformation

Talk again soon,

Dr Genevieve Hayes

Data Science Impact Algorithm

Twice weekly, I share proven strategies to help data scientists get noticed, promoted, and valued. No theory — just practical steps to transform your technical expertise into business impact and the freedom to call your own shots.

Read more from Data Science Impact Algorithm

The number one piece of advice I give to data professionals is to have more conversations. Conversations lead to good things. Things like job offers and stronger relationships. But more importantly, the opportunity to deepen your knowledge through learning from the wisdom of others. I owe many of the best opportunities I’ve encountered in my career to proactively seeking them out wherever I can. My podcast is a weekly example of the power of conversation. But if there’s one thing all these...

While working as a data scientist in a large organisation, I was once called on to attend a meeting with representatives of a big tech company and multiple senior leaders from my business area. The tech reps were there to sell us a new product. The data scientists had been invited because we were the intended users. Although there were a dozen or more people in the room, I was the only person who came prepared with any questions - 2 1/2 pages of them, to be precise. I let everyone else have...

Like many readers of this newsletter, I identify as a data expert. It took me years of study and hard work to build this expertise, and it’s something that I’m not willing to lose. This is not just because of everything it took to get here, but because my expertise forms an integral part of my identity. And the idea of losing it feels, in many ways, akin to losing a part of who I am. So, when I recently started hearing from data professionals who spoke of slowly losing skills as they...