|
While visiting my parents recently, I had a conversation with my (non-technical) 80-year-old dad that went something like this: Dad: The problem with AI is that it makes mistakes. If you don’t know if it’s right or wrong, then it’s useless. Me: But you use (OCR tool) Abbyy all the time. That’s AI. Is that useless? Dad: No. Because of Abbyy, I get stuff done in a fraction of the time it would otherwise take me. But I still have to review everything it does because it makes mistakes. What this conversation highlighted was that, to my dad, it’s clearly not the fact that AI makes mistakes that determines its usefulness. It’s whether it saves him time and whether there’s a feasible path to mitigating any errors. This struck me as almost perfectly echoing the words of statistician George Box: “All models are wrong but some are useful.” It also clarified something for me about how data scientists should think about the models they build. By all means, reduce the number of mistakes your model makes, because usefulness will always be partly a function of accuracy. But accuracy isn’t the only thing that determines whether a model is useful. Before obsessing over another percentage point of model accuracy, pause a moment and ask yourself these two questions:
My dad figured this out on his own, without a statistics degree. He uses an imperfect AI tool productively every day because the time saving is significant and his review process catches the mistakes. That’s not a workaround. That’s good judgement. And it’s exactly the same judgement data scientists should be applying every time they decide whether a model is ready to deploy. Talk again soon, Dr Genevieve Hayes |
Twice weekly, I share proven strategies to help data scientists get noticed, promoted, and valued. No theory — just practical steps to transform your technical expertise into business impact and the freedom to call your own shots.
When I’m looking for guests for my podcast, the first place I turn is invariably my bookcase. The authors of data science books have already demonstrated their authority in the field to the point of developing their own original IP. That makes them ideally suited to a podcast about data science expertise. And it’s no coincidence that many of those authors started off by writing blogs. It turns out conference organisers do something similar. When Cynthia Dunlop, who helps organise two major...
A couple of years back, I watched a TV series called The Fear Index. The series focused on a hedge-fund manager who had developed an AI system to optimise fund profits. Of course, as you would expect for a TV show - 🚨Spoiler Alert🚨 - everything falls apart when the AI starts taking highly illegal and frequently fatal actions in order to drive the market and achieve its goals. I enjoyed the show immensely, but at the time, felt it was far-fetched. However, recent reports of an OpenAI agent...
It’s no secret that many data scientists chose this profession in part because they enjoyed maths and wanted to avoid writing essays. When I was managing a data team, my team members would happily spend hours writing code. But ask them to write up what they’d done and suddenly everyone was too busy. Getting them to document their results in the form of a report was a lot like pulling teeth. And I understood why - to them, writing felt like a distraction from the “real” work. But over time, I...