Twice weekly, I share proven strategies to help data scientists get noticed, promoted, and valued. No theory — just practical steps to transform your technical expertise into business impact and the freedom to call your own shots.
The Failure Mode AI Makes Worse
|
Shortly after starting work in a new role, I was assigned responsibility for a data report that my predecessor, “Charlie”, had previously maintained. The report was scheduled to run on the first day of each month and had been used for years, seemingly without a hitch. However, on examining the code behind the report to figure out how it worked, I was shocked to discover a small calculation error that meant every report run to that point had actually been wrong. Not obviously wrong. The numbers were close enough to correct that they looked completely plausible. But for years, decisions had been made on the basis of figures that were slightly off - without anyone thinking to question them once. When I raised it, there wasn’t much that could be done. Charlie had already left the organisation. I told the end-users, corrected the report, and kept going. But the experience haunted me in the years that followed. My greatest fear as a data scientist has never been the loud errors - the ones that crash a piece of software as soon as you press “run”. It’s the quiet mistakes that scare me, where the program still runs to completion, but produces results that are subtly wrong. AI is now writing much of the code that Charlie previously would have written. But it doesn’t mean the problem has gone away. In fact, AI has made it even worse. In the agentic AI era, this problem is called “silent correctness”. And it’s what happens when an agent uses facts that are true to draw conclusions that are false, producing outputs that look perfectly fine - right up to the moment a decision goes very wrong. In this Value Boost episode of Value Driven Data Science, Jia Huang, lead research engineer at A*STAR and author of Designing AI Agents, joins me to explore why silent correctness is the most dangerous failure mode in agentic AI systems and what data scientists can do to catch it before it causes serious harm. In just 13 minutes, you’ll discover:
Listen now on Apple Podcasts or Spotify, or click the link below: Episode 120: The AI Silent Correctness Problem Talk again soon, Dr Genevieve Hayes |
Data Science Impact Algorithm
Twice weekly, I share proven strategies to help data scientists get noticed, promoted, and valued. No theory — just practical steps to transform your technical expertise into business impact and the freedom to call your own shots.
Correlation does not equal causation. Every data scientist has had that drummed into them. In fact, when I taught data science, I used to be the one doing the drumming. However, as every data nerd’s favourite comic, “xkcd”, once pointed out: correlation “does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there’”. This is the part most data scientists miss. Without correlation, there can be no causation and often, it’s because researchers noted the existence...
I once worked in an organisation where every member of the data science team was assigned an $8,000 laptop. It sounds way cooler than it was. We’re not talking about the Rolls-Royce of MacBooks here. Rather, it was a massive brick of a device that ran Ubuntu Linux and was so heavy it rarely left my desk. My team had convinced our employer to buy them because fitting the machine learning models we needed required powerful GPUs - and with security concerns limiting our cloud access, this was...
Although Python is my usual weapon of choice, I'm not averse to building the occasional model in Excel when the use case fits. Having originally trained as an actuary, there was a point in my life when I literally lived and breathed Excel. It still feels like second nature - which is probably why I've never felt tempted to use AI assistance when building Excel models, despite using it when writing code. However, after speaking recently to a fellow data professional about how much AI struggles...