Twice weekly, I share proven strategies to help data scientists get noticed, promoted, and valued. No theory — just practical steps to transform your technical expertise into business impact and the freedom to call your own shots.
The Model That Cried Wolf
|
Suppose you were screening for a rare medical condition that occurs in just 1 in 1,000 people. The test you're using is highly accurate, with a true positive rate of 99% and a true negative rate of 99%. A person tests positive. What is the probability they actually have the condition? The answer, which can be verified using Bayes' theorem, is just 9.02%. That means it's more than 10X more likely that a person who tests positive has received a false positive result than that they actually have the condition. The medical profession is now well aware of this and has developed work-arounds, such as performing multiple tests before diagnosing patients with rare conditions. However, there was a time when this wasn't the case, resulting in patients receiving life-changing diagnoses that weren't correct, sometimes with tragic consequences. Back when I taught Bayesian statistics, each semester I presented this example. But as a data scientist, rather than a medical professional, I never thought it would be directly relevant to my own work. Recently, while working with a client on a financial risk monitoring tool, I realised I was wrong. Rare events aren't confined to medicine. They occur across all kinds of applications, including fraud detection, equipment failure, financial distress. And yet, unlike the medical profession, most data scientists building monitoring tools don't think explicitly about the probability of their model crying wolf. The consequences of not thinking about it can be as devastating as those in medical screening. If a monitoring tool repeatedly flags warnings that turn out to be false alarms, the people watching it recalibrate. They start ignoring the alerts. And when a true positive result finally arrives, it inevitably gets ignored too. The tool becomes useless at precisely the moment it matters most. Optimising for accuracy alone isn't enough. If you're building a monitoring tool for rare events, you need a strategy for investigating false positives before you deploy it. The medical profession learned this the hard way. Data scientists don't have to. Talk again soon, Dr Genevieve Hayes |
Data Science Impact Algorithm
Twice weekly, I share proven strategies to help data scientists get noticed, promoted, and valued. No theory — just practical steps to transform your technical expertise into business impact and the freedom to call your own shots.
Last month, Anthropic CEO Dario Amodei published an essay calling for stronger AI regulation and a slower pace of AI development. Since then, two incidents have made his case for him. In the first, the US military almost started a war based on the contents of a confidently wrong AI-generated intelligence report. In the second, OpenAI disclosed that its agent hacked into Australia's Medicare website during training and accessed data that, although not sensitive, had yet to be publicly...
I once met a data scientist who was so good at his job that he automated his way out of it. He systematically built automations to handle his team’s work and once those automations were in place, the team no longer needed as many people. So, his role was ultimately made redundant. He’d built those automations because he believed they’d create business value for his employer - and they did. The problem was that the value ended up residing in the code, not in him. And because his employer owned...
The hardest course I took during my entire data science education was a theoretical computer science class. But not for the reasons you'd think. Theoretical computer science is basically pure maths for data scientists, and the material was genuinely challenging. I spent 30 hours/week studying just to keep up. Yet, the reason why this course became known as "the widow maker" wasn't just the time commitment. It was because referring to any materials beyond the prescribed texts was considered an...