The Widow Maker


The hardest course I took during my entire data science education was a theoretical computer science class.

But not for the reasons you'd think.

Theoretical computer science is basically pure maths for data scientists, and the material was genuinely challenging. I spent 30 hours/week studying just to keep up.

Yet, the reason why this course became known as "the widow maker" wasn't just the time commitment. It was because referring to any materials beyond the prescribed texts was considered an act of academic misconduct.

Every proof we had to complete as part of the course was just a Google search away, so this was the only way the instructor could guarantee we'd learned the work.

I understand where he was coming from in creating this rule. The problem was the perverse incentives it created.

Students became frightened of doing too well in their homework for fear of claims that they had cheated. Some students even deliberately left in errors, to avoid the inevitable investigation a "perfect" score might trigger.

I have never felt prouder than the day I finished that course, but to this day, I'm still not 100% convinced the instructor's approach was educationally sound.

Here's the thing...

We're seeing this exact same situation playing out again when it comes to AI usage.

Schools are banning AI use to make sure students actually learn, which is fair enough on the face of it. But it's also creating AI witch-hunts, where every em-dash is treated as a confession and good students are being accused of cheating for the crime of producing work that seems "too good".

LinkedIn recently added a button to report "AI slop" and writers have taken to deliberately leaving in grammatical errors and typos as proof their posts are their own.

As a firm believer in the value of expertise, I understand the reasoning behind this. And figuring things out for yourself does force you to learn more than if a technological solution is at hand.

Yet, scientific progress has always been made by successive generations of scientists standing on the shoulders of the giants who came before them.

AI is a major technological leap for humanity. At some point, we need to stop treating every trace of it as evidence of a crime and start figuring out how to stand on this particular set of shoulders, too.

The widow maker didn't just teach students who cheated to fear getting caught. It also taught students who didn't to fear seeming too capable. This time around, we need to be careful about which lessons we're trying to teach.

Talk again soon,

Dr Genevieve Hayes

Data Science Impact Algorithm

Twice weekly, I share proven strategies to help data scientists get noticed, promoted, and valued. No theory — just practical steps to transform your technical expertise into business impact and the freedom to call your own shots.

Read more from Data Science Impact Algorithm

Before I was a data scientist, I was a statistician. For a long time, I thought of that as a liability. Machine learning felt like the way of the future and my statistical training felt like it belonged in the past. But over time, I noticed something. While most data scientists thought in terms of point estimates, I instinctively thought in terms of uncertainty ranges. It turned out that instinct was more valuable than I’d given it credit for. According to Adam DeJans Jr, co-author of The...

Suppose you were screening for a rare medical condition that occurs in just 1 in 1,000 people. The test you're using is highly accurate, with a true positive rate of 99% and a true negative rate of 99%. A person tests positive. What is the probability they actually have the condition? The answer, which can be verified using Bayes' theorem, is just 9.02%. That means it's more than 10X more likely that a person who tests positive has received a false positive result than that they actually have...

If your stakeholders could take two rocks, bang them together three times, spin around, and make a better decision, they would. That’s not a criticism. It’s just the truth about what stakeholders actually want. Your stakeholders do not wake up in the morning hoping that a new predictive model will be waiting in their inbox when they arrive at work. And they do not lie awake at night, wishing that the following day their data scientists will present them with more accurate forecasts. What they...