My Data Science "Superpower" was Completely Useless


5 years ago, my data science “superpower” was code optimisation.

I could take a piece of analysis code that took 30 minutes to run, and with a few tweaks, make it run in 2 seconds or less.

Spoiler alert: the secret is vectorisation using NumPy.

It was a trick I picked up while doing my Computer Science Masters, where several assignments included strict runtime constraints.

In fact, I got so good at doing this, that I often found myself spending hours optimising code, even when the potential benefits were trivially small - shaving off microseconds and making my code marginally more elegant and precise.

By the time I’d finished, my code was a work of art.

Here’s the thing…

Nobody cared.

Yes, writing functional, efficient code is important.

But at the end of the day, unless you’re working in a high-performance computing environment (such as stock-trading or self-driving cars), it usually doesn’t matter if your code runs in 2 seconds instead of 10.

This is the trap so many technical people fall into. We optimise the wrong things.

We spend hours perfecting code that already works well enough for the business problem at hand.

The business doesn’t care if your code is beautiful. They care if it solves their problem.

Focus on getting the right answer to the right question quickly. Optimise for business impact, not the elegance of your code.

Talk again soon,

Dr Genevieve Hayes

Data Science Impact Algorithm

Twice weekly, I share proven strategies to help data scientists get noticed, promoted, and valued. No theory — just practical steps to transform your technical expertise into business impact and the freedom to call your own shots.

Read more from Data Science Impact Algorithm

Like most data scientists, I started my career in traditional 9-to-5 roles. With bills to pay and limited experience, a regular job felt "safe". Then COVID hit. Nothing felt "safe" anymore, and my dreams of working for myself no longer seemed so insane. I finally made the leap - and never looked back. Many data scientists share similar dreams of breaking free from traditional employment, but don't know where to begin. Yet, here's what I've learned along the way: there's no single path to data...

A data scientist I know once told me of a manager who would end every team meeting with the same advice: “keep kicking goals.” What does that even mean? Which goals? With what resources? How do you even measure what a goal is? Unless you’re actually talking to a football team, “keep kicking goals” is meaningless. Yet this is what some data scientists face. Well-meaning managers who say “go do data science” without giving direction on what that actually requires. And then they wonder why...

Want to know the book that's had the greatest influence on my thinking as a data scientist? I bet you'll never guess. It's Jurassic Park (the book, that is - definitely NOT the movie). As a teenager, I read everything I could find by Michael Crichton and his message of focusing on solving the problem while ignoring all irrelevant distractions still influences the way I work. Whenever I feel overwhelmed in my work, that's what I remind myself. I've been doing it for decades and will do it for...