Correlation does not equal causation. Every data scientist has had that drummed into them. In fact, when I taught data science, I used to be the one doing the drumming. However, as every data nerd’s favourite comic, “xkcd”, once pointed out: correlation “does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there’”. This is the part most data scientists miss. Without correlation, there can be no causation and often, it’s because researchers noted the existence...
3 days ago • 1 min read
I once worked in an organisation where every member of the data science team was assigned an $8,000 laptop. It sounds way cooler than it was. We’re not talking about the Rolls-Royce of MacBooks here. Rather, it was a massive brick of a device that ran Ubuntu Linux and was so heavy it rarely left my desk. My team had convinced our employer to buy them because fitting the machine learning models we needed required powerful GPUs - and with security concerns limiting our cloud access, this was...
7 days ago • 1 min read
Although Python is my usual weapon of choice, I'm not averse to building the occasional model in Excel when the use case fits. Having originally trained as an actuary, there was a point in my life when I literally lived and breathed Excel. It still feels like second nature - which is probably why I've never felt tempted to use AI assistance when building Excel models, despite using it when writing code. However, after speaking recently to a fellow data professional about how much AI struggles...
10 days ago • 1 min read
When I’m looking for guests for my podcast, the first place I turn is invariably my bookcase. The authors of data science books have already demonstrated their authority in the field to the point of developing their own original IP. That makes them ideally suited to a podcast about data science expertise. And it’s no coincidence that many of those authors started off by writing blogs. It turns out conference organisers do something similar. When Cynthia Dunlop, who helps organise two major...
14 days ago • 1 min read
A couple of years back, I watched a TV series called The Fear Index. The series focused on a hedge-fund manager who had developed an AI system to optimise fund profits. Of course, as you would expect for a TV show - 🚨Spoiler Alert🚨 - everything falls apart when the AI starts taking highly illegal and frequently fatal actions in order to drive the market and achieve its goals. I enjoyed the show immensely, but at the time, felt it was far-fetched. However, recent reports of an OpenAI agent...
17 days ago • 1 min read
It’s no secret that many data scientists chose this profession in part because they enjoyed maths and wanted to avoid writing essays. When I was managing a data team, my team members would happily spend hours writing code. But ask them to write up what they’d done and suddenly everyone was too busy. Getting them to document their results in the form of a report was a lot like pulling teeth. And I understood why - to them, writing felt like a distraction from the “real” work. But over time, I...
21 days ago • 1 min read
While visiting my parents recently, I had a conversation with my (non-technical) 80-year-old dad that went something like this: Dad: The problem with AI is that it makes mistakes. If you don’t know if it’s right or wrong, then it’s useless. Me: But you use (OCR tool) Abbyy all the time. That’s AI. Is that useless? Dad: No. Because of Abbyy, I get stuff done in a fraction of the time it would otherwise take me. But I still have to review everything it does because it makes mistakes. What this...
24 days ago • 1 min read
There’s a meme that’s been doing the rounds recently that shows a text message from someone’s significant other, who has noticed a $15k withdrawal from their joint bank account and assumes it’s for an engagement ring. Below that is the punchline - a screenshot of an Anthropic bill for $15k. Some people are now spending so much on AI that their “oh crap” billing moments have become a running joke. But if you think things are bad now, it’s only going to get worse. We are currently living...
28 days ago • 1 min read
During the COVID lockdowns, I decided to learn about cybersecurity as a way to fill in the time. I even signed up to sit the notorious 24-hour OSCP exam, although I never actually sat it, as I soon discovered cybersecurity wasn't for me. One thing I did take from the experience, however, was the concept of offensive and defensive security. Offensive (or red team) cybersecurity experts spend their days finding vulnerabilities in systems before hackers can exploit them. Defensive (or blue team)...
about 1 month ago • 1 min read