Yeah, It’s Hardly an Original Pun
So… the United States ran a massive numerical “experiment” last night, and it didn’t turn out the way any of the experts predicted.
What does this tell us “numbers people” about our chosen fields?
Simultaneously, I think the answer is “something important” and “nothing at all.”
But first, an aside.
Aside: Careful What You Wish For!
For my entire adult life, I’ve longed for a True Outsider candidate for US President. One who is not beholden to the money of big firms in order to get to office. One who could truly Shake Things Up.
Well, um, I finally got my wish, but did it have to be This Guy? He seriously worries me. I don’t think it’s a risk to PowerPivotPro’s business for me to say that most people reading this recognize that this dude is dangerous – to the USA and the world.
That’s not the same thing as saying he will be dangerous of course – I like to think that right now he’s getting The Talk from some very serious, deliberately-faceless people from the darkest corners of the so-called Deep State (the folks who never leave, regardless of who wins elections). And they’re explaining to him that his leash is a lot shorter than he expected. And maybe he’s having some epiphanies.
Anyway, “fingers crossed” is not a strategy, but that accurately describes where I’m at this morning. Maybe he surprises us in a good way. Please?
Never Trust Single-Trial Predictions!
Predictive Analytics fundamentally relies on lots of trials, lots of experiments, lots of training – before it can offer much in the way of accurate advice on the future. You need to run highly-similar experiments, many times, and feed their results back into the system.
Over the past year, I’ve been having a long-running friendly debate with two very dear and very intelligent friends. In short, the debate has been about how much trust to put into the prediction markets. They’ve long “liked” the prediction markets, and I’ve long said they’re near-worthless.
In short: Rob 1, Friends 0. Heh heh.
But in fairness to them, they haven’t really disagreed with me in any entrenched sense. They’ve been more asking me WHY I haven’t trusted the prediction markets.
My answer has always been that Predictive Analytics fundamentally relies on lots of trials, lots of experiments, lots of training – before it can offer much in the way of accurate advice on the future. You need to run highly-similar experiments, many times, and feed their results back into the system.
And a single Presidential election, something that happens every four years, is about as far from “lots of similar trials” as you can get. This election, furthermore, was clearly about as dissimilar to past elections AS YOU COULD POSSIBLY CONCEIVE.
So it’s unfortunate for us Numbers Types that so much of the celebrity attention on Analytics has been focused on this election – one of the places where Analytics is BY FAR at its weakest and least-useful.
I hope it doesn’t undermine us in places where we actually DO make a tremendous difference.
And if you do take some heat, maybe these observations will help deflect it.
“Wait, Rob, Were you Predicting a Trump Win?”
Oh hell no! Not at all. I’m almost as surprised as anyone today. An email I sent to those friends on Saturday sums up where I was at:
The Highlighted Part is the Important Part
“Is This the End of Nate Silver?”
Hey, Nate is an amazing guy. And he has an amazing team. They are an awesome crew and nothing about this election changes that. I will continue to read FiveThirtyEight on a regular basis… especially about topics other than politics.
BUT. In the political sphere, they have an impossible job. Predicting once-in-a-lifetime events with virtually non-existent prior information is basically impossible. It’s a wonder, in hindsight, that Nate EVER rose to fame playing the political predictions game. I feel for him, today, having to defend his entire reputation like this. On the flip side though, hey, that reputation has been very good to him.
Forget it. It’s stupid. It’s more intellectually honest to say “flip a coin” than it is to expect someone to accurately predict these sorts of things, no matter how smart and skilled they happen to be.
(Wouldn’t it be AWESOME for Nate to just come out and say “yeah folks this is impossible, we’re going to cease predicting elections?” But he can’t do that even if he wants to – too many other people would be hurt, too many corporate masters would be displeased.)
OK, so what ARE Analytics Good At?
First off, we’re amazing at breaking down, tearing apart, and/or digesting cold hard facts. You know, things that have indisputably actually happened. Which is where most analytics, and the vast majority of them that add value, are being performed today… and will continue to be performed, well into the future.
Yeah, I’ve heard that dismissive “stop looking in the rearview mirror at the past, look FORWARD to the future” argument made in favor of predictive analytics versus analytics on actual data, and you know what? Bullshit. You know, there’s two things about the so-called “past” that make it super valuable – 1) There’s no doubt about facts, you can call them 100% accurate predictions if you’d like and 2) the “past” is also synonymous with where you are right now. And that is super valuable, to know where you are. Imagine how useless our smartphones’ mapping technology would be without the GPS component. Hating on factual analysis is an anti-intellectual attempt by certain vendors to sell their stuff into a market that has yet to even realize 10% of the value of the existing factual data it already has.
As a company, PowerPivotPro alone has PROVABLY created hundreds of millions of dollars of value for our clients via Factual Analysis work. And that doesn’t include all of the impossible-to-count value created via our books and articles. Don’t for a moment get fooled that Predictive (or Machine Learning) is in some sense a replacement for Factual work.
And predictive analytics definitely have a valuable role to play! We’re currently diversifying into them as well, so follow what we DO over the coming years as much as what we SAY. But they are not going to replace Factual Analysis any time soon. And we’re only going to deploy predictive analytics responsibly, in cases where there’s a lot of relevant history on which to train the models.
Which brings us to an interesting point: even future-looking predictive models are 100% reliant on the much-maligned “past.” Funny, huh?