Statistics and science aren't the same thing. Moreover, statisticians aren't fortune tellers, yet we continue to treat them as if they are. It's time to stop.
Nate Silver is a statistician and self-appointed electoral guru. Well, at least that was the story until the 2016 and 2020 elections proved that pollsters and statisticians are mere mortals.
Yet, the Nate Silver mythos persists. His website, FiveThirtyEight.com (named after the number of electors in the Electoral College), features statistical analyses, models, and predictions. The Orlando Sentinel says the site "is affiliated with ABC News and uses hard data and statistical analysis as the backbone of its sports, politics and economics content." Wow. Hard data and statistics. It's hard to argue with that.
Because I'm a die hard Seattle Sounders fan, let's take a look at FiveThirtyEight's prediction for which team would win the MLS Cup (the biggest trophy in Major League Soccer) in 2020. But first, a little background.
Like most American sports, MLS has a regular season followed by playoffs that culminate in the Big Game. For the NFL, it's the Super Bowl. For MLS, it's the MLS Cup. Not that I'm bragging (okay -- maybe a little), but the Seattle Sounders appeared in the MLS Cup final three times in the last four years and won two of them (2016 and 2019). We always have a good team.
Before the 2020 season began, FiveThirtyEight published its predictions for which teams would win the MLS Cup. Let's examine the twelve teams the site thought had the best chance to win. How have the predictions held up following the first round of the playoffs?
Ouch. You'd be far better off just tossing a coin than paying attention to FiveThirtyEight's predictions. The six best teams (by FiveThirtyEight's reckoning) were either eliminated or failed to qualify for the playoffs.
"WAIT!" I hear you say. Coronavirus. Injuries. Surely, it isn't fair to hold FiveThirtyEight accountable for poor predictions. Okay, fine. Let's look at the predictions FiveThirtyEight made on November 9, which was after the regular season ended. The website had a lot of data to work with, such as actually knowing which teams were good and which ones weren't. Their work was made easier by the fact that the teams that failed to qualify were already known. Now, how did FiveThirtyEight's predictions do?
Better but not great. Once again, the team it thought was best got eliminated, as did the 4th and 5th best teams. If you were a betting (wo)man, you certainly wouldn't use FiveThirtyEight's analysis to make your wagers.
Does This Matter? Sports Predictions Are Stupid
Sports predictions are nothing more than a fun pastime for those of us who like to follow sports. So, on the surface, none of this matters.
But it does matter a little. Why? Because of what the Orlando Sentinel wrote. Its statement that FiveThirtyEight is a site whose predictions are based on hard data and statistics confers on it an authority it doesn't deserve. The public wrongly assumes that statistics and science are the same thing. They are not. Like a microscope or ruler, statistics is a tool. Sometimes, this particular tool is used by tools.
Furthermore, statisticians aren't fortune tellers, yet we continue to treat them as if they are. It's time to stop.
