MONEYPUCK: Why Most People Need To Shut Up About 'Advanced Stats' In The NHL
There was a really interesting article in the Chicago Sun-Times last week about “analytics” being “the secret formula” for the Blackhawks’ success. It shows how the NHL in 2014 is like baseball in the late ’90s or early 2000s.
Fans, writers and a large percentage of teams stubbornly pretend that they have analysis all figured out, when they aren’t anywhere close. The Blackhawks are clearly one of the open-minded, smart ones — asking questions, compiling data and gaining actionable insights. Other good teams assuredly are. But many probably aren’t, and when it comes to media and fans, most are clueless.
The mainstream media’s discussion of hockey’s “advanced stats” involves shouting three terms at you: Fenwick, Corsi, and if you’re lucky, PDO. If you understand these stats, you’re probably laughing. For the uninitiated: Fenwick and Corsi measure shots on goal to estimate puck possession. The only difference between them is that one includes blocked shots. PDO literally adds a team’s shooting percentage and save percentage at even strength.
To continue the baseball comparison, they’re similar to WHIP (literally, walks plus hits divided by innings pitched) and OBP (literally, times on base divided by plate appearances).
They’re great stats, for what they measure.
But, a seven-year-old can compile and understand these stats entirely.
The most advanced and best use of these stats is 5-on-5 Fenwick-close percentage. It acts as a very accurate proxy for time spent in the opponent’s zone. It only counts even strength ice time in close games (tied or one-goal games in the first two periods, and tie games in the third/OT).
It’s not complicated.
The LA Kings led the NHL this year in 5-on-5 Fenwick-close percentage, with a mark of 56.48%. That means they took 56.48% of the total shots directed on net (not including blocked shots) in all of the “close” moments of their 2014 NHL games. And that their opponents, you might guess, took 43.52%. This means that they are very good at controlling play. You likely understand this quite well.
Every hockey player, coach, fan and writer will tell you that possession in hockey is very important. You can use fancy math to prove this (more shots = more scoring chances = more goals = more wins, duh), but it’s not necessary. Nobody is arguing this fact.
The only people that dispute these “advanced” stats are old, stubborn fools and people who are wholly ignorant.
Their only argument seems to stem from the fact that the stats sound fancy with their fancy names. Fenwick, Corsi and PDO are all named after the people who “created” these simple formulas. If on-base percentage was called “JAMES” or “BEANE,” it would probably see more resistance, despite how dumb that is.
Wouldn’t it be harder to argue something like “puck possession” (POS), “attacking percentage (‘AP)” or, and I’m getting crazy creative here, but “shooting percentage plus save percentage (‘SPSP?’) (‘SHSV%?’)?”
It would also make people realize that these things are not “advanced.” NHL video games have measured puck possession for years. They just don’t attach the fancy name.
And this is where mainstream discussion of “advanced stats” or “analytics” ends.
Of course, the stats aren’t used properly, and:
National media loves citing possession stats and acting like they measure team quality.
To wit: The predictions for the current Anaheim Ducks vs. Los Angeles Kings series have been inane. The Ducks are the higher seed. The Kings are the better possession team.
Which leads to some shitty analysis.
Despite their impressive record, it’s surprisingly hard to find many fans who take the Ducks seriously as legitimate Cup contenders, and they’ll probably be considered slight underdogs against the Kings.
The fancystats weren’t especially kind to them during the second half of the season.
He’s right. It is surprisingly hard to find people who take the Ducks seriously. (Though I’ll have to ignore his unkind fancystats comment, as he merely links to Fenwick and PDO graphs. On 12/3/13, the Ducks close-score Fenwick % was 50.8%. At the end of the season, on 4/12, it was 50.1%.)
The Ducks were indeed considered (very) slight underdogs to the Kings in the betting marketplace. So… why aren’t they being taken seriously?
Two factors that favor the Kings: (1) the Kings were a much better puck possession team than the Ducks, and those analytics/advance stats certainly favor an L.A. series win, and (2) Just like I picked the Kings to beat the Sharks because of the Jonathan Quick factor, same holds true here.
This is standard. Kings are the better possession team, which I know because I know “analytics,” so, yeah, they’re “certainly favored.” Ignore all other stats and analysis and home ice and matchups and the fact that they’re BARELY favored.
It gets worse.
“the advanced stats… indicate that the presumptive underdog Kings are actually a much superior team to Anaheim.”
They’re not underdogs. But, no, they don’t indicate that. Blatantly incorrect.
Advanced statistics from Extra Skater show the Ducks (15th in Fenwick differential, first in PDO) are headed for quite the crash in the postseason, while the Kings (first in Fenwick differential, 23rd in PDO) are one of the best possession teams in the NHL.
This takes things a step further, adding PDO into the mix.
Translated: The Ducks are an average possession team and the Kings are a very good one. And their PDOs are very different, because the Ducks had the highest 5-on-5 shooting percentage in close situations in the NHL this year, at 10.7%. The Kings had the third-lowest, at 6%. This is a MASSIVE difference. It’s reasonable to hypothesize that the Ducks are overrated and the Kings underrated by their respective records because of this difference.
But that’s only true if you think that the shooting percentage difference is due to luck, rather than something like skill or style of play. Saying they’re high and low does not mean they aren’t relatively accurate.
The Ducks were were third in the same category last year, at 9.3%. The Kings were 16th, at 7.9%. All shots are obviously not created equal, and, obviously, all players are not equally skilled at shooting.
We’re talking about a difference of 4.7 percentage points. The Ducks scored 60 more goals than the Kings this year. 0.73 goals per game is MASSIVE. Seemingly-small differences can be hugely significant.
The Kings are almost certainly underrated by their seed, and the Ducks are almost certainly overrated by theirs. But anyone with a brain knows not to use seeds as indicators of skill (UConn/Kentucky, anyone?). The betting marketplace is well aware of the very simple stats of puck possession, shooting percentage and save percentage. They are assuredly factored into the series price, at least to a reasonable degree, and the series price said this series was almost 50-50.
There’s nothing wrong with picking either team. But acting like you did “analysis” and deduced that one team is significantly better without presenting new information or a new opinion on existing information is not “analysis.” It’s lazy, and it’s wrong.
If you want to be lazy, you can check the betting odds, and go, “Hmm, this series is basically 50-50.”
Use of PDO usually sucks. People yell that it will “regress,” meaning that “the longer a player or team plays, the closer its PDO will get to 1.” That is true. However, all it means is that a team’s PDO gets more accurate as more games are played and sample size increases.
It does not mean that every team deserves to have a PDO of 1.
If every team deserved a PDO of 1, that would literally mean that all players and teams are equally skilled at shooting, and all goalies are the exact same.
Shooting percentage and save percentage involve luck/randomness, especially in the short term, but they also involve a ton of skill.
Basically: PDO can spot lucky and unlucky players and teams in the short term. If a dude has a 1.5 PDO in the first five games of the year, he’ll probably “regress.” A team having a high shooting percentage for an entire year (or two years in this case), doesn’t mean they’re “destined to crash.”
This should end things.
Nevertheless, there are inconsistencies in the association between possession and winning that bother me. Across the eight seasons of single-game data I compiled to create the Puck Prediction model, shot differential (which correlates strongly with Fenwick Close) was only predictive of 54.7% of game results. To put this in perspective, you’re more likely to pick the winner of a game correctly by simply choosing the home team than you are by choosing the team with the better shot differential entering the game.
Puck possession is a good thing and plays a role in wins and losses. This is not controversial or confusing. Puck possession is not everything, nor is it close to everything.
It’s likely the case that stronger possession play leads to a marginal increase in win probability, and over the course of a long season, this increase translates into additional wins and points. The same, of course, can be said of goaltending and team shooting, but these are less reliable from game to game than controlling the puck. But when it comes to single games, or small numbers of games (i.e., playoff series), no variable, even Fenwick Close, is as predictive as you might expect.
The reason that people use Fenwick is that it is a very good stat at measuring what it intends to measure.
It’s really good and consistent at measuring puck possession. A stat like shooting percentage is susceptible to a ton of randomness, which is why we can’t really say how much better the Ducks are at shooting than the Kings.
Which is why picking sports outcomes is really fucking hard.
As I’ve written before, betting moneylines are the undisputed heavyweight champions of prediction. If you’re an “analyst” using “analytics” to predict a playoff matchup, and your analysis tells you that one team is “far superior” and the other team is “headed for quite the crash” and “not taken seriously,” while the betting market says the series is basically 50-50, you’re either a rich genius, or you should double-check your work.
This happens in every sport. “Experts” are not experts. The ones who sound the most sure are usually the most-wrong. That’s a fact of life.
Still, it’s especially annoying in hockey, because the mainstream guys tout their analytical minds when all they’re doing is citing a very simple statistic that every reasonably intelligent person understands, and acting like that makes them experts.
Media hacks aside, the Blackhawks “get” it.
Nothing I’ve written is revolutionary. It just shows how, like in most areas, the observers know a lot less than the doers. Read the piece on the Blackhawks, and it’s clear they are actually using “analytics.” Don’t get me wrong: There are people that exist that look at actually-advanced stuff like this, but they’re not in the media, nor are they your average nor even average die-hard fan.
They’re tallying every shot — on goal, blocked, or simply missed. They’re tracking where each one was shot, where it was aimed, how it was shot, against whom it was shot. They’re tracking who brought the puck into the offensive zone, and in what manner — was it chipped in? Rimmed around? Carried in? Passed in? They’re noting who tends to start their shifts in the defensive zone, and how often they tend to finish in the offensive zone. They’re keeping an eye on the quality of competition each player is facing, and how they fare.
I guarantee they’re finding real meaning behind the numbers. And I’m sure they’re not alone among good teams. Because when the consequences to your predictions and decisions are money and wins, you typically don’t care about stupid shit like faking it to sound like an “expert.”