A sure thing? A look at how win probability is determined

The Daily Princetonian Sports is excited to announce a new article series, “Beyond the Numbers.” This series will take in-depth looks at various topics related to sports analytics and sports science. In the first installment of this series, we investigate what goes into determining in-game winning percentage — or win probability — and what we can do to make it more accurate.

Think back to the last time your favorite team blew a lead in a game. Maybe it was a college basketball team looking to stage a massive upset in March Madness, only to give up the lead in the final minutes of the game. Maybe it was a baseball team that had a three-run lead going into the bottom of the ninth, only to lose in extra innings. Or maybe you’re a Warriors or Falcons fan and know all too well what it means to blow a lead.

Regardless of the situation, chances are when you watched the game live or looked back at the recaps, you read or saw or heard something like “This is incredible; so-and-so team had a so-and-so percent chance of winning this game, and still lost!” With greater public interest in analytics and the data behind it, these stats are becoming much more commonplace. The Falcons at one point had a 99.7 percent chance to win Super Bowl LI before Brady and company made a roaring comeback. In 2001, the Cleveland Indians overcame a 12-run deficit to beat Seattle, who had a 99.89 percent chance to win the game and then a 98.1 percent chance to win in the bottom of the ninth! Just this past weekend, the Oregon Ducks had over a 99 percent chance to win their game against Stanford — twice — before losing in overtime.

And while not every game turns out this way, there are nonetheless many examples where teams with an almost certain chance at winning wind up losing the game. Call it a repeated miracle, but one has to wonder whether or not these probabilities are actually accurate; are we truly that lucky to be witnessing these 0.01 percent chance events once every few weeks? Or is something fundamentally wrong with how we approach win probability? And what even is win probability anyway? Here’s your chance to find out.

Determination of win probability — A “general” formula for how to win

Win probability is calculated differently depending on which sport is being considered. However, there are a few similar characteristics in every sport’s determination of the metric. Across sports, win probability comprises three general criteria: score of the game, time left in the game, and the winning team’s “big play prevention” defense.

The relevance of the score of the game in determining win probability should be obvious. If a team is up big, chances are that that team is going to win the game. However, this doesn’t tell the whole story. Namely, this leaves the definition of a “big lead” ambiguous. In every sport, a “big lead” is different. Conventionally, sports media and the general public will treat a two-score lead in any sport as “safe”; the chances of a team’s losing becomes exponentially smaller once they are up two scores. However, the significance of two scores varies depending on the sport. For example, avid soccer fans will argue that a two-goal lead is almost insurmountable, while baseball fans will be less than comfortable if their team is only up two runs. Similarly, two touchdowns in the NFL is a much “safer” lead than two touchdowns in NCAA football, where scoring is much more prevalent. Determining a “safe lead” is hard to do by itself, which is why the introduction of other variables is necessary.

That brings up the next variable: time left in the game. Time is the most important factor in determining win probability because it is the only finite variable with a definite endpoint. It is impossible for an NBA team to take the lead and win in the fifth quarter, just like an MLB team can’t take the lead in the 10th if they’re losing after the ninth. Generally, win probability updates significantly at checkpoints in a game such as the end of a quarter, halftime, or the change of an inning. In between stoppages, win probability is much less accurate because the result of the current possession is unknown; this is especially true in football where drives can last anywhere between a few seconds and over 10 minutes. Win probability also likes to pick teams that have the lead going into the latter half of the game, which is unsurprising. For example, in the first 48 games of this NFL season, only 14 teams won the game after being tied or trailing to start the second half. To put it another way, over 70 percent of all games were “won” before the second half began. Of course, this doesn’t take into account teams that gave up the lead and took it back, but nevertheless win probability is strongly determined by who is winning at critical junctures of the game. The exception is baseball, and this is because of the uncertainty of relief pitchers. For baseball, win probabilities tend to hover around 60–70 percent for a team up by three or fewer runs, until the end of the seventh inning, after which those numbers then skyrocket to over 85 percent. As more game uncertainty is erased with passing time, the team that is going to win becomes much clearer.

The final — and perhaps most interesting — component of win probability is a team’s ability on defense to prevent the big play. A “big play” is characterized best as one that gives another team a significant advantage to score against the defense. For baseball, this could be any non-single hit. For football, any play over 20 yards is usually considered a big play. For soccer and hockey, big plays come in the form of penalty plays (power plays, free kicks and penalty shots). This should come as no surprise, as it makes sense that big plays lead to blown leads or comebacks. What is surprising, however, is how little win probability takes into account the performance of the team trying to come back. Rather, all of the emphasis is placed on the defense trying to keep the other team from coming back.

For example, last season the Portland Trail Blazers and Los Angeles Lakers both had games against the Cleveland Cavaliers in which they led by six points at halftime. Despite Portland’s being a significantly better team than LA, the Blazers had just a 0.2 percent higher win probability than the Lakers in that game. In contrast, the Cavaliers had two games where they were up by two points going into halftime, one against Chicago and the other against Miami. In the Chicago game, they had a win probability more than 5 percent higher than they did in the Miami game! The metric was not based on Chicago or Miami and the ability of each to make a comeback; rather, it was based on the Cavaliers’ ability to hold each of those teams to fewer points. The Cavaliers’ fast-break system played well against the Bulls half-court system, and hence they were much more likely to beat the Bulls than the Heat.

Not included — Momentum, immeasurables, and how to improve the metric

With the three above variables making up the bulk of win probability calculation, there are clearly many stats left out. Some of these stats that are addressed below significantly impact our perception of the game, yet do not seem to make their way into win probability.

Get the best of ‘the Prince’ delivered straight to your inbox. Subscribe now »

The first that comes to mind is momentum. We’ve all seen a team that has struggled to score runs finally bust open the game after getting their first player across the plate. We’ve all seen a hockey team that finds new life after scoring the first goal in a two-goal deficit. Whether we can measure it or not, few would deny the existence of momentum in every sport. Momentum can be the driver behind a 12–2 run in basketball, or a 10-point fourth-quarter comeback, or even a come-from-behind win. Yet it is next to impossible to measure and will probably continue to be for the foreseeable future. Momentum is a phenomenon that we all know to exist, yet cannot seem to quantify.

Win probability — because of its inability to measure momentum — is very unresponsive to shifts in the tone of a game. As far as win probability is concerned, every score is the same. There are no emotional pick-me-ups, nor are there statement plays. Hence, win probability becomes much more uncertain in the face of a huge momentum swing. Oftentimes, this is where we see the biggest comebacks happen; we all know that feeling when our team scores late in the game and it just “feels” like they are going to win. Yet win probability will tell you there’s still only a 15 percent chance your team wins. Is there a way to correct this? Probably not, simply because momentum will almost certainly never be measurable.

The other factor is home field advantage. People tend to think that a team down at home faces a better chance of making a comeback than a team on the road. Surprisingly, however, the opposite seems to be true; in the past year of competitive sports, more than 62 percent of all comebacks came from teams on the road, according to analysis by ‘Prince’ Sports. Win probability seems not to care whether a team is at home or on the road. Maybe disregarding location is the best approach, but there are certainly some risks in not factoring in home field advantage. Going back to the third variable of win probability, it is much easier for a team to play defense in an environment they know well. Whether it is their familiarity with the friendly confines of a dome in football or the dimensions of an outfield, the home team seems like it would have an advantage in preserving the lead. Why do the stats say otherwise? That remains a mystery and perhaps a story for another day.

What should now be less of a mystery is the basis behind the win probability metric. While interesting in its design, it is actually quite a simple method that allows sports analysts to track a team’s potential for success. And while the metric’s ability to tentatively predict the future is remarkable, perhaps the most interesting part of the metric is its failures. When outcomes like Super Bowl LI and the 2016 Finals happen, and we get to see how unlikely these outcomes were in light of the data, they become that much more exhilarating and unforgettable.