The discussion around “analytics” in sports often comes with a lot of criticism for those numbers. That criticism is perhaps stronger still when the numbers in question come from a network broadcasting an event, and when the process that created them is not fully transparent. ESPN and their ESPN Analytics department have taken particular fire for this at times, and they’re now under criticism for it again with their Boston Celtics-Miami Heat coverage.
Ahead of the start of that Eastern Conference Finals series, ESPN ran a Twitter graphic indicating that their numbers gave Miami just a three percent chance to win the series. That was blasted at the time, especially by Heat fans. But it’s taken on further life since Miami’s 111-105 win in Game 2 Friday, putting them up 2-0 in the series. Despite that loss (the aftermath of which is seen above), ESPN’s SportsCenter Twitter account tweeted out that their analytics numbers still believe the Celtics have a 65 percent chance of winning the series.
The Heat are STILL underdogs in the series despite a 2-0 lead 😲
(via ESPN Analytics) pic.twitter.com/HkGGlVq5kS
— SportsCenter (@SportsCenter) May 20, 2023
How they got to those percentages isn’t entirely spelled out, but there is some information on it. A 2015 ESPN.com writeup from ESPN Stats & Info’s Ben Alamar spells out some of what goes into the ESPN Basketball Power Index, which is the source of these numbers. It produces both an offensive and a defensive rating number for teams, indicating how many points above or below average they’d be projected to score/allow per 100 possessions against an average team. The sum of those ratings is how many points they’d be favored by against an average team.
The ratings themselves are generated by a process that starts by inputting scores from every game. Then there are adjustments made to factor in opponent quality, game location, distance travelled (for road games), distance travelled, rest, and preseason expectations (Vegas expected win totals and the previous year’s performance). That’s how you wind up with the full Basketball Power Index rankings available, with the latest ones there putting Boston first overall with an 8.9 rating (6.5 offensive and 2.4 defensive) and Miami 14th with a 0.8 (-0.5 offensive, 1.2 defensive, inexact adding likely due to rounding).
And, as per Alamar’s 2015 writeup, the BPI numbers and then those other factors are used to project the chance of teams winning each remaining game, and 10,000 simulations for each game are then run to get to the percentages. That’s how you get the percentages tab, which has the Heat with just a 34.7 percent chance to advance to the NBA Finals and just a 5.8 percent chance to win the title, compared to the Celtics’ 65.3 percent and 46.8 percent respectively. In the Western Conference Finals, the Denver Nuggets lead the Los Angeles Lakers 2-0, and are given a 81.7 percent chance to advance and a 39.2 percent chance to win, with the Lakers given 18.3 percent and 8.2 percent chances respectively.
There are challenges to using a model so heavily based on the regular season (and also incorporating even preseason expectations) for playoff projections, though. Yes, there are many cases where dominant regular-season teams wind up doing well in the postseason as well, but playoffs also involve a lot of upsets. That includes this run from the eighth-seeded Heat, who only made it into the postseason through a win over the Chicago Bulls in the eight-seed game of the play-in tournament, but then took down the top-seeded Milwaukee Bucks in the first round, the fifth-seeded New York Knicks in the second round, and are now up 2-0 on the second-seeded Celtics.
And some of that is about various Heat players outperforming what they did in the regular season; for example, Jimmy Butler averaged 24.7 points per game during the campaign, but is putting up 28.5 during the postseason, while Tyler Herro has jumped from 20.7 points per game to 22.7 in the playoffs. Teams and players get hot sometimes, and the Heat certainly have. And some of them have taken note of the way ESPN Analytics views them:
“I’m sure the assalytics, assaholycs, or whatever they call themselves had us not beating Milwaukee in this first round.”
– @ThisIsUD reacts to ESPN analytics giving them a 3% chance to beat the Celtics in the Eastern Conference Finals. #HEATCulture
📺 https://t.co/YVOVpqoxnX pic.twitter.com/g1ZRgFnHSc
— Dan Le Batard Show with Stugotz (@LeBatardShow) May 16, 2023
The overall issue here isn’t necessarily the modeling, but the confidence it’s portrayed with, and the lack of full explanation on how the different factors are worked in. And this also gets further attention because it’s coming from a NBA broadcaster in ESPN (albeit not the one airing the Celtics-Heat series, which is on TNT). If this was just from an unaffiliated stats site, it wouldn’t receive as much attention. By contrast, ESPN making these kinds of calculations and then pumping them out on their high-profile social media accounts gets a lot of attention and engagement, but it’s not good attention.
This is similar to what happened with ESPN’s NFL Draft coverage and its repeated on-screen indications that ESPN Analytics declared there was a less than one percent chance that Kentucky quarterback Will Levis would still be available at pick 22. (He wasn’t taken until pick 33, the second pick of the second round.) That claim seemed even more strangely bold, considering the non-quantifiable factors involved with general managers’ decisions. But the pre-series one here from ESPN also seemed ludicrously bold, considering how much randomness can show up in any given game. And it didn’t seem like the level of confidence any individual would apply:
This may be what the model says, but I would bet there's not a single employee of the ESPN Analytics department who actually believes the Heat have only a 3% chance of winning. https://t.co/OdKbyGYCrm
— Michael David Smith (@MichaelDavSmith) May 15, 2023
And that’s really the thing here. If these models were tweaked to have less confidence in particular results, they’d seemingly be more reasonable. Of course, there’s no way to completely and accurately evaluate if a model is working or not for an individual sports game or series; perhaps the Heat’s start to this series is in the three percent of outcomes the model allows for, but we’ll never know, because life comes with one simulation rather than 1,000.
However, even by different mathematical approaches, the 97 percent confidence seemed very high. A 2019 analysis on best-of-seven series from LinkedIn data scientist Kenneth Tay on his personal site, also using a 1,000-simulation model, found that a team with a 70 percent chance of winning any given game in a series would still have a 13 percent chance of losing the series. And with the Celtics posting a 69.5 percent winning mark against all opponents this season, and the Heat winning at a 53.7 percent clip against all opponents, it seems quite high to give the Celtics a 70 percent chance of beating the Heat in any particular game. (The NBA has been calculated to have a higher percentage of top teams winning titles than other North American sports, but even with that, this feels excessive.)
But these models seem to be designed with very little tolerance for randomness, or for the weaker regular-season team prevailing. And the history of sports playoffs comes with a whole lot of unexpected events. As Billy Beane famously said in Moneyball, “My **** doesn’t work in the playoffs. My job is to get us to the playoffs. What happens after that is ******* luck.” And we’re perhaps seeing ESPN’s BPI not working in the playoffs.
[SportsCenter on Twitter; photo from David Butler II/USA Today Sports]
