It’s always interesting to hear sports broadcasting executives weigh in on ratings measurement changes, and it illustrates how inexact widely-reported” ratings” numbers can be. No reported number is actually a perfect measure of how many people are watching at a given time. And even the widely-accepted and cited Nielsen numbers have shifted dramatically over time.
That’s been seen in the past around moves such as Nielsen’s 2020 full incorporation of out-of-home numbers, and their 2021 revisions to that approach. It’s also been seen with months-later corrections to certain events, such as this year’s Super Bowl LVII. And it’s now really being seen around changes to how they’re planning to report Thursday Night Football streaming numbers on Prime Video this fall, incorporating some of Prime Video parent company Amazon’s internal metrics.
On Monday, that drew unusually-strong criticism from Fox’s Mike Mulvihill and ESPN’s Flora Kelly, two of the most notable figures in this space. And on Tuesday, it drew further notable criticism from CBS Sports chairman Sean McManus and Video Advertising Board (a trade group representing a wide range of companies, including ESPN, NBC parent Comcast, CBS’ Paramount+, Cox, Hulu, and more) CEO and president Sean Cunningham. To start with, here’s what McManus said about the situation when asked by Sports Media Watch‘s Jon Lewis on a NFL on CBS preview call Tuesday:
“A fair and accurate audience measurement, across all platforms, as you know, is absolutely vital to our industry. Anything that is not impartial and unbiased is unacceptable to us. I must say that we think it’s extremely odd and unfortunate that different rules are suddenly applying to one platform. I’ll leave it at that statement.”
And the VAB’s Cunningham sent a letter to Nielsen CEO of audience measurement Karthik Roe Tuesday also complaining about this and asking for it not to be implemented. He and the VAB also released that to the media. Here are some key quotes from that:
While asked by many of our major members (including NFL programmers) to justify, explain, rationalize or prove any of the above, Nielsen has not given any level of meaningful detail that proves any of it is correct. The first obvious danger of Nielsen’s planned installation of unproven upside audience dynamics for Amazon’s TNF is that those factors can help cast Amazon as an over-achieving NFL outlier versus its competitors on the larger stages of: NFL partner strength, relative consumer appeal of competitive NFL media platforms, NFL programming valuations and NFL programming rights negotiations. These are areas that an objective third-party measurement/currency provider should not be creating advantages in for one of its clients, while simultaneously (and knowingly) creating competitive disadvantages for a group of its other clients.
It is also of great concern that the well-publicized “gaps” between Nielsen’s previous TNF audience estimates & Amazon’s first-party audience TNF estimates (Amazon was always
double-digits higher than Nielsen for the same TNF games), have now apparently been “reconciled” by Nielsen’s new methodology of calculating and integrating Amazon’s first-party data into the third-party NFL data set. Thus the impact data’s +18% rise in Amazon TNF household audience per Nielsen’s new Amazon first party methodology vs. Nielsen’s Panel and the 19% rise vs. Panel+Big Data are troublesome (particularly to all other NFL publishers), to say the least.
That letter also notes that the VAB and their members are okay with incorporating first-party data, but think that should be uniform:
To be clear, all of our NFL programmers have great enthusiasm for a first-party data driven future – done the right way. In fact many might argue that the much-higher Amazon first party data driven VPVH & OOH figures may indicate that Nielsen’s third-party estimates of NFL VPVH & OOH “are way too low” …as Nielsen’s own analysis proves its national panel to be still systemically under-counting relative to Big Data, etc.
That debate is a distraction from the more important issue, which is those two data sets are not yet ready to co-exist in the same measurement/currency data set…not in the way that has been laid out by Nielsen.
This is certainly going to be something to watch this fall. If Nielsen does go ahead with this integration despite the criticism, the Amazon TNF numbers are likely to look closer to the internal ones they’ve previously released, and are likely to be above the Nielsen ones issued last fall. And the NFL itself would certainly be happy about that, especially around the criticism it’s taken for moving TNF to streaming
But the other NFL broadcasters are quite clearly not thrilled that Nielsen is incorporating first-party streaming data for Amazon and not for their own streaming platforms. (Where their own numbers have previously been well above the Nielsen numbers.) We’ll see how it all plays out, but this will be a situation to track.
