AI probably isn’t the life-altering technology that some people want us to think it is, nor is it useless technological pablum like NFTs. The reality is that AI is already being incorporated into many aspects of our lives, is still in its infancy, and will probably eventually have value.
For writers, AI has become a boogeyman that some evangelists claim will replace us. So far, it hasn’t proven terribly interesting, aside from when it spits out something deranged. That will eventually change, and AI will probably become a necessary tool in a writer’s toolbox.
These days, our understanding of AI is juxtaposed with industry leaders telling us it’s the future while the actual work resembles the musings of a 12-year-old who has no idea what they’re talking about. This was perfectly encapsulated in Gannett’s failed AI sports writing program that went viral last month.
Yup, that's a Gannett paper running AI-generated high school football stories. Yup, it's terrible. pic.twitter.com/VkuM1vNpy1
— Steve Cavendish (@scavendish) August 21, 2023
Gannett-owned newspapers such as the Columbus Dispatch, The Tennessean, and The Indy Star started publishing AI-generated recaps of local high school football games in August. They almost immediately went viral due to absurd phrases such as “a close encounter of the athletic kind” and how one team “took victory away from another,” not to mention several instances of broken code.
“The Worthington Christian [[WINNING_TEAM_MASCOT]] defeated the Westerville North [[LOSING_TEAM_MASCOT]] 2-1 in an Ohio boys soccer game on Saturday,” read one article.
Gannett paused the AI program soon after it became a viral laughingstock.
Jay Allred the CEO of Source Media Properties, which includes LedeAI, the company that created the AI-generated articles used by Gannett, spoke with Micah Loewinger for the latest episode of On The Media. He shared his point of view on the situation as well as how it happened.
"Close encounters of the athletic kind" became a meme after readers found the phrase in hundreds of AI-generated local news articles. In his first extensive interview, Jay Allred, the CEO of the company behind the flub, spoke about what went wrong: https://t.co/A9RuWt8pQ5
— On the Media (@onthemedia) September 27, 2023
First, he explained how the AI system works and how it decides how to describe a football game.
“Let’s just take the state of Ohio, for example. We look at all of the high-confidence games in the state of Ohio, and then we’re analyzing the box score,” said Allred. “So if we’re looking at a football game, we’re trying to figure out, was it a close game? Was it in overtime? Was it a blowout? Was it a come-from-behind win in the fourth quarter? We’ve kind of grouped those different outcomes into scenarios. And then we’re going to pull from a library of pre-written templates, plug those variables into those pre-written templates, choose a headline. They’re all pre-written, and they have variables that get plugged in. Typically with our clients, we connect to their CMS via an API (application programming interface), go in, and then programmatically create that asset on behalf of the customer, title it, assign an author, tag it for SEO, and then we can either publish those assets in draft status, or we can actually automatically publish them for the customer.”
As for whether or not readers would be confused into thinking a human being is writing the game recap, Allred said that he thinks it’s entirely clear to them what is happening.
“The readers understand it’s information. It’s not journalism,” he said. “Of course, a lot of times readers want the content to be longer and to include player names and photos and video. So there’s a certain sense of we’re fulfilling the information need, but not necessarily the information they want.”
He also admitted that AI can’t be relied upon to get granular facts and information on its own. Humans remain critical in how the system works and what it says.
“Number one, AI is not in a place where you should trust it as a reporter,” he said. “And number two, they’re really, at least in the high school sports zone, there isn’t a data set that would allow us to be able to confidently report things like player names and video and photo and to be able to accurately identify all of those things. You need humans for that.
“Every single word, every comma, every semicolon in our database has been written by a person, and then it’s been checked by another person and checked by a person after that. It’s what allows us to be confident in all cases that if we’re using our standard data set, that the content that we’re producing is accurate as long as the data is accurate, and it’s very accurate.”
As far as what went wrong with Gannett, Allred says that they rushed to meet a deadline and ended up bringing a buggy product to the public before it was ready.
“We had written some custom code for that particular customer, and the code had bugs in it. It had not been tested as well as we would normally have tested it because we had internalized a deadline that made it very, very important to us to kick off right with opening night of high school football season,” he said. “And some of those things that showed up in those Gannett articles, especially the errors, were the result of a small company working really, really hard to get ready for a launch with a very big company.
“As far as the awkwardness of the phrasing and the now infamous ‘close encounters of the athletic kind,’ a human being wrote that. A person wrote that good is a subjective measure. And we got called out on a few phrases and they are no longer in our database. It was as simple as taking them out. It was strange to become a meme, honestly, but we’re in a very early stage of deployment in the local news industry, and I wish that we did not have a part in a deployment that didn’t work.”
Allred says he’s very aware of the fears around AI, especially in journalist and writer circles. There’s an assumption that companies like Gannett will eventually look to replace human writers with AI in growing numbers. And while high school football games might not be high-stakes events in the grand scheme of things, there could come a day when AI is used for exactly those kinds of situations.
“One of the things that we learned from this process was that scale should be something that is approached cautiously,” said Allred. “And if I had it to do over again in our biggest launch ever, I would have launched in one site and checked every single piece of content up and down, and we would have found the exact same errors and awkward phrasing. And those objections would have rolled through, and then we would have been able to change those and fix those bugs and then launch in two sites a week later. And when those two sites were okay, we could have added a third and a fourth. High school sports are a low-risk beat. What if this had been crime reporting? Yeah, what if these had been arrest reports? Real harm could have been done. As leaders in the industry, I think it should give us all pause.
“No one was harmed by our awkward phrases and poorly placed variables inside of our high school sports reporting, but had we been doing arrest reports, there could have been incredible harm. The reality is we’re all going to be dealing with this.”
When asked if he worries that his product could eventually lead to human writers and journalists losing their jobs, he said “I think about that every single day” and that he loses sleep over the idea. Ultimately, he wants LedeAI to be a tool that helps journalists, not replace them.
“I think that the organizations that believe [AI can replace human staff] are not news organizations, as you and I understand them,” said Allred. “Is the intention…to find efficiency and to do that through less people? Or is the intention to create more value for consumers so that we can get the nose of this airplane pointed up and we can start to create a future where local news entrepreneurs can think of local news as a good small business?”
As for the backlash that the Gannett AI program received, and AI receives in general from the journalism community, Allred says that he hopes that people can see the eventual benefits without assuming the worst.
“I think that our industry has a tendency to respond to stuff like AI from a very defensive position. It’s super understandable,” said Allred. “Our industry has done nothing but cut newsrooms to the bone for going on two decades now. I wish we could get into spaces where we understood that we were more all in this together and that we are trying to figure it out.
“I think we, as an industry need to be able to hold multiple things to be true at the same time, which is malevolent deployment of AI inside of our industry is going to hurt our industry. Intentional, thoughtful deployment for the benefit of readers and communities and reporters can benefit our industry. Both things might happen. I hope it’s the second.”
Given the ways that media companies like Gannett have strip-mined their journalism staffs and newsrooms over the years and looked for more and more ways to cut costs, it’s hard to ask people to trust them when it comes to properly using something like AI as a tool instead of as a replacement. Not to compare AI to the nuclear bomb (though some might), but it doesn’t matter what the intentions of the creator were, it only really matters what the people in charge do with it.
