A conversation with the author of RAND’s new follow-up report on the diminishing role of facts and data in the news.
Eighteen months ago, WhoWhatWhy talked with political scientist Jennifer Kavanagh — a researcher at RAND Corporation — about the think tank’s report titled “Truth Decay.” The report details a set of very specific trends that has reshaped the media landscape. It looks at how, amidst the cacophony of 24/7 news and information that pours in at us every day, we seem to have lost sight of what constitutes truth, facts, and actual information. The signal-to-noise ratio has shifted overwhelmingly toward noise.
In this week’s WhoWhatWhy Podcast, Jennifer Kavanagh returns to talk about Part 2 of the report, titled “News in a Digital Age.” In this second report, she explains how, using computer programs and artificial intelligence, RAND analyzed thirty years of news data to find empirical evidence of what she describes as the diminishing role of facts and data in digital political discourse.
The new report, she explains, goes in-depth on social media, the speed and volume of information, the democratizing of access, and the lack of discrimination on the part of the public between high- and low-quality information.
She talks about differences in news styles, how they are best measured, and the differences in perception of news and facts when presented in print vs. delivered on cable or online.
The report is, as she describes it, a unique way of measuring the news.
Click HERE to Download Mp3
Full Text Transcript:
As a service to our readers, we provide transcripts with our podcasts. We try to ensure that these transcripts do not include errors. However, due to time constraints, we are not always able to proofread them as closely as we would like. Should you spot any errors, we’d be grateful if you would notify us.
| Jeff Schechtman: | Welcome to the WhoWhatWhy podcast. I’m your host, Jeff Schechtman.
A year and a half ago on this podcast, we talked with Jennifer Kavanagh, a political scientist and researcher at RAND, about their report entitled Truth Decay. The report detailed the set of very specific trends that had reshaped the media landscape. It looked at the increasing volume of opinion over fact, creating disagreement about basic objective facts, fueling distrusted, formerly respected institutions, and elevating personal, subjective experience to the ultimate truth. Since that report, the situation has only gotten worse. |
| Jeff Schechtman: | More opinion passing as news, news as confirmation bias, and arguments and advocacy now seem to be the principle drivers of our news infrastructure. Is this just a blip, a cyclical trend brought into bold relief in the Trump era or is something else going on, the long-term consequences of which we may never be able to realize and recover from? RAND has just released its second research report on Truth Decay, and I’m once again joined by Jennifer Kavanagh. Jennifer Kavanagh is a political scientist at the RAND Corporation and associate director of the Strategy, Doctrine, and Resources Program at the RAND Pardee Graduate School. She has degrees from the University of Michigan and Harvard, and it is my pleasure to welcome Jennifer Kavanagh back to the program to talk about Truth Decay, Part Two. Jennifer, thanks for joining us here at WhoWhatWhy. |
| Jennifer Kavanagh: | Yeah, thanks for having me. |
| Jeff Schechtman: | It’s great to have you here. How is this report both different from and an extension of the previous report on truth decay that RAND had done? |
| Jennifer Kavanagh: | Well, the first report really laid out a framework for thinking about the problem that we call Truth Decay, this diminishing role that facts and data increasingly play in our political and civil discourse. One of the factors that we pointed to as a driver of this was changes in the information ecosystem. And in that section, we talked about things like the rise of social media, the speed and volume with which information flows now, the increasing availability and access, which, you know, democratizes access, makes it easier to get news, but also makes it harder to distinguish between high quality and low quality information. |
| Jennifer Kavanagh: | And so what the second study did was it really dove into that specific driver, these changes in the information ecosystem. We asked the question, “How has news presentation changed over time?” How has the language and the tone and the style with which newspapers, television, cable television, and online platforms, how has tone and style of this presentation changed over the past three decades? |
| Jeff Schechtman: | And of course, it’s very easy to think about this in light of current events then, since 2015 and 2016. But this report takes a much longer view. It really looks at the period roughly over the past 30 years. Talk about that. |
| Jennifer Kavanagh: | That’s right. Right. So what we really wanted to do with look at these long-term trends. It’s easier to focus on what’s happening today and feel like things are very different than before. But unless we actually measure and compare how things have evolved over time, it’s difficult to say what’s changed and what hasn’t changed. So, you know, for newspapers, we’re able to look at that full 30-year period from, you know, the late eighties through 2017. And what we find is that, actually, newspapers have stayed largely the same. |
| Jennifer Kavanagh: | We do a comparison across the year 2000 and we find that there’s a little bit of change, a little bit of a shift towards a more narrative and storytelling-like style in news. But newspapers are largely the same. Whereas if you look at a shorter term comparison, like 2012 to 2017, comparing print and online platforms, you see a really big difference between the print and the online platforms in that time period. And so what I think that this research shows is that certain parts of the media ecosystem, certain pieces of this news presentation have stayed largely the same, but these cross-platform differences are pretty significant. |
| Jeff Schechtman: | Are there specific tipping points that show up in terms of when some of this has changed? |
| Jennifer Kavanagh: | Our analysis was based around comparing across specific time periods and specific platforms. So, we chose the year 2000 of our comparison point for the newspaper analysis, for our analysis of broadcast television, because 2000 was the year when, you know, Internet usage really passed the tipping point where pretty much everybody had internet access and usage was really widespread. So, that’s why we chose that tipping point. |
| Jennifer Kavanagh: | The question of whether if we took a different approach of kind of using a year-wise, where we changed the cutoff point year by year, we would see different results, is a good one, is one that we could look at in the future. |
| Jeff Schechtman: | Can we see as we look at this trend, over the course of 30 years, that there is kind of a cross-pollinization, a nexus between the changes that have taken place with respect to the Internet, with respect to cable and television, how that’s impacted newspapers and vice versa? |
| Jennifer Kavanagh: | Well, I think overall you see a general shift towards a more traditional style of news presentation that focuses on, you know, who, what, where, when, and why, that’s grounded in events and context to something that’s much more subjective that includes argumentation and advocacy and a lot of personal perspective. And so I think you can see this trend. It happens a little bit in newspaper over the 30 years. It happens a little bit more in broadcast television over the 30 years, and then it happens a lot. The differences in those different types of characteristics are significant when we compare across platforms. |
| Jennifer Kavanagh: | So you see that, you know, cable news is much more conversation-based, much more argumentation and emotion than what you see in broadcast television. Same thing if you compare online to print journalism. So, I think you do see this general shift. It’s hard to trace out causality, but one possible argument would be that we’re seeing this shift because of the competition between different outlets for audiences and a sense that the audience wants something that they can relate to, which may be something that’s more subjective and personal in its exposition. |
| Jeff Schechtman: | Is there a sense of print media in particular, newspapers specifically, trying to catch up? Can you see evidence of that? Trying to catch up with what we’ve see in cable news and in broadcast? |
| Jennifer Kavanagh: | I mean, you see small changes in that direction. But especially for print and also for broadcast television, the changes are really very small. So, I don’t think you see them trying to imitate completely these other types of platforms. I think you see small changes, which may be them responding to different types of demand signals from their audiences. But I think that they present information in a different way to a specific audience, and certainly the audiences of the different platforms overlap. |
| Jennifer Kavanagh: | But they are cognizant of, you know, the role that they play in the media ecosystem. And so those changes haven’t been that significant. I think that’s important to note, that while cross-platform changes are significant, which means that we have much more variety in the way news is presented, there still are options for people who are looking for a more traditional style of news presentation. |
| Jeff Schechtman: | What about traditional online sources that publications that are really indigenous to online? What do we see there? |
| Jennifer Kavanagh: | Well, so that’s what we call, you know, online journalism, and we look at a number of different outlets there. You know, there, you do see something that looks very different than print journalism. It’s much more subjective, much more personal in its communication of ideas, personal experience, ways of advocacy and argumentation. So, I have a point that I want to convince you of, meaning that it’s more biased and less objective in that sense. |
| Jeff Schechtman: | Talk a little bit about how this research was conducted, some of the computer power and some of the things that you did to really pull this all together. |
| Jennifer Kavanagh: | Sure. We developed, you know, what I’ll refer to as, you know, data sets of textual data, whether that’s newspaper, old newspapers, or transcripts of television broadcasts. We used a program called RAND-Lex. RAND-Lex is a text analysis software that we developed here at RAND, and the software analyzes text data. So, it can look for changes in styles. There are 121 linguistic characteristics that it analyzes, things like personal perspective. So, how much is the kind of personal side coming in? |
| Jennifer Kavanagh: | It looks for emotion, like motion cues, whether that’s anger or sadness or happiness, positive, negative. So, it has all these different things that it looks for. It scans through the text and is then able to detect differences across two sets of text data. We compared across the year 2000, and so it can tell us is the data before, is the text, the news presentation before 2000 significantly different on these different characteristics than the news presentation? Or is it significantly different in print journalism versus online journalism? And so, you know, that is the way that we identify what’s changed and what hasn’t. |
| Jeff Schechtman: | Do you get a sense in those that were researching and working on this that there’s almost a kind of cognitive dissidence if you’re a wide consumer of information? If you watch cable news and also read newspapers and print journalism, that on the same day with the same events happening, that there’s a real fundamental difference in what you’re hearing or reading about? |
| Jennifer Kavanagh: | Well, I think, you know, you do see common themes. You do see information that is presented that, you know, you can tell it’s the same story. But I do think that different platforms highlight, you know, very different aspects of those stories and tell them in very different ways, and I think that’s important to recognize. It’s important to recognize that there’s no problem with watching cable news. There’s no problem with reading online journalism or watching broadcast television. |
| Jennifer Kavanagh: | It’s just that each one of these platforms provides information a different way. As long as you’re aware of that and you’re aware that you’re getting different types of information in different packages, then I think there’s no problem with consuming any one of those sources. |
| Jeff Schechtman: | What did you find in terms of these linguistic changes in this 28-year, 30-year time period? |
| Jennifer Kavanagh: | Those are the changes that I’ve been talking about in terms of changes in emotion, changes in personal perspective, changes in references to authority, or references to context versus kind of more storytelling. Those are all linguistic characteristics and ways in which the presentation of news has changed over time. |
| Jeff Schechtman: | Talk about the storytelling aspect and the way that’s changed. |
| Jennifer Kavanagh: | Well, I think if you look at newspapers, you know, we don’t see a lot of change, but the change that we do see is that rather than just presenting facts in a kind of straightforward way, those facts will be told in a story. So, an example would be rather than presenting facts on homelessness, a story about what it’s like to be homeless from the perspective of a homeless child or a homeless adult. You’re getting the same information about homelessness, and the statistics can still be woven in and the facts, but you’re fundamentally reading a story about it rather than just a straight news article. That’s the change that we see when we look at newspapers over time. |
| Jeff Schechtman: | And is it your sense, is it the research sense, that this is a particular cycle, a trend with respect to this? Or is this just a completely new way of looking at this that’s here to stay? |
| Jennifer Kavanagh: | Probably something that’s here to stay. I think that the changes that we’ve seen over time are pretty small. I think the changes that we focus on are what we know. The key observation that comes out of our report is that what’s really driving this sense that news presentation is different is changes across platforms. And so that’s likely to stay, because I don’t expect us to lose any of these platforms. I expect print journalism and broadcast and cable and online journalism all continue to exist, and we may continue to get more, and those differences may get bigger as these new platforms evolve, as new media changes increasingly, adapts to kind of the way people use it and the different technologies that are available. So, I think that those changes are likely here to stay. |
| Jeff Schechtman: | And talk about it from the point of view of objective facts and the way those have been handled in the research that you’ve done. |
| Jennifer Kavanagh: | Well, we use proxies for, you know, what counts as an objective fact. It’s hard. The software can’t detect what’s a fact and what’s not. Instead, it can look for things like certainty versus ambiguity. So, we see a lot more ambiguity and a lot more opinion, opinion cues, things like, you know, I think or I feel in the new media, so in cable and in online journalism when compared to broadcast and print journalism. Whereas in print journalism, we context to places, to authority figures, things that are kind of grounded in what we would think of as facts. So, you know, that’s how we’re able to tell that there’s this shift towards a more opinion, subjective-based news on these different platforms and then a little bit over time. |
| Jeff Schechtman: | Was there anything in this research that was particularly surprising when you pulled it all together? |
| Jennifer Kavanagh: | Yeah. Actually, I really expected newspapers to have changed a lot more. I expected to see bigger differences over time, and we just don’t see them. They’re just very small and I was surprised by that. |
| Jeff Schechtman: | And in broadcast and cable, for example, did you see changes? I mean obviously, it’s a shorter period of time, but did you see changes in evolution there? Or is it pretty much today the way it kind of came online? |
| Jennifer Kavanagh: | Our measurement there is a little bit … We have a little bit of a blunt instrument, doesn’t necessarily allow us to get to that level of detail. We definitely do see that broadcast television over time has become more subjective. The changes aren’t huge, but they’re bigger than what we see in newspapers. So, again, more storytelling, more conversation, more unplanned speech, interviews, things like that. |
| Jennifer Kavanagh: | And then, you know, cable is very clearly different in terms of the way it presents information. In that case, you know, it does seem there’s a little bit of evidence that it accelerates over time. The differences are a little bit smaller, if you just focus on the early period. But pretty much those differences have been around since the two platforms emerged. |
| Jeff Schechtman: | And what’s next in this process and this research? What are the other things that you all want to look at? |
| Jennifer Kavanagh: | Well, I think that, you know, there’s a couple of specific questions that come out of this research that we’d like to look at. The most important being do these different ways of presenting news actually affect how people interpret news and use that information or act on it? You know, just the fact that news is presented differently across different platforms may not be something to be concerned about unless it is affecting the way people interpret that information. So, maybe people are more susceptible to disinformation campaigns online or on cable news than they are on other platforms. |
| Jennifer Kavanagh: | That would be something that’s interesting to know just from a disinformation perspective and thinking about how we deal with the proliferation of disinformation that we see today. So, that’s one area of research. We’re also conducting a number of other efforts that are looking at the media space more generally. We have a project looking at media literacy and the role that media literacy might play in responding to the spread of disinformation. We have a study looking at how people use media, so what platforms do they use and how they use them together and how do they view those platforms. We have a study looking at trust in media and what drives that. And we have a study looking at media governance, so what [inaudible] that we need to govern this increasingly diverse media space that we have? |
| Jeff Schechtman: | What, if anything, did you find with respect to the dissemination of this news in social media from the various platforms, whether it’s print journalism that pushed through social media or even broadcast and video journalism that goes through social media? Is there a difference in the way that’s perceived through social media as opposed to through its original source? |
| Jennifer Kavanagh: | Well, we didn’t look specifically at social media, so we just looked at the online journalism. The reason for that is that social media itself is so vast and so diverse that really we would need a whole study focused specifically on social media. I do think it’s important to note that, you know, social media, using social media to get news can mean a couple of things. One is that you’re using it as a pathway to get to something else, so you look at social media and you follow The New York Times and you click on the article and you go to The New York Times website and you read it versus you rely on posts from your friends about news. |
| Jennifer Kavanagh: | No, the first is essentially using New York Times. It’s just a way of getting to New York Times, of navigating to it, in which case, you know, the news that you’re getting is the same as you would get if you just went straight to the platform. And the second is you’re using information, but it’s secondhand, filtered through someone else’s eyes to you. Those are kind of two different things. Looking at social media is really important. We just couldn’t take it on specifically in this project. |
| Jeff Schechtman: | What about the length of material, the length of stories in newspapers, the length of broadcast news reports? Has there been any difference in that in terms of as it relates to people’s attention spans? |
| Jennifer Kavanagh: | Well, we didn’t look, you know, at the relation to people’s attention spans. We do notice that when we look at online journalism, the stories do tend to be much shorter. You’re getting kind of many but shorter stories, so we did notice that when we were building our data set. But we didn’t then look to attention span, although there is a lot of good research on the role that the getting news in small bites is playing in terms of our ability to concentrate for long periods of time. |
| Jeff Schechtman: | Still lots of information to gather in all of this. |
| Jennifer Kavanagh: | Yeah, there’s a ton of … The challenge in this space is that, you know, you peel off one question and you answer it and it reveals five more and then there are 50 more that you didn’t actually take on in the first place. So, I think this is why, you know, our first book laid out a really ambitious research agenda that we see as something that the whole research community, academia thinktanks, and other organizations can take on together. Because the only way is to come at it from many angles. |
| Jeff Schechtman: | Finally, Jennifer, what do you think that media people that are reading this, what can they take away from it? What can they learn from this? |
| Jennifer Kavanagh: | Well, I think it’s useful to understand the way in which different platforms are presenting information so differently and being aware of that. You know, for a consumer, it’s important just to be aware that you are getting these very different kinds of information on these different platforms. And for someone who’s working in media, you know, I think the relevance here is probably most important in the next step when we look at how interpretation of news depends on the way it’s presented. But I think it’s useful for people who are working in media to understand and be cognizant of the fact that we can measure these differences across platforms and if that’s intended, if it’s intended for them to be different for specific reasons, that’s one thing. If these changes have kind of emerged and the people writing the stories aren’t 100% aware of the way in which the information has changed, you know, that suggests a kind of different pathway and a different set of questions. |
| Jeff Schechtman: | Jennifer Kavanagh, thank you so much for spending time with us today. |
| Jennifer Kavanagh: | Thanks so much. This was fun. |
| Jeff Schechtman: | Thank you, and thank you for listening and for joining us here on radio WhoWhatWhy. I hope you join us next week for another radio WhoWhatWhy podcast. I’m Jeff Schechtman. If you liked this podcast, please feel free to share and help others find it by rating and reviewing it on iTunes. You can also support this podcast and all the work we do by going to whowhatwhy.org/donate. |
