James T Hamilton recommends the best books on the Economics of News

You developed the first comprehensive theory on the ‘economics of journalism’ in All the News That’s Fit to Sell. Please fill us in – what is this field all about?

The economics of journalism helps you see reporting, not as a mirror of reality, but as a product that emerges from the interplay between the demand and supply of information. We think of economics as decision-making of individuals under conditions of scarcity. When you look at the media, things that are scarce include reader attention and reporting resources. When people criticize the media they often claim bias or conspiracies are behind reporting; economics helps you see journalism as a function of supply, demand and the profit motive.

An Economic Theory of Democracy

(1957) by Anthony Downs is the first book you cite on this topic. Tell us about the text and why it makes your list.

Downs comes up with a great way to describe our information lives. He notes that we each have four information demands: as consumers, as workers, as audience members, and as voters. As consumers, we’re in search of products. As workers or producers we’re in search of information that helps us do our job. As audience members, we like to be entertained. And as voters, we’re in search of information that helps us choose among candidates.

“When de Tocqueville wrote Democracy in America he commented on the proliferation of newspapers in the United States.”

Downs points out that the first three markets work relatively well, because if you don’t seek out the information, you don’t get the benefit. So if you want to find a movie you’ll go to Metacritic or IMDb. But the rationale of getting individual benefit from searching out information breaks down for the voter demand. In particular, he says, even if you care a lot about which candidate wins, even if more information might help you choose the better candidate from your perspective, the statistical probability that your vote will determine the outcome of the election is so small that viewed from an investment perspective, the cost of becoming informed about politics swamps the likely benefits to the individual from seeking out information to cast an informed vote. Most people sit back and free ride, letting someone else read about local school boards or global warming mitigation.

He gives that phenomenon a name. He calls it ‘rational ignorance.’ It’s rational from an individual perspective, but not from society’s perspective. What it does is set up a big gap between what people want to know as audience members and what they need to know as citizens. Downs’s theory of ‘rational ignorance’ explains why stories that are important to society might never get told.

In All the News that’s Fit to Sell, one of the things I explain is that there is a market for public affairs information because of the three Ds – duty, diversion and drama. People feel they have a duty to become informed. For some people, C-SPAN is as interesting as ESPN. And for some people, human-interest political stories, political scandal, and horserace coverage also give them some substantive information.

The news business has changed so dramatically since Downs published in the 1950s. How would you update his analysis?

Those four information demands are consistent across time, but how they are met is affected by technology. When Downs was writing, those information demands were often bundled in the form of a newspaper that would give you entertaining information, listings that would help you find a movie and who was running for the local school board. Sometimes, as an entertainment reader, through serendipity, you’d also get exposed to political information. The biggest change since Downs is the arrival of the internet, which broke the bundle and made it much easier to find information about those first three demands. It lowered the fixed costs of getting into the business of meeting somebody’s information demands about sports or giving them an entertaining story about an upcoming movie. So, since Downs, the first three roles you have – as a consumer, a worker and a couch potato – this could be the best of all times.

But in your life as a voter, unbundling has meant that stories about local politics, which were once subsidized by other news, are much harder to tell. In particular, if you look at local newspapers, they once had classified advertising, which generated forty per cent of their revenue. That’s gone away and so the biggest change since Downs wrote is the increasing pressure on how you would satisfy the voter demand.

So does that mean that Downs’s theories help us understand, for instance, why American political parties are moving to extremes?

In All the News That’s Fit to Sell I show that media bias, and the perception of media bias, is a little bit like product differentiation. If you look at the political press in the United States through about 1860, the press was often subsidized by political parties because of rational ignorance. The market was not generating optimal political coverage. The parties had an interest in generating that information so they subsidized the press, in part through post office appointments to sustain publishers.

In the 1870s, you had the invention of the high-speed press, which was capable of printing thousands of copies per hour. The high cost of that press meant that it became more important for you to become a very large newspaper. The way you became a very large newspaper in a city was you took the word ‘Democrat’ or ‘Republican’ out of your title and you tried to offer a view of politics that was more objective. So I view objectivity as a commercial product, generated, in part, by the high fixed cost of running a newspaper.

Fast forward to today’s internet age, the fixed cost of having a website is relatively small, so you can sustain providing opinion and information with a small audience. That allows you to go to extremes in your coverage.

Speaking of the early history of media, please tell us about Pulitzer Prize-winning sociologist Paul Starr’s panoramic history, The Creation of the Media (1982).

It’s a wonderful book, in part because it shows that the founding fathers also confronted this problem of rational ignorance. Back at the time of the founding, it was already the case that, at an individual level, it did not always pay for a person to be fully informed about politics. Paul Starr makes a huge contribution to history by showing that the founders tried to subsidize discussion about public affairs through postal subsidies. They made it relatively cheap to send newspapers through the mail. Actually it was free for editors to send copies of newspapers to other editors across the country. That dispersal of information made it a lot cheaper for newspapers to talk about politics. When de Tocqueville wrote Democracy in America he commented on the proliferation of newspapers in the United States.

“It is very hard to fully monetize the value you create through investigative reporting.”

So the founders saw the need to subsidize political discourse. But if you fast-forward to the 1980s, during the deregulation of television, the chairman of the Federal Communications Commission said, “the public interest is defined by the public’s interest.” Today, there is a tension between people who believe that if the story is important it will get told and the people who believe, like I do, that there is a set of stories which don’t pass media market tests, that would pass a cost-benefit test from society’s perspective.

I’ve shown in my book that if you spend the time to do an investigation and tell a story that changes laws, the benefits of that law spill over to many people who might never be your readers or viewers. It is very hard to fully monetize the value you create through investigative reporting. That means that the media underprovides coverage of public affairs. There are different ways that you can address that. The way the founders did it, Paul Starr shows, is through postal subsidies.

In your work, you uncovered a particularly profound market failure. You did a telling empirical analysis of the market for mayhem in Channeling Violence. Please give us a précis of your findings.

I got into studying the media through the economic market for TV programming. I had been studying the Environmental Protection Agency’s Toxic Release Inventory, a yearly list of polluters and what is coming out of their smoke stacks. One day I saw a proposal being considered by Congress that would’ve shown who was advertising on violent television programs. Something in my mind clicked. I thought, you can view television violence like a pollution problem. People who provide violent entertainment programs are not trying to stimulate violence, they’re trying to attract an audience, primarily males 18 to 34, and sell them to advertisers. But, as they’re doing it, they’re exposing kids, aged 8 to 18, to violent programming. The research shows that some types of violence in some contexts will stimulate some kids to aggression. That negative impact on society is not something people think about when they’re producing violent programming. That negative spillover led me to write a book about the market for violent programming.

Next on your list is a memoir. What does Paper Route: Finding My Way to Precision Journalism (2012) tell us about the economics of journalism?

One way we can comprehend how news markets change is by following one individual. This autobiography traces the career of Phil Meyer, a reporter who helped bring the methods of social science into reporting through what he termed ‘precision journalism.’

When Meyer—as a reporter covering Capitol Hill for the Akron Beacon Journal—found it hard to gain the attention of sources in DC, he decided to dive into documents as a way to generate original stories. After learning how to use computers and survey research during a Niemen Fellowship at Harvard, he led a survey of Detroit in 1967, which helped win a Pulitzer for its exploration of the causes of a riot. That led him to think about how you could use the methods of social science, data collection, falsification, and apply it to journalism. As an academic, he trained a number of journalists. Later on, reporters and editors created an award in his honor. The Philip Meyer Award is given each year to the best work that uses the tools of social science to do investigative reporting.

At the Stanford Journalism Program you are building on precision journalism, advancing the use of data in investigative reporting.

Data journalism and storytelling are intertwined. You have to reach before you teach. Data make it possible for you to tell a different type of story. In my book, Democracy’s Detectives, I profile the career of Pat Stith, a Pulitzer Prize-winning reporter from the News and Observer in Raleigh, North Carolina. He started reporting in the mid-1960s, doing a story about the poor water quality in an African-American neighborhood of about one hundred and twenty homes by knocking on doors. In the 1970s, he wrote about a polluted river by walking an 18-mile stretch of the river and recording when he saw factories polluting through drainage pipes. By the 1990s he could use data collected by the state government and the EPA to write a story about hog farming’s impact on water quality in North Carolina. Eventually he wrote stories about water quality in wells across the state, stories which were made possible by the collection of data by government and by the drop in the price of computing power. So data can help you tell different stories.

Let’s turn to Watergate’s Legacy and the Press (2011) by Northwestern University professor Jon Marshall. Tell us about the book and what light it sheds on the economics of journalism.

In Watergate’s Legacy and the Press Jon Marshall shows that the investigative impulse, the impulse to hold institutions accountable, has been present in every era of journalism. The type of outlet that tells the story can change over time. Sometimes we think of investigative reporting in America as starting with a big burst in the Progressive Era. Marshall tells about McClure’s, the muckraking magazine that published exposés of Standard Oil. He traces how from that time through the fifties, sixties and seventies up until when the Washington Post broke news of the Watergate scandal, there’s always been this investigative impulse.

“Another way to nurture investigative journalism is through the evolution of the field of computational journalism.”

Investigative reporting usually has three characteristics: it’s original, it’s substantive and somebody wants to keep it secret, usually the government. Those three things are all related to economic principles. Original means you have to expend resources to create the story. Substantive issues of interest to the community means that, when you tell the story, the benefits can spill over to many other people and that makes it difficult to capture the value created by the work. Then the secret part: people try to throw obstacles in your way; government officials don’t like to be held accountable, even in the current administration. They’ve done a great job with open data policies in the United States. They make data available to help build a business; they call it DC to VC. They do a great job of crowd-sourcing, asking for your ideas and information. But still, if you look at Freedom of Information Act requests, government agencies don’t rush to reveal data that will help us hold them accountable.

Marshall seems to envision an alternative economic future for investigative journalism, in which the work is nourished by nonprofits, crowd-sourced and broadcast by blogs. Do you agree with his vision?

There are five incentives that get a story told. One is pay me; that is the subscription model. One is, I’m going to sell your attention to someone else; that’s advertising. One is I want to change the world by changing what you think about; that’s the nonprofit model. One is I want your vote; that’s the partisan model. And one is ‘I like to talk;’ that’s the expression model. Across time you see a variation in the relative instance and importance of those incentives. When people ask, ‘What’s the business model that will support investigative journalism?’ I answer ‘I don’t think there is one model.’ I think that we will see relative changes in the importance of each of those incentives.

I agree with Jon Marshall that nonprofits are potentially going to play a greater role in provision of investigative reporting. In my book, one of the things I talk about is that nonprofits, like the Texas Tribune and Voice of San Diego, are sustained, in part, by the same people who contribute to federal election campaigns, people who have an interest in the political life of the community. Nonprofits tend to do things that are a byproduct, in the private market, of public affairs: things that are having an impact by changing people’s understanding of a public policy issue. That becomes the direct measure of how the nonprofit is functioning. If you look at ProPublica or Chalkbeat or the Center for Investigative Reporting, all three of those groups have devised ways that try to measure the public impact of the stories that they tell.

The Watchdog That Didn’t Bark (2013) by public interest journalist and media critic Dean Starkman is your final selection. What is the gist of this book?

Starkman’s book is great. He focuses on the story of the financial collapse and asks why we didn’t see it coming. He explains that a lot of business reporting is what he calls ‘access reporting.’ A lot of business reporting is very quick, it’s aimed at investors, and it is often telling the stories of elites for elites. He talks about the difficulties of supporting ‘accountability journalism,’ the public affairs reporting that we’ve been talking about. He argues that there was less ‘accountability journalism,’ relative to the ‘access journalism,’ leading up to the financial collapse because that type of work takes a lot of time and effort. People don’t want to say ‘we had a great quarter this period because we engaged in fraudulent transactions.’ So he talks about the difficulties of financial reporting and the difficulties of holding institutions accountable. One of the reasons I love this book is because it is anchored in a particular topic, it outlines what was covered and what was overlooked.

Can you change the economics of journalism so that blind spots as important as looming financial crises are covered in the future?

In a world of rational ignorance, where there is a gap between what you need to know and what you want to know, how do you get the stories told that are expensive but that are necessary to hold institutions accountable? I address this question in my book, Democracy’s Detectives, in the last chapter, “Algorithms and Accountability.”

First there are policy levers. The Freedom of Information Act, which just turned fifty in July, needs to be fully implemented. A new bill was passed to strengthen FOIA. If implemented, it will lower the cost of telling the news we need to know. Also, right now the US government sponsors a lot of research and development related to the environment, education, and national defense but purposefully does not fund much research into journalism. If journalism was acknowledged as a source of market failure, resources could be directed toward developing content-neutral and platform-agnostic tools to help journalists. And, it would be great if the Internal Revenue Service would make it easier for online public affairs new sites to earn nonprofit status.

Another way to nurture investigative journalism is through the evolution of the field of computational journalism. Better use of data and algorithms will lower the cost of discovering stories. Additionally, computational journalism can help you tell stories in more personalized and engaging ways. If you had a news site that knew you as well as Amazon knows you, knew whether you liked documents or video, knew if you like hard news or human interest, if that site offered you a differentiated way of understanding the news, that product differentiation would be something that you would pay for. Internet news is tough to charge for because if facts are available elsewhere, people can get an easy substitute for what you’re offering. Computational journalism can develop techniques for personalization and engagement, to create a product that news consumers are willing to pay for, because it isn’t replicated everywhere on the internet.

So computational journalism holds the promise of altering the supply and demand for investigative journalism?

Computational journalism will lower the cost of discovering stories, because finding stories is always like drilling for oil. Better data and algorithms can help. But also, on the demand side, computational journalism can help you tell stories in a more engaging way. An example might be virtual reality. Right now, if you discover facts other people can use them; facts cannot be copyrighted. But if you produce a piece of virtual realty that can be copyrighted, that makes it harder for people to appropriate your work. If you can tell a story in a more engaging way, you have a higher chance of monetizing it, which will support public affairs reporting.