At PolitiFact.com, the St. Petersburg Times' fact-checking Web site, we love discussions of the nuts and bolts of public policy, so Paul Gary Wyckoff's new book, Policy and Evidence in a Partisan Age: The Great Disconnect, easily made it to the top our reading list. Wyckoff believes that actual data doesn't get enough attention in policy debates, usually because people either don't understand or aren't interested in statistical evidence. We rely on anecdotes, theories and case studies to make policy decisions, even when better data exist.
This inattention to real-world evidence makes us believe that government is more powerful that it is, Wyckoff contends. We're an "oversold society," he writes, and that plays itself out in different ways on the left and the right. Conservatives, for example, believe that the government has tremendous power to suppress economic growth through tax increases. Wyckoff says research says otherwise: Given the country's current debt, neither tax cuts nor increases will have much effect on economic growth.
Many liberals, on the other hand, believe that public schools can produce better outcomes through more spending and smaller class sizes. Wyckoff says a literature review of the empirical data shows the effects of increased spending are easily overwhelmed by the child's family and socioeconomic status.
This might sound depressing: If policy doesn't change anything, why even try? Actually, Wyckoff says, we should view policymaking through a different prism. When it comes to tax increases, we should look at programs individually and decide if the costs are worth the gains, rather than hinging the argument on economic growth. When it comes to education and welfare, we should understand that policies will effect little measurable change and ask ourselves what are society's moral and ethical obligations to the poor.
Wyckoff is a professor of government and directs the public policy program at Hamilton College in Clinton, N.Y. He teaches courses in applied statistics, ethics and cost-benefit analysis, and examines issues such as health care, education, Social Security and welfare.
Below is a lightly edited question-and-answer session with Wyckoff via e-mail.
One of the chapters in your book asks the question, "Can we spend or tax cut our way to prosperity?" You conclude the answer is no, that past data on the economy suggests that public debt counterbalances both tax cuts and spending. You say the real choices are more modest and straightforward, more like those facing ordinary families. What do you think is a current example?
Here's an example from the Washington Post. Recently, Gov. Kaine of Virginia spoke on the state's economic and revenue forecast. In part, he said:
"We have kept the Virginia business climate strong. As you all know, Virginia has been named the most business friendly state in America seven times during my tenure as governor. That's why we have seen so many globally innovative companies like IKEA, Volkswagen of America, SRI, AREVA/Northrop Grumman, Canon, MeadWestvaco, Orbital, Hilton, CSC, EDS, CGI, and Rolls Royce choose Virginia for major investments in the last three years."
The idea seems to be that if we just keep taxes on business low, we can keep companies investing in Virginia and have more money for everything - education, health care, welfare and transportation. But the literature suggests that the effects of state taxes on economic activity are very small - so small that reducing tax rates results in lower tax revenue. This means that giving tax breaks to companies means less money for schools, hospitals and single mothers. So states face hard choices about what they value most - more jobs at any cost, or better education for our kids, or better hospitals for patients, or better benefits for the children of poor mothers. This is very similar to a family trying to decide if they should spend their money on swimming lessons for Jimmy, a vacation for the whole family, or a down payment on a new car.
Recently, I had a conversation with an activist who said that media fact-checking doesn't mean much to him. The mainstream media is biased, he said, therefore our fact-checking is biased. My answer to him was that here at PolitiFact we cite all our sources and hope that readers can draw their own conclusions, even when they disagree with our ultimate rulings. What do you think of this exchange, and is there another answer I could have given him?
I would start by asking the activist how he gets the information he uses to form his opinions, if not through the media. Opinions must be formed based on some sort of data, or else we lose the capacity to learn about the world. Opinions based only on instinct or internal biases trap us into a dead-end, self-perpetuating view of reality. To paraphrase Winston Churchill, data is the worst way to form our opinions, except for all the other ones.
If you can get your activist to go that far with you, the next question is what kind of data to use to form his opinions. Today it is possible to bypass the media and get information directly from think tanks, blogs and Internet databases. The important point is to be aware of the quality of the data you're getting. As the book explains, data from large-scale experiments and literature reviews of empirical work in the field is much better than low-quality data such as anecdotes, case studies and raw figures. The latter suffers from problems with sample size (the cases may not be representative of the population) and confounding factors (some other thing may have caused the results being observed).
How do you think the ideas in your book are reflected in the debate on health reform?
One area that has been oversold is the benefits of prevention. The theory is that if we can just prevent disease, or catch it early in its course, we can cut the costs of treatment dramatically and save lots of money. But the literature suggests over and over that it is hard to save money through screening and prevention activities, because you have to provide these programs to large numbers of healthy people in addition to the few who actually have the disease in question. If you test everyone for diabetes, for example, you may get 50 or 100 negative tests before you find someone with the disease. According to Congressional Budget Office chief Douglas Elmendorf, and reported in Charles Krauthammer's Washington Post column of Aug. 14, 2009, "Researchers who have examined the effects of preventive care generally find that the added costs of widespread use of preventive services tend to exceed the savings from averted illness." This is a good example of the dangers of proceeding strictly according to our intuition without checking the data. The idea that prevention saves money makes perfect sense, so it's easy to use it to oversell a policy, but the data tell us that the world is more complicated.
Do you think we as a society have enough data to make good decisions? Do we need more data? Or, is the data out there, but we don't look at it? Or some mixture?
I do think we have the data to make good decisions. As pointed out by Ian Ayres in his book Super Crunchers, the digitization of data and the sharply reduced costs of storage and dissemination of that data make it easier than ever for individuals to get high-quality information. The problem is that we really don't know how to analyze all that data. As discussed in my book, if citizens and politicians have training in economics, it is overwhelmingly theoretical in orientation, with little empirical content. The most common postgraduate training for politicians is law school, and that provides almost no instruction in statistics. As a result, we enjoy an ever-increasing stream of information, but we lack the tools to make sense of it.
I recently read another book, The Happiness Hypothesis, by psychology professor Jonathan Haidt. One of his ideas is that we are emotional creatures: We often take positions and then we look for evidence to support that position. Do you agree with that idea?
Oh, yes, the "filtering" of data to find information consistent with our prior beliefs is a common thread in psychology. In his book Judgment Misguided, Jonathan Baron calls this "my-side bias." One of the more subtle ways in which this works is that when we are confronted with data confirming our views, we accept it without question, but when we are confronted with contradictory data, we scrutinize it thoroughly, looking for holes. As a result, different types of data face hurdles of different heights, distorting our view of the world.
To combat this problem, Baron prescribes "active open-mindedness," by which he means actively seeking information that contradicts your views.
Policy and Evidence in a Partisan Age: The Great Disconnect
By Paul Gary Wyckoff
168 pages, $26.50