Column: Hold predictors' feet to the fire

The turn of the calendar is often a time that forecasters and prognosticators issue their new forecasts of what will happen in the year ahead. Will the stock market rise? Will unemployment fall? Will Republicans win the Senate? This is the time of year anyone who predicts things for a living (or as an avocation) tends to tell you what they think.

But a report out this week from economists at Goldman Sachs shows an equally important part of that forecasting process. They went back to their year-ahead predictions from last January to see how they did.

In this case, the Goldman team did pretty well. Of 10 predictions, they were correct on seven, accurately forecasting, for example, that the economy would not tip into recession in 2013, that the housing market would continue its recovery, that corporate profit margins would stay fat and that inflation would remain low.

Their most clear-cut mistakes were in expecting capital spending to accelerate (it didn't, rising only about 2 percent, not the 6 percent Goldman economists forecast) and in expecting the unemployment rate to fall only slightly (it fell much more than they expected, because people left the labor force surprisingly fast).

Put aside the details of Goldman's hits and misses for 2013. What's important for others in the prediction game is how they go about the exercise itself. The world is full of people telling you what they think will happen in the future. If you have any doubt, turn on CNBC or ESPN during any given day. But what separates good forecasters from people just talking out their hindquarters is that the former will do a rigorous self-review of what they've gotten wrong and right.

Before you can do that, of course, you have to be specific about what you're predicting. It's one thing to go on cable news and predict with confidence that Hillary Clinton will be the Democratic nominee for president in 2016.

What level of certainty do you assign to that prediction? Ninety percent? Or 60 percent? Using vague language is a way of avoiding making any testable prediction at all.

And you have to write you forecasts down. All of them.

When you just make verbal, off-the-cuff predictions, it's easy to remember your successful forecasts and forget the bad ones. This is a fallacy that many amateur (and too many professional) stock pickers fall for. If you were clever or lucky enough to buy Facebook back in July at $26 a share, good for you! (It has since more than doubled.)

But to determine how accurate your forecasts were overall, you need to tally up how you did on all your stock picks, not just the one memorable success, and compare your results to how the overall stock market did. No fair giving yourself a pat on the back just for matching what the Standard & Poor's 500 index did.

It is that failure to reckon honestly with past successes and failures that leads to the "overconfidence bias" that is prevalent among everyone from investors to sports gamblers.

The great corrective about this is to do what the Goldman team did with their economic predictions. Write them all down. And regularly mark your conclusions to market. Assess what you got right, what you got wrong, and why, and use it to inform how you make projections in the future.

Neil Irwin is the economics editor of Wonkblog, the Washington Post's site for policy news and analysis.

© 2013 Washington Post

Column: Hold predictors' feet to the fire 12/17/13 [Last modified: Tuesday, December 17, 2013 4:47pm]

© 2014 Tampa Bay Times

    

Join the discussion: Click to view comments, add yours

Loading...