Real-time forecasts from the Atlanta Fed’s real gross domestic product (GDP) nowcasting model—GDPNow—have been regularly updated since August 2011 (the model was introduced online in July 2014). So we now have a nearly five-year history to allow us to evaluate the accuracy of the model’s forecasts. The chart below shows forecasts from GDPNow (red dots) alongside actual first estimates of real GDP growth (gray bars) from the U.S. Bureau of Economic Analysis (BEA). For comparison, the blue dots in the chart are the consensus (average) forecasts from the Wall Street Journal Economic Forecasting Survey (WSJ Survey).
The initial estimate of real GDP growth for a particular quarter is usually published at the end of the subsequent month. The WSJ Survey consensus forecasts plotted above were released about two weeks before these estimates. To maintain comparable timing with the WSJ Survey, the GDPNow forecasts shown in the chart are those constructed on or before the 12th day of the same month.
Occasionally, there has been relatively large disagreement between GDPNow and the WSJ consensus. For example, GDPNow predicted that GDP growth would be below 0.5 percent for five out of 19 quarters between 2011 and 2016, and the lowest WSJ Survey consensus forecast for any of those quarters was 1.3 percent. Nonetheless, the average accuracy of the GDPNow and WSJ Survey consensus forecasts has been similar: the average absolute forecast error (average error without regard to sign) for GDPNow was 0.56 versus 0.60 for the WSJ Survey consensus.
Studies have shown that the average or median of a set of professional forecasts tends to be more accurate than an individual forecaster (see, for example, here and here). Therefore, it’s surprising that GDPNow has been about as accurate on average as the WSJ Survey consensus. To see just how surprising this result is, I used the fact that the WSJ Survey provides both the names and forecasts of its respondents. From these, I constructed a panel dataset with each respondent’s absolute forecast errors and their absolute disagreement (difference) from the consensus forecast. Using a standard econometric technique (a two-way fixed-effects regression), we can then calculate each panelist’s average absolute GDP forecast error and their average absolute disagreement with the WSJ Survey consensus. These points are shown in the scatterplot below.
There is a clear inverse relationship between average forecast accuracy and average disagreement with the WSJ Survey consensus. However, GDPNow’s accuracy and disagreement statistics do not fit the general pattern. GDPNow (the orange diamond in the chart) was more accurate on average than all but six out of 49 WSJ panelists, though at the same time it differed from the consensus by more on average than all but four of the panelists.
What should one infer from all of this? Differences in forecasting method could be part of the explanation. GDPNow differs from many other approaches to nowcasting in that it is essentially a “bean counting” exercise. It doesn’t use historical correlations of GDP with other economic series in the way that commonly used dynamic factor models do, and it also doesn’t incorporate judgmental adjustments (see here for more discussion of these differences). During a period when the economy has been giving very mixed signals, perhaps it doesn’t come as a surprise that GDPNow’s forecasts occasionally deviate quite a bit from the WSJ Survey consensus. Time will tell if GDPNow continues to perform at least as well as the consensus.
By Pat Higgins, an associate policy adviser in the Atlanta Fed’s research department