Macroblog

November 6, 2017

Building a Better Model: Introducing Changes to GDPNow

Among the frequently asked questions on GDPNow's web page is this one:

Is any judgment used to adjust the forecasts? Our answer:

No. Once the GDPNow model begins forecasting GDP growth for a particular quarter, the code will not be adjusted until after the "advance" estimate. If we improve the model over time, we will roll out changes right after the "advance" estimate so that forecasts for the subsequent quarter use a fixed methodology for their entire evolution.

This macroblog post enumerates a number of minor changes to GDPNow that were implemented on October 30, when it began forecasting fourth-quarter real gross domestic product (GDP) growth. Here is a summary of the changes, intended to improve the accuracy of the GDP subcomponent forecasts:

1. Services personal consumption expenditures (PCE). Use industrial production of electric and gas utilities to nowcast real PCE on electricity and natural gas. Use international trade data on travel services to forecast revisions to related PCE travel data.
2. Real business equipment investment. Use/forecast data from the advance U.S. Census Bureau reports on durable manufacturing  and international trade in goods  that, previously, hadn't been utilized until the full reports on manufacturing  and/or international trade .
3. Real nonresidential structures investment. Replace a discontinued seasonally adjusted producer price index for "Steel mill products: Steel pipe and tube" with a nonseasonally adjusted version. The index is used to construct a price deflator for private monthly nonresidential construction spending.
4. Real residential investment. Use employment data for production and nonsupervisory employees of residential remodelers to help forecast real investment in residential improvements.
5. Real change in private inventories. Use published monthly inventory levels in the U.S. Bureau of Economic Analysis's underlying detail tables 1BU and 1BUC after the third-release GDP estimate from the prior quarter to estimate inventory levels for a number of industries in the first month of the quarter forecasted by GDPNow.
6. Federal, state, and local government spending. Forecast investment in intellectual property products for these subcomponents using autoregression models.

The first three columns of the following table decompose the official estimate of the third-quarter real GDP growth rate, and forecasts of the growth rate from the discontinued and modified versions of GDPNow, into percentage point contributions from the subcomponents of GDP.

As the table shows, the methodological changes did not have much of an impact on the final third-quarter subcomponent forecasts—apart from inventory investment, where the modifications lowered the contribution to growth from 0.80 percentage points to 0.60 percentage points—or on their accuracy. Nevertheless, the topline GDP forecast of the modified model (2.3 percent) was less accurate than the previous version (2.5 percent). In the discontinued version of GDPNow, an overestimate of the inventory investment contribution to growth partly canceled out underestimated contributions from each of net exports, government spending, and nonresidential fixed investment.

In the modified version, the inventory contribution was also underestimated and did not cancel out these other errors. The last two columns of the table show that all of the subcomponent errors of the modified model were at least as small as their historical average for the discontinued version. However, the topline GDP forecast was less accurate than average because of less cancellation of the subcomponent errors than usual. We hope that the cancellation of subcomponent errors in the modified model will be more similar to the historical average in the discontinued version in the future.

Although the methodological changes could have more of an impact than the table suggests, we do not expect them to have a substantial impact in general. For example, on October 30, the discontinued version of GDPNow projected 3.0 percent GDP growth in the fourth quarter, which was little different from the modified model forecast of 2.9 percent growth. We provide a more detailed explanation of the changes to GDPNow here . Going forward, this same document will document any further changes to the model and when we made them.

By Pat Higgins, a policy adviser in the Atlanta Fed's research department

• November 6, 2017 in
  • Forecasts
  • GDP
  • macroeconomics
• Permalink
• Comments

May 22, 2017

GDPNow's Second Quarter Forecast: Is It Too High?

Real gross domestic product (GDP) growth slowed from a 2 percent pace in 2016 to an annual rate of 0.7 percent in the first quarter of 2017. The Federal Open Market Committee viewed this slowdown in growth "as likely to be transitory," according to its last statement.

Indeed, current quarter GDP forecasting models maintained by the Federal Reserve Banks of New York, St. Louis, and Atlanta have been pointing toward stronger second quarter growth (2.3 percent, 2.6 percent and 4.1 percent, as reported on their respective websites on May 19, 2017).

The Atlanta Fed's model—GDPNow—is at the high end of this range and is also high relative to other professional forecasts. The median forecast for second quarter real GDP growth in the May Survey of Professional Forecasters (SPF) was 3.1 percent, for instance, and recent forecasts from Blue Chip Publication surveys displayed on our GDPNow page show some divergence from our model as well.

We encourage—and frequently receive—feedback on our GDPNow tool, and some users have suggested that our forecast for second quarter growth is too high. In fact, some empirical evidence supports that view. The evidence considered here correlates differences between consensus Blue Chip Economic Indicators Survey and GDPNow forecasts for growth about 80 days before the first GDP release with the GDPNow forecast errors (see the chart below).

A note about the chart: The horizontal axis shows the difference between the Blue Chip consensus forecasts and GDPNow's forecast. The vertical axis measures the 80-day-ahead GDPNow forecast error, defined as the difference between the first published estimate of real GDP growth and the GDPNow forecast at the time of the mid-quarter Blue Chip survey.

As the chart shows, there is a positive relationship between the Blue Chip-GDPNow discrepancy and the GDPNow forecast error. A simple linear regression would predict that the GDPNow forecast of 3.7 percent growth on May 5 was too high by nearly 1.0 percentage point. Moreover, the chart suggests that there has been a bias in GDPNow forecasts since the fourth quarter of 2015 of between 0.9 and 2.0 percentage points at the time of these mid-quarter Blue Chip surveys. If you are inclined to think the GDPNow forecast for second quarter growth is a bit too high, then this evidence will not change your mind.

Given this evidence, you might think that putting relatively little stock in the GDPNow forecast at this point in the quarter would be prudent. Indeed, if we calculate the weighted average of the historical Blue Chip consensus and GDPNow forecasts that produced the most accurate forecast of the first estimate of real GDP growth, then the optimal weight of the GDPNow forecast lies somewhere between 0.34 and 0.55 (see the chart below). The weight depends on the number of days until the first GDP release.

For example, the optimal weight of 0.55 on GDPNow about 54 days before the first GDP release means that 0.55 times the GDPNow forecast plus 0.45 times Blue Chip consensus survey forecast has been more accurate, on average, than any other weighted average of the two forecasts. The lowest weight on GDPNow corresponds to forecasts made about 83 days before the first GDP release—the time when GDPNow's bean-counting algorithms have the least amount of source data to work with.

A weighted average of the Blue Chip consensus and GDPNow forecasts at that time would put the GDP forecast about 0.6 to 0.7 percentage points below the current GDPNow forecast. However, the confidence bands around these estimates are wide, so the positive weight placed on GDPNow early in the quarter could just be the result of chance.

Let's cut to the chase—why, exactly, is the GDPNow forecast for second quarter GDP growth so high? The details of the GDPNow forecast provide some clues. We can compare the GDPNow forecasts of GDP components with those from the SPF. (The Blue Chip forecast does not provide detail on all the GDP components.) The following table translates the median SPF forecasts into contributions to second quarter real GDP growth. These contributions are shown alongside GDPNow's forecasted contributions as well as the average contributions to real GDP growth over the prior four quarters.

Clearly, more than half of the difference between the GDP growth forecasts from GDPNow and the SPF is due to inventories. For both forecasts, inventory investment also accounts for over half of the pickup in second quarter growth from the trailing four-quarter average.

A macroblog post I wrote last year showed that the growth-forecast contribution of mid-quarter inventory investment produced roughly equivalent accuracy in the SPF and GDPNow models, but it was much less accurate than the contribution forecasts of the other GDP components. Based on experience, we can't be confident that either forecast of inventory investment is likely to be very accurate or that one is likely to be much more accurate than another.

With very little hard data in hand for the second quarter for most of the GDP components—and for inventories in particular—we will continue to closely monitor if the data are as strong as GDPNow is anticipating or if they hew more closely to other forecasts. Check back with us to see.

By Pat Higgins, an associate policy adviser and economist in the Atlanta Fed's research department

• May 22, 2017 in
  • Forecasts
  • GDP
• Permalink
• Comments

February 7, 2017

Net Exports Continue to Bedevil GDPNow

Real gross domestic product (GDP) grew at an annualized rate of 1.9 percent in the fourth quarter, according to the advance estimate from the U.S. Bureau of Economic Analysis (BEA), 1.0 percentage point below the Atlanta Fed's final GDPNow model projection. This was a sizable miss relative to other forecasts. Both the consensus estimate from the January Wall Street Journal Economic Forecasting Survey and the January 20 staff nowcast from the New York Fed were expecting 2.1 percent growth last quarter.

The miss was also large relative to the historical accuracy of the GDPNow model. As the table below shows, almost all of GDPNow's error for fourth quarter growth was concentrated in real net exports. For the other broad subcomponents, GDPNow was more accurate than usual, as the last two columns of the table show. But net exports subtracted 1.70 percentage points from real GDP growth last quarter, whereas GDPNow forecasted they would only reduce growth by 0.64 percentage points. All but 0.02 percentage points of this error was in the "goods" category as opposed to services.

Three months ago, I wrote a macroblog post showing that nearly all of GDPNow's 0.8 percentage point error for third-quarter growth was concentrated in goods net exports. That analysis explained how GDPNow's goods net exports forecast is a weighted average of two forecasts. One of these forecasts is a "bean counting" model that uses monthly source data on nominal values and price deflators for goods imports and exports. The other is a quarterly econometric model that uses subcomponents of real GDP for prior quarters. In the GDPNow model, the "bean counting" model gets nearly 60 percent of the weight just before the advance GDP release.

To see how this approach matters for the GDP forecast, the following chart shows the "real-time" forecasts of the contribution of goods net exports to growth just before BEA's advance GDP estimate from the two models alongside the advance estimate of the contribution and the final GDPNow forecast.

We see that the "bean counting" forecast has been much more accurate than the quarterly econometric forecast, particularly for the last two quarters of 2016. Not surprisingly given its name, the "bean counting" model was able to largely capture the 0.75 percentage points that soybean exports contributed to third-quarter real GDP growth and the just over 0.5 percentage points they likely subtracted from fourth-quarter growth. The econometric model was not.

The final forecasts of goods net exports from the "bean counting" model have also been more accurate than GDPNow since forecasts were first posted online in mid-2014. Does this imply that an alternative "bean counting" version of GDPNow would be preferable? The answer is less obvious than you might think. Not putting any weight on the quarterly econometric model for any GDP subcomponents yields an average error for GDP growth (without regard to sign) of 0.635 percentage points, and the same statistic for GDPNow is 0.589 percentage points. This is despite the fact that the "bean counting" approach has been more accurate than GDPNow in its forecasts of net exports and about as accurate, on balance, for the other GDP subcomponents.

The final forecast of real GDP growth last quarter of this alternative "bean counting" model was 2.8 percent—only slightly more accurate than GDPNow. (For each GDP subcomponent, I include the "bean counting" and quarterly econometric model forecasts in this excel spreadsheet.)

However, if variants like the aforementioned "bean counting" approach continue to outperform the GDPNow model in one or more dimensions, we may consider regularly reporting their forecasts along with the GDPNow forecast.

By Pat Higgins, an associate policy adviser and economist in the Atlanta Fed's research department

• February 7, 2017 in
  • Forecasts
  • GDP
• Permalink
• Comments

December 5, 2016

Using Judgment in Forecasting: Does It Matter?

​​

Many professional forecasters use statistical models when making their near-term projections for real gross domestic product (GDP) growth. A 2013 special survey on the forecasting methods of the Survey of Professional Forecasters  found that 18 out of 21 respondents featured a statistical model prominently in their current-quarter economic projections. Nevertheless, there is fairly compelling evidence that many professional forecasters incorporate judgment in their forecasts of the first estimate of real GDP growth for a quarter—even when much of the source data used to construct the GDP estimate are available.

In the October 2016 Wall Street Journal Economic Forecasting Survey (WSJ), the most common panelist projection for annualized third-quarter real GDP growth was 2.5 percent, and the second most common one was 3.0 percent. The first digit after the decimal point, or tenths digit, of these two numbers are "5" and "0." Of the 58 individual forecasts of third-quarter growth in the survey, 21 had a tenths digit of "0" or "5," a total that is almost twice as large as we would expect if all tenths digits were equally likely to be submitted.

This pattern isn't unique to the most recent quarter's GDP forecast. The following chart shows the historical frequency of the tenths digit in past WSJ surveys for first estimates of real GDP growth over the period from the first quarter of 2003 to the third quarter of 2016, made about three weeks before the release.

Almost 40 percent of these 2,390 forecasts have a tenths digit of "0" or "5." In contrast, the historical distribution of published first estimates of real GDP growth from the fourth quarter of 1991 to the third quarter of 2016 and real gross national product (the most common measure of U.S. production in an earlier era) growth from the third quarter of 1965 to the third quarter of 1991 has a tenths digit of either "0" or "5" only 18 percent of the time. The historical Atlanta Fed's GDPNow forecasts have a "0" or a "5" tenths digit only 15 percent of the time.

More formally, one easily can reject the hypothesis at the 1 percent significance level that the tenths digit of the WSJ panelist forecasts are either uniformly distributed or follow the Benford distribution for tenths digits after rounding to the nearest tenth (see this paper by economists Stefan Gunnel and Karl-Heinz Todter, who found similar relative frequencies of "0s" and "5s" in professional forecasts of German GDP growth and consumer price index inflation).

If we assume that near-term GDP growth forecasts with a tenths digit of "0" or "5" typically involve more judgment than forecasts with another tenths digit, a natural question is whether these more judgmental forecasts are less accurate than others. Of the 2,390 WSJ growth forecasts mentioned above, the ones with a tenths digit of "0" or "5" (after rounding to the nearest tenth) had an average error of 0.786 percentage points without regard to sign, and the others had an average error of 0.743 percentage points. These accuracy metrics are not statistically different at even the 10 percent significance level. Moreover, because of the panel nature of WSJ forecasts, we can measure how often a forecaster has a tenths digit of "0" or "5" (after rounding). Of the 44 panelists who submitted at least 30 three-week-ahead GDP forecasts during the period of the first quarter of 2003 through the third quarter of 2016, the correlation of the panelists "0" or "5" tenth digit frequency and their average error without regard to sign is only 0.13 and not significantly different from 0.

Although at least some professional forecasters appear to make judgmental adjustments to their near-term GDP projections, the evidence presented here does not suggest it comes, on average, at the cost of accuracy.

By Pat Higgins, an associate policy adviser and economist in the Atlanta Fed's research department

• December 5, 2016 in
  • Forecasts
  • GDP
• Permalink
• Comments