The Atlanta Fed's Nicholas Parker outside the Atlanta Fed. Photo by Sadat Karim
Surveys make us smarter.
Without surveys, we would know far less about the state of the economy and about the country in general. Unemployment, retail sales, inflation expectations, small business access to credit, wages—these and countless other economic vital signs are the focus of meticulously crafted surveys.
The U.S. Census Bureau's numerous surveys produce economic data that form the raw material of forecasts and policies. But those data show primarily what has already happened. To supplement census data and enrich the information with expectations from businesspeople, the Atlanta Fed conducts its own surveys and collaborates on surveys with other regional Reserve Banks.
Looking to the future
"A lot of what we try to understand with our surveys is what people expect to happen, or how they see things unfolding," said Nicholas Parker, survey director in the Atlanta Fed's research department, "because we think these expectations are influential in firms' decision making."
Atlanta Fed president Raphael Bostic echoed that notion. During remarks at a recent gathering of central bank survey experts at the Atlanta Fed's Miami Branch, Bostic stressed the value of asking lots of people what they're thinking, doing, and expecting.
"In my academic career, survey data were an enormous help in gaining a fuller perspective on real-time events that we can't capture from backward-looking data reports," said Bostic, a professor at the University of Southern California before joining the Atlanta Fed in June 2017.
One of the Atlanta Fed's signature surveys is the monthly Business Inflation Expectations (BIE) survey. The BIE asks southeastern businesses about their sales levels and profit margins and how they expect their costs to change over the coming year. Taken together, cost predictions from thousands of businesspeople may provide some indication of inflation expectations, Parker explained.
That's important, because research has proven that inflation expectations have great influence on the actual long-term trajectory of prices. Inflation expectations also can affect behavior. If you expect prices to rise, you'll likely buy things before they do. On the other hand, if you think prices will fall, you'll probably delay purchases.
Though mainly geared toward prices, the BIE is not strictly about inflation. In addition to the standard queries that solicit information about the future direction of cost changes, the survey includes "special questions" that address policy issues. For instance, recent special questions concerned the business effects of Hurricane Irma and the expected impact of the federal tax law passed in late 2017.
Atlanta Fed surveys are no academic exercise. Bostic often takes information drawn from the surveys—such as the standard inflation expectations measures and readings of the impact of last year's hurricanes—to meetings of the Fed's policymaking Federal Open Market Committee.
New survey already adopted by others
The Atlanta Fed is busy developing a new survey. In testing for more than three years, the Business Decision Makers Survey (BDS) seeks to measure changes in firms' expectations and associated uncertainty in reaction to shifts in public policy or economic conditions. For instance, do firms plan to add personnel as a result of the newly passed tax bill? Have they become more certain about these plans?
"We're trying to understand more about uncertainty and how it is affecting people's decision making and shaping their views of the future," he said.
The BDS's influence will go beyond the Southeast. The Census Bureau liked the question design from the BDS enough that it has incorporated it in a survey of 40,000 manufacturers. And the United Kingdom's central bank will use the survey to solidify understanding of the impact of the UK's impending departure from the European Union.
Global group of survey experts planning first-of-its-kind project
The Bank of England's decision to adopt the Atlanta Fed survey was no accident. The move resulted from discussions within a global organization of more than 30 central banks. This group meets annually to discuss and refine survey practices and methods. In fact, the Atlanta Fed hosted the 2017 meeting in November at its Miami Branch, where Bostic spoke.
Another collaboration emerged from that gathering. Several central banks, including the Atlanta Fed, are coordinating a new collaborative international survey. It will examine companies' adoption of certain digital technologies and the long-term effects on those firms' pricing and employment.
Central banks don't take this work lightly. Surveying has evolved into an exacting discipline that involves far more than simply emailing questions and compiling responses. It is especially critical to make surveys meaningful and accurate when the results inform processes as important as monetary policymaking. As a result, surveyors are scrupulous about formulating questions and assembling groups of sample respondents that reflect the larger population.
"The whole purpose of survey methodology is to limit error, or bias," Parker said. "You are looking for every possible way you can remove error from the process."
Mobile technology changing everything
Not surprisingly, building a quality survey is painstaking. For example, surveyors must balance awareness of the burden they're placing on respondents with their own informational needs. They must also make sure that questions aren't so long that they discourage participation or compel respondents to rush.
Technology has introduced urgency and nuance to survey construction. As pen-and-paper surveys have slowly yielded to online alternatives, surveyors have had to focus on question length and typeface size. These considerations become especially relevant as more and more people complete surveys on mobile devices, Parker said.
A mobile user, for instance, is less likely to answer a question that extends beyond one screen. Parker also considers whether mobile survey takers prefer "paging," or moving to an entirely new screen, versus scrolling. And do people favor bigger fonts and buttons? Parker believes they do.
To help answer these questions, Parker employs a tool on his computer monitor that displays exactly how a question will appear on a smartphone. "It's a whole new canvas," he says of the mobile device. "Above all, you want to make the experience as easy and frictionless as possible."
Refining surveys, very carefully
Surveyors are augmenting their toolbox in other ways. Parker has recently begun using software that analyzes text, allowing his surveys to ask open-ended questions—as opposed to a simpler multiple-choice approach—that ideally prompt more thoughtful and thorough answers. Open questions also improve on multiple-choice queries, Parker points out, because even the best survey designers cannot always anticipate all possible answers.
So how does Parker efficiently analyze paragraphs from thousands of respondents? Software employing machine learning techniques allows him to uncover common themes in responses, ideally leading to greater insights. As his use of text-analysis software demonstrates, Parker and other survey scientists constantly refine their tools and techniques. But such refinement can be tricky. For example, once established, a time-series survey—one that gauges views over months or years—is difficult to alter. Make too many changes, Parker explained, and the information's usefulness could be diminished or even rendered worthless.
Taking care to safeguard the integrity of his survey, Parker will likely tweak the BIE survey in 2018.
"We're producing data series for long-term research," Parker said. "We hope the data we collect will allow us to make a useful contribution to the understanding of firms' decision making, uncertainty, and inflation expectations. We think hard about our work, and it will continue to improve and evolve."
Charles Davidson
Staff writer for Economy Matters