2018 Financial Markets Conference--Policy Session 3: Learning about an ML-Driven Economy

2018 Financial Markets Conference—Policy Session 3: Learning about an ML-Driven Economy

This session examines the size of the likely impact of machine learning (ML) on the economy. Will changes to business models affect statistical measures of macroeconomic performance? If macroeconomic policymakers—especially monetary policymakers—conclude that ML is significantly affecting the overall economy, how should they respond?

Transcript

Vincent Reinhart: My name is Vincent Reinhart. I'm the chief economist and macro strategist with Bank of New York Mellon Asset Management North America. If you're confused by that, you should be. I think I've had three different titles in the succession of our programs. The reason actually is you're watching a merger midstream. I work at Standish, a fixed-income place. We've rolled together a couple of the other asset managers just last month. We're a 600-person firm with about $560 billion of assets. I'm their chief economist. That's enough about me.

This session addresses some of the potential economic consequences of the exponential expansion of artificial intelligence. By that, we mean what happens at the confluence of the rapid expansion of data availability, big data; storage, the cloud; computational speed, Moore's law; and technique, machine learning [ML]. Sometimes it feels like we're trying to drink from the fire hose, but we've got three of our panelists who will consider what it is going to do to the global economy, whether those effects can be measured accurately, and how central bankers should respond as well as other regulators.

Now we have three great panelists to address these issues and more. Sitting from this side, Rob Kaplan, president of the Federal Reserve Bank of Dallas; Carolyn Evans, the head economist and senior data scientist at Intel; and Charlie Evans, president of the Federal Reserve Bank of Chicago. Their bios are in the program, but you will find to your disappointment, the program doesn't identify if the two Evanses were really separated at birth.

However, if they or their near relatives contributed DNA to Ancestry.com or some of the likes, we will be able to answer that question whether or not they want us to. At some point, we're going to get into the question of privacy issues as well.

To start it off, we'll turn to Charlie, who's got some slides, then Rob, and then Carolyn. They'll speak for about 12 to 15 minutes. Thank you.

Charles Evans: Thank you. Thanks, Vince. Thanks to the organizers. It's really great to be here. This has been a fabulous conference. I really enjoyed the sessions so far. Before I begin, let me remind you that my comments here today are my own and not necessarily they don't reflect the views of the Federal Open Market Committee or the Federal Reserve System. This session is about how machine learning and related automation may potentially influence the macroeconomy, and in response, how a central bank should conduct monetary policy.

As we've heard at this conference, innovations in machine learning and artificial intelligence are big news, and they provide lots of opportunities and challenges for economists. First, let's consider the opportunities. We now have access to massive amounts of new data and better computing power. This is really cool for me and my staff as we are fact finders and data users. After all, most economists are data scientists at heart.

These tools are great for basic economic research, especially for the work of microeconomists, but there's lots of scope for macroeconomists as well. Some of these data may help us more precisely identify key policymaking parameters. For example, how consumer spending and business spending respond to tax changes. They may also help improve our ability to forecast short-term movements in the economy. In particular, we could see some exciting new indicators of the business cycle derived from new sources of big data.

Here, though, I have to agree with [Fed] Governor Randy Quarles and his comment last night. There may be limits to what we can learn from big data, at least for a while. The business cycle is, in essence, about small data. There just haven't been a lot of business cycles. Each has idiosyncratic features. Also, big data weren't available during past business cycles, so deep data cross-cycle comparisons are not easy. Basically, we're stuck with this situation, and that's sort of normal macroeconomics. This session is less about the use of big data and more about the consequences of large-scale technological innovation.

Now let's discuss the challenges. Such innovation could create challenges for monetary policymakers, if it leads to hard-to-identify changes in the structure of the macroeconomic relationships that might influence the business cycle. In the end, that's what we monetary policy folks are concerned about, the business cycle. The potential structural changes that come with innovation can affect the evolution of inflation and employment. As such, they may have implications for the achievement of our dual mandate objectives of maximum employment and price stability.

For instance, these changes could generate headwinds for inflation that mean we might need to provide more accommodation to reach our inflation target than we have in the past. But frankly, we don't know that. Perhaps these forces will lead to higher inflationary pressures that policy might have to counteract. There's a lot that no one really understands yet.

Of course, making policy in the presence of uncertainty in a changing economy is nothing new. We've dealt with plenty of structural changes before, and some were addressed in the previous session. History offers numerous examples. Chairman [Alan] Greenspan back in...he grappled with this issue almost 20 years ago when he considered the possibility of a new economy. Let me take just a little excerpt from an important speech that he gave.

"There is no question that events are continually altering the shape and nature of our economic processes, especially the extent to which technological breakthroughs have advanced, and perhaps, most recently, even accelerated the pace of conceptualization of our gross domestic product. We have dramatically reduced the size of our radios, for example, by substituting transistors for vacuum tubes. Thin fiber-optic cable has replaced huge tonnages of copper wire. New architectural, engineering, and materials technologies have enabled the construction of buildings, enclosing the same space but with far less physical material than was required, say, 50 or 100 years ago. Most recently, mobile phones have been markedly downsized as they have been improved. As a consequence, the physical weight of our GDP is growing only very gradually. The exploitation of new concepts accounts for virtually all of the inflation-adjusted growth in output."

That's the end of the quote. My memory of this is maybe the economist said GDP is getting lighter. This, from a technological standpoint, is really something. We don't know how various innovations will ultimately play out and how they will affect the economy, but I like to show in talking about the Federal Reserve and our objectives what I refer to as this bulls'-eye chart.

The bulls'-eye chart could have been drawn at any time, but it goes well back to our January 2012 long-run strategy for monetary policy where the Federal Reserve says we're supposed to provide monetary and financial conditions to support the attainment of maximum employment and price stability here on this chart. Maximum employment is that natural rate of unemployment, uⁿ, most recently the summary of economic projections has pegged that at 4-1/2 percent as the median, and our inflation objective is 2 percent on the PCE [personal consumption expenditures]. You can see our record going back to 2009 with a very high unemployment rate and just now approaching 2 percent.

Figuring out the effects of these developments is complicated. The sign, magnitude, and timing of their impact are all uncertain. Technological advances can lead to conflicting effects. For instance, internet commerce may make markets more competitive. This might lead to lower prices and push inflation lower in the short run, but it may also allow companies to price-discriminate better, making markets less competitive and leading to higher average prices.

These new technologies can affect the natural rate of unemployment, too. Online job boards and other technology may be improving matching efficiency. If so, the natural rate would be lower, but these new technologies can also cause the natural rate to rise. This would happen, for example, if people became more specialized in labor markets, less fluid as a result.

In this chart, I have graphed the most recent natural rate of unemployment at 4-1/2 percent. I'm not quite sure, I don't recall off the top of my head how far we let it drift in previous years, but certainly there were some commentaries when the unemployment rate was at about 8 percent, that maybe it would be very difficult to achieve unemployment below 7 percent and maybe 6 percent. In various folks' minds, some people had a much higher natural rate of unemployment. We will struggle with trying to identify that. We've struggled to understand the effects of changing u-star before.

In the 1990s, there were indications that the rise of labor market intermediaries, such as temporary help firms, was lower in the natural rate. And in 2010, an increase in vacancy measures without a drop in unemployment led some to conclude the natural rate had risen. Dynamic issues related to innovation may also cause difficulties for policymakers, because some effects might be different in the short run than in the long run.

Technological change also poses important challenges for the standard statistical measure of prices. Again, the effects can go both ways. On one side, standard measures might not be factoring in correctly the time cost borne by the user. For instance, when booking your travel tickets online, you cut the intermediary, and that's probably efficient as a whole, but you also have to do more work for yourself than you used to. The same is true for pumping your own gas or using the self-checkout line.

If not accounted for, this would understate inflation, but we might also be overstating inflation by not incorporating quality improvements, increased varieties in products, the amount of free content, and the like. Of course in either direction, such inflation mismeasurement has consequences for output mismeasurement as well.

Monetary policy relies on economic relationships between the tools we control, for instance, the short-term interest rate, and our policy objectives. Technological innovation may be changing these relationships. For instance, it must be much easier for firms to change online prices than it is for them to change prices in a physical store. That might make prices in the overall economy less sticky, which would change the parameters of the Phillips curve relationship that is important to much of monetary policy analysis. But this is pure conjecture at this stage.

Technological advances create a difficult picture to read and present a challenge for policymakers. The FOMC's goal is to reach the center of the bulls'-eye where the economy is at its natural rate of unemployment and inflation is at 2 percent. The inflation target is the choice of the central bank. As I've just discussed, technological change could also have some effect on the natural rate of unemployment. It could also influence how the economy responds to monetary policy adjustments, and thus the speed at which we're able to obtain our policy objectives.

However, the magnitudes and even the signs of these effects are highly uncertain. Fundamentally, what's important is the Fed's ability to deliver on our mandated policy goals of full employment and price stability, hitting the bulls'-eye. A policy focused on hitting mandated outcomes and managing risk against adverse scenarios can avoid missteps that might come from strict adherence to a fixed policy rule, something I often refer to as outcome-based policy. Execution of such policy often requires using informed discretion in instrument setting. In doing so, a central bank can do a better job in delivering on its ultimate employment and inflation targets. Indeed, we may never be able to come up with good estimates of how the various cross currents associated with AI and machine learning are affecting the aggregate economy, but we will be able to observe whether our current policy coincides with either restrictive, disinflationary financial conditions, or undesired inflationary pressures. We can then adjust the setting of policy accordingly.

While structural change may make our task more challenging, it's something we've been dealing with for a long time, and hopefully we can continue to navigate our way through it by keeping a close eye on our policy objectives. Thank you very much.

Reinhart: Thank you for that. I would point out that at the time the staff talked about Chairman Greenspan's lighter GDP as saying there's more GDP, but you can put it in a smaller box. I want to remind everyone that we do want you to submit questions through Pigeonhole, so please keep them coming, and now, President Kaplan.

Rob Kaplan: Thank you. Thanks for having me here at this conference. I think artificial intelligence and machine learning is a very important topic. At the Dallas Fed, we refer to this by another name, technology-enabled disruption, or this is part of technology-enabled disruption. In fact, we think of so closely with the Atlanta Fed that we're going to cohost a conference in a few weeks to talk about this broader topic.

Today, I'm going to talk about what I think technology-enabled disruption is, including AI and machine learning. Number two, how does it impact and how might it impact my thinking on policy? And number three, how might it affect my thinking on financial stability? I'll explain why I include that. I will say, as you'll notice from my comments, I've got in this area, I've got more questions than answers. If I've learned anything about looking into this area, it's important to keep an open mind, which I'll come back to and be open to learning.

I think AI and machine learning is an important structural change in the economy. Now a lot of people I talk with say, "Listen, technology has been replacing people, and it's been going on for decades and decades." I personally think what's going on now is different. I say this not as an economist, but as a former businessperson. It strikes me that technology and the distribution of technology power is more pervasive certainly than at any time in my lifetime. I think that may have to do with the cloud and many other aspects of the economy where computing power is now widely distributed. I think that makes what we're seeing now different.

I'm noticing that technology is replacing people at an accelerating rate that I, at least, have not seen in my career. What are the implications of it? Number one, certainly new business models are replacing old business models. People and companies are investing in technology to replace people with technology. The thing that's striking is when we used to talk about technology, we talked about northern California or Boston and a few places in between. Today, we're talking about every business. It's hard for me to think of a business that isn't being affected, whether it's a Coke bottler in Florida who's on the board of the Atlanta Fed who I've spoken with, who talks about how it's affecting the way they price, the way their salesmen or sales people have to act, the way their manufacturing works.

It's affecting dramatically car dealers throughout the United States who are selling most of their products now online, not through a salesperson. Salesperson used to be the most important person in a car dealer. Now that person is a commodity. The automotive technician, though, that person is the most valuable person now in a car dealer. That is a dramatic change, and it's certainly, for another example, affecting financial services.

The thing that's striking is while businesses are using this, consumers now have access to these very powerful tools. The impact of all this, I believe, is that business pricing power is being more constrained than any time in my lifetime. Unless you have a product that is very distinctive, you are likely finding it hard to increase your prices. By the way, while you're trying to agonize over the ability to increase your prices, there may be a disruptive competitor in your industry that could explode your entire business model, and you're very well aware of that, and that is further limiting your pricing power.

Because of the capital markets are so liberally funding many of the companies that are at the cutting edge of this, and you can imagine the examples who are trading at P/Es [price/earnings] that are over 1,000 times, people used to complain 15 years ago about competing with foreign countries that were dumping their goods. How would you like to today be competing with a company that doesn't even need a gross margin for the product they're selling, because they're using it as a traffic driver to fund other products and to get other traffic? That's happening to a lot of businesses. It's explaining why pricing power is so limited.

What are businesses doing to respond to this loss of pricing power? What I find is they're investing more in technology to replace people, number one. Number two, they are, like never before in my career, looking for greater scale, because while they may not think scale will help give them pricing power, it may be a way to combine costs so they can at least defend the margins they have. I think this is one of the reasons, for those of you who watch this, why we're seeing merger activity at record levels. I used to do mergers for a living. If you look carefully, and I still do, at announced mergers and their reason for them, I think you'll find that the dominant reason for many of these mergers is technology-enabled disruption, loss of pricing power, desire for scale in order to thin margins.

Used to be thought of that consolidation in industry meant you might have more pricing power. Today, consolidation industry is more likely driven by a loss of pricing power and a hope that you can at least defend your margins, or defend them longer, or have more money available to invest in technology to further replace people. That's what's going on, on the business side.

On the workforce side, that impact, I think, may even be more striking, at least to me, in that adaptability is now a key feature of workers who are going to successfully make it through and deal with technology-enabled disruption. Workers are far more likely to have to fire other workers during their career, and they're far more likely to get fired. They're far more likely to be in a company that will be disrupted out of existence or a function that may be restructured and disrupted out of existence.

I think there's a big change, and this is all a theory, and these are questions that we're trying to work on at the Dallas Fed, but elsewhere, there's 46 million workers in the United States that have a high school education or less. The statistics show, so far, that if you got a college education, you might, in fact, your participation rate in the workforce is much higher, your unemployment rate is much lower, and I think you're likely to be more adaptable in this type of information-driven, technology-enabled, disruption-driven economy.

On the other hand, if you are one of those 46 million workers who has a high school education or less, you may very well be more frequently seeing your job either restructured or eliminated. In a tight workforce, I think you'll find another job. The question is do you go from that call center worker that we talked about in the earlier example a couple hours ago, where does that worker go? Where does the truck driver go in the future that gets disrupted? Unless they get retrained, they may well go to a job that is where they make less income that is less productive than the one that they left.

This is one of the conundrums, I think, for me as a policymaker. Every business I look at and every industry I look at is more productive than it was 10 years ago. People say to us, "Don't you guys understand? Business is more productive." Yeah, but we measure productivity workforcewide, not industrywide, workforcewide. I think what's happening is while companies and industries, this is a theory, a thesis, while companies and industries are getting far more productive, what's happening to the workers that are being either seeing their jobs restructured or being eliminated? Where are they going? If they are, in fact, going to less productive jobs, and we measure productivity workforcewide, is that a part, at least, of the explanation why productivity growth has been more sluggish?

We've got a lot more work to do this, but at the Dallas Fed, this is a working theory we have. They think this may be one of the reasons why despite all of this visible increase in business productivity, you may not be seeing workforce productivity advance the way you'd expect.

These are questions. What's the impact on productivity? What's the impact on inflation? My own view is while cyclical forces, cyclical inflationary forces are building in a very tight labor force, this structural headwind of technology-enabled disruption is likely limiting worker negotiating power. I've already mentioned, I think it's limiting pricing power. Certainly the question is what's the impact on wages? I think the impact on both wages and inflation in my own view is likely more muted.

The last comment I'd make is there's probably in the workforce, and we're seeing this in the environment, there's likely a greater gap in this country between the haves and the have-nots, heavily based on level of education and adaptability. I think the challenge, I believe, for the United States, which is why we spend a lot of time around the Fed working on this is we've got to improve math, science, and reading scores in the United States. We now rank 25th out of 35 OECD [Organization for Economic Cooperation and Development] countries. Those skills are more important in the adaptability world we're heading into.

Our early childhood literacy, college readiness may well be far more important, and certainly skills training. We're working on skills training at junior colleges all through the United States. The question is, are we doing it fast enough to keep up with this accelerating pace of technology-enabled disruption? I don't think we are keeping up. We've got to spend much more time, I believe, in this country, and I think as a policymaker, I think about, are we focusing enough on human capital? We focus a lot on investment. We focus on a lot of gadgets, functions, but are we focusing enough on the quality of our human capital, and is it lagging, all this technological development?

I would listen very carefully to the paper that we did in the last session, and this idea of distribution and dissipation and lags. I think one of the lags is we're not investing in our human capital enough to keep up with the rate of technology-enabled disruption. Anyhow, it's a question for the Fed and as a policymaker, your first question to always ask is, "So what's the impact of this structural driver on our tools?" We know a lower fed funds rate seems to be having a very stark and pronounced effect on tightness in the labor force. Does it affect inflation in the same way that it might have 10 or 15 years ago? Question.

Second, in terms of our analysis, aggregate data is what we often use, but given each industry is different, and given as it was discussed earlier, there's a dispersion of how technology is affecting industries, do we need to get away more from aggregate data? There are challenges to that, because it takes a lot more work, and segment much more how we look at technology-enabled disruption by industry.

Can we run a tighter labor force? Is the NARU, natural rate of unemployment, naturally lower because of this trend? These are all questions I don't know the answer, but I do know it's critical that we keep an open mind, that we're open to learning. I also think while data analysis is very helpful, and certainly if we could segment data more in our analysis, I think that would be helpful, talking with businesses and contacts by industry, my view, which we do frequently and broadly at the Fed, especially in the regional banks, I think it's never been more important, because I think data will lag this development. I don't know that there's good data. There will be good data on technology-enabled disruption that it may well lag the actual trend. We just have to be mindful of that.

Last comment, I mentioned financial stability. One of the concerns I have when I look at machine learning and artificial intelligence is when I look at the financial markets. I'm speaking as someone who spent his entire adult life, my entire adult life, in the markets. I am very concerned about the percentage of trading in markets, or very mindful of it, I should say, trading in markets that is quant driven, electronic driven, particularly in the last hour or two of trading during the day. I'm very mindful of various strategies that are various forms of vol targeting, if you're familiar with that, risk parity that are quant driven.

There are varying estimates of how big these strategies are. The best estimate I've heard somewhere between, this is a broad one, $250, $500 billion. The issue is these type of vol electronic trading strategies are linear when the market is going up, and then the amount of selling pressure they create is exponential on the way down, like a put. You're familiar with puts, how they work. You exponentially lose more money on the way down.

My concern, and we saw a little bit of this, I hope not coming attractions, in the first two weeks of February. I don't know that this is well understood, and I also don't think there is great data on this in the shadow financial system. We can't do stress testing. The Fed obviously does not oversee the nonbank financials. We do do stress testing on banks, which gives me a lot of comfort, but I think there may be a need for more stress testing and greater visibility on what's going on in the nonbank financial sector, because I worry about the growth in artificial intelligence and machine learning are speeding up and maybe creating more embedded leverage than we recognize. Thank you.

Reinhart: Thank you. I was particularly thrilled to hear a senior Federal Reserve official use the word conundrum. There'd been, for some reason or another, 12 years where they've been avoiding that. Now Carolyn, you can take the floor, please.

Carolyn Evans: First of all, I also wanted to say thank you for inviting me to speak. It's been a great conference so far. I'll also mention these are my views and not the views of Intel. I'm going to talk sort of in three chunks. The first is very briefly the history of AI and why that matters now, and then three ways in which machine learning affects the economy or structural changes we might see, and then finally talk about some accelerators and impediments.

First of all, the history of AI. That was touched on in this morning's first session. The fact is, it's been around for a long time, so 1950, Turing wrote a book [asking], "Can Machines Think?" It's been around for a while. OLS [ordinary least squares], sort of basic linear regression, is supervised machine learning, right? You call independent variables features. You call the dependent variable the target variable. You call estimation training. It's essentially the same thing. Clearly there's differences. One difference is machine learning cares a lot more about yhat, and OLS cares more about beta hat. We care more about causality. Both fields care about it, but there's sort of a difference in emphasis.

That's the history, but why now? Why is it taking off right now? I would say there's three reasons. I'm going to refer back to these at the end when I talk about accelerators and impediments. The first one is compute. We have much more compute power now than we did historically. That has enabled sort of the revolution going on right now. The second is data. We can call it big data. It doesn't have to be very big to get some interesting insights out of it. Data is the second one. Then also algorithms. Again, this morning's talk discussed that a little bit. That's sort of why now? Why is it becoming so important now?

Next, I'll talk about what are three ways that we can think about machine learning affecting the economy? I'm going to sort of have three different aspects I'm going to talk about. I'm going to give some very concrete examples. These examples will sort of illustrate how it's already affecting the economy, and then help us think about or help me think about what could be the broader impacts? I'm going to talk about supply, demand, and business models. As economists, we like supply and demand. Then how do those two come together business models?

When we think about supply, and we've had some discussion about that already, but some of the things that we can think about are it can really reduce costs. Google, for example, there was a recent paper out about how they've dramatically reduced cooling costs at their data centers by looking at the data, doing some analytics, and figuring out how they could optimize and reduce cooling costs. That's one thing.

Another thing is supply chain optimization. There's been a lot of progress made in optimizing supply chains using machine learning. Both of these are ways in which businesses can reduce costs. They could also change the way that supply chains are organized. To me, those are two potentially important ways that supply could be affected by machine learning.

Then I'll talk about demand. Two things, sort of two areas I like to think about first of all are product recommendations. For everybody who has a Netflix account, you know you can go and they'll give you recommendations about what movies you should watch. There's a machine learning model behind that. These days, they don't necessarily just use what you rated before. They'll use, "What device did you watch it on? How long did you watch it for? When did you stop? Is that show like other ones that you've watched?" That's all feeding sort of a product recommendation in terms of what movies to watch.

Amazon also, we've all, if you use Amazon, they have recommendations about what products you could buy. We all are familiar with that, and it sort of makes our life easier. But one sort of implication that I'd like to suggest is in a sense, it increases the efficiency of my shopping process. It reduces my search. It gives me more time to do other things, as opposed for searching for what I'm going to buy.

The second example that's going to be along these same lines is something called the Amazon Go store. I don't know if anybody's been there? It's in Seattle. You have an app on your phone, and you just swipe it as you go in. You literally just put stuff in your basket, and you walk out. I've been there a number of times, and I went with my mother recently. I was picking up stuff and putting it in her basket, just sort of doing this. We walked out, and she looked, she said, "Oh, they didn't charge me. What happened?" They had charged me. They had seen that I was actually taking that stuff up, and they charged me instead.

How does AI play a role there? Well, if you look at the patents that Amazon sort of applied for a number of years ago, one example they gave is how do you tell the difference between a hand that's picking up an object and a hand that's putting it down? You can go to the Go store, and I've tested it, because I kind of wanted to see, "Well how does this work?" Picked stuff up, walked around the store, and then put it back, and I didn't get charged for it. There's a lot of deep learning that goes into that.

You know what kind of deep learning this is? It's image recognition. You have to have a training data set where you have all sorts of images of arms putting things down and picking things up. You're dependent variable can be 01, is it putting it back or picking it up, and then you've got a whole data set that you can use then to when people are in the store, and you can sort of figure out, "Are they picking it up or putting it back?" That's just one example.

What does this perhaps mean for the broader economy? Again, it streamlines that customer experience. Rather than having to wait in line, I can just sort of get through there very quickly. Again, that changes the amount of time that I have to spend on these other things. Just a question, does it change the labor/leisure trade-off? That I have more time to work or to do something else, because I'm not doing things like search, which is a case of product recommendations. I'm not doing things like standing in line when I'm buying something. That's sort of one implication for structural, these sort of consumer side things.

The other question that I have is, does it create a stickier customer? When we talk about prices, we often don't like stickiness, but when a company is talking about customers, they do like stickiness. They want a customer to stay with them. If a company like Amazon has all of my buying history, and therefore they can make a better recommendation to me in terms of what I buy, does that make me more likely to stay with Amazon? It sort of changes that customer/company relationship also via the ability to make recommendations, in terms of your history.

Okay, so I talked about supply, I talked about demand. Let me talk about business models now. This is something that's really interesting. When you think about the traditional business model, so say an auto manufacturing company, you get parts, you have workers, you put them together, you sell them to a passive consumer, and the consumer buys the car and goes and drives with it.

Now let me contrast that with a new type of business model that has emerged. Rather than it being sort of an end-to-end business model, it's sort of multimodal. You have different sides of the business providing value. Let me just give you an example. Let's say one example is ride sharing. We can talk about Lyft, we can talk about Uber. You have the drivers there driving, so they're providing something to the business model. The customers are also providing value. They're providing value in a number of different ways. The more customers that Uber or Lyft has, the deeper the market, so the more able they're going to be to attract drivers.

Okay, so just my simple presence of using that company contributes value. I'm also contributing value in terms of my data. Machine learning is important to a company like Uber, because they have to do things like predict how long my wait time is to get a car, or predict which driver they should give me. I don't know if anybody's ever used Uber Eats, but that's sort of how long is it going to be before I get my food?

The more customers I have coming to my platform, the more data I have. Therefore, the better is my product, because I can train my models on a broader set of data. It's a different model than the consumer being just a passive purchaser. The consumer is actively contributing to the business model. That has a number of implications. One, it changes the nature of production, the nature of competition, the way that value is created, and it also could have the stickiness impact that I talked about on the consumer side.

Also, I'm not the monetary policymaker here on the panel, so I'm going to leave that to you guys, but the question that I have is, does that change the nature of prices in the sense that because the consumer is contributing something, does that sort of change the nature of pricing in that industry as well?

Okay, so I talked about supply, demand, business models. The last thing I'll talk about accelerators and impediments. Accelerators, the first one I'd say is cloud. Okay, so why is cloud important? First of all, it changes the cost paradigm for compute. In order to train a machine learning model, you need a lot of compute, but maybe you don't need that compute sort of for now or for the next five years. Maybe I just need that compute very briefly.

When I have access to the cloud, I can dial up and down how much compute that I have so that it makes it easier for me to train my models, or to do the kind of machine learning that I want to. That's how cloud is an accelerator. It also reduces the cost, rather than having to pay an up-front cost to buy a server to do my compute, I can just dial up and down on AWS [Amazon Web Services].

Cloud is also important because it enables AI on the edge. In the technology industry when we talk about edge, what we're talking about is doing compute sort of at the sensor in the factory or in my smartphone or something that's not sort of the central data center or the server. Because of the cloud, you're able to get more of that AI at the edge. For example, if I tell my Apple Watch, "Start a workout," there's no compute that happens on my watch. Instead, it's sort of communicating with the cloud. That's why cloud is an accelerator.

The second accelerator I would mention is internet of things or IoT. I'd include smartphones with that as well, because those are sort of things on the internet of things. Those are generating a lot of data. We are very, very much at the advent of what's going to happen. We heard in the last session about there's lags in these things, there's implementation, there's capital costs. I completely agree with that, because I think it just hasn't nearly gone as far as it could, but as that sort of spreads out, I see that as a big accelerator.

Smartphones, in the U.S., we have about 60 percent penetration rate, so 60 percent of households have a smartphone. China is maybe 90 percent. You go to a country like India, maybe 20 percent of people have a smartphone. There's a lot of room to run, in terms of increasing smartphone penetration, which in turn is increasing data generation, which in turn feeds AI and ML.

Third accelerator, AI for everyone. That's sort of what I call it, but there's been new products like Azure Machine Learning, Amazon SageMaker where you can essentially go and do drag and drop ML. you can say, "Here's my data set. Clean my data set, draw line. Estimate my data set, draw line. Do my prediction," and given that one of the impediments to wide spread adoption of ML is the fact that there aren't sort of people who are able to do that. There's just sort of a shortage of those kinds of workers, to the extent that these tools spread, I think that that's likely to be an accelerator.

Finally, I'll mention an impediment. There's a lot of other impediments as well, but there's just one that I wanted to mention, and that's privacy concerns and regulation. I don't know if everybody's familiar with GDPR, which is the General Data Protection Regulation in Europe. It's coming actually this month, it'll be active. It essentially increases dramatically privacy protection for basically any entity in Europe. There's large fines associated with it, various restrictions on using data on any European entity.

Now data is what feeds ML, so to the extent that data is not available or is restricted in use, that's an impediment. We'll sort of see where that goes, but that's sort of one thing that I think we need to keep an eye on. That was my comments, and I look forward to your questions.

Reinhart: Thank you, Carolyn. You don't have to wait very long to get a follow-up question when you were testing at the Amazon Go store and picked something up, and then put it back down. Did you have to put it back down in the same place?

Carolyn Evans: I didn't try. I'm going to try that next time. That's a really good question, because I don't know, I kind of tend to be a neat, tidy person, so I try to put everything where it's supposed to go. I put it back in the same place, but I should try that.

Reinhart: Okay, so the store was more organized when you left than when you got in, is that it?

Carolyn Evans: Possibly, just possibly.

Reinhart: Before we turn to your questions, and we've been getting a lot of good ones, I'd just like to take advantage of three people in the room who've recently thought very hard about the issue of learning about an ML-driven economy, our panelists. Is there anything in the presentations by your colleagues that you heard that you would amend your comments, or you would like to add on or just generally opine? We'll go from Charlie, Carolyn, Rob.

Charles Evans: No, I'm fine. I'm fine.

Carolyn Evans: Yeah, me too.

Kaplan: The one comment that struck me, I was listening carefully, that Carolyn said is about these accelerators, which made me think my view is this is likely accelerating, but your point is the more ubiquitous all this comes, it's going to go like that. We're going to be dealing with a whole new reality.

Carolyn Evans: Well, to the extent, especially IoT, or internet of things, because the more data you have, the more able you are to train your models, and the more insights you get. It does accelerate. Yeah. We'll see where it goes, but yeah, I would say that's fair.

Kaplan: That last comment, it made me wonder this. We've been talking a lot about this lack of dynamism, the fact that such a large percentage of new jobs in this country come from new businesses, but we've also been noting it's harder. We've been sluggish. I wonder you have any reaction, Carol, how is this impacting the ability for a small businessperson to start a new business? Helping or hurting?

Carolyn Evans: I would say there's two aspects of that. One is, and this isn't going to only speak to ML, it's more generally. As a small business, I'm able to go to the cloud to do my compute. Maybe I need that extra compute, and now I don't have to buy a server. I can run my email. I can run my accounting. I can do all sorts of things that I used to not be able to do. In that sense, I think it's good for small business.

What I would say perhaps makes it more difficult is when you have players like the Facebooks, like the Googles, like the Amazons in the world who already have all that data and have a very sticky relationship with their customer, I think it's more difficult. Though I will amend that by saying Amazon has essentially become a marketplace for small businesses to sell their stuff, right? I think there's probably two aspects to that.

Reinhart: A related aspect, is it harder for a small business if they can't actually recruit the talent they need when there are such big players out there?

Carolyn Evans: Is that for me?

Reinhart: Either.

Carolyn Evans: Okay. Yeah, so I think potentially yes. I think you talked to recent undergrads about where they want to work, where's the sort of hot place to work. It's the Googles. It's maybe not Facebook as much anymore, but the Googles, the Apples, the tech companies. Actually, when I was in academia for a while, and this didn't happen 10 years ago, but now they're going to go have a start-up and make a million dollars a year. That's sort of another sort of trend that I see.

I will mention a thing that, in a sense, sort of plays against that a little bit is to the extent we have this AI for everyone so that, and I'm referring back to the earlier comment where we have our workforce, and we'll retrain them or tool them up to be able to do some of these things. Those types of tools, like Azure Machine Learning Studio, those sort of make it easier to do that. I think it could cut both ways, and at this point, I don't know which way it'll go.

Reinhart: Okay. Twelve years ago, the answer would have been Goldman Sachs on university.

Carolyn Evans: That's right. That's right.

Reinhart: We've gotten a number of good questions. I want to represent everybody's interests here. They really group into four categories. What about market structure? What about the microstructure and welfare implications of all this? What does it say about the macroeconomy, and with regard to macro, how about some issues of finance, financial markets, and regulation?

In terms of market structure, first, I guess the question is with declining margins and the commoditization of products, is there a race to the bottom? Essentially, there's often a first-mover advantage to benefit from the economies of scale and network externalities. Is increased concentration inevitable? And if so, pricing power increase because of the dominance of a few firms, or reduce because of reduction in search cost? We'll go to Rob first, because there was a follow-up to hear you say a little bit more about margins. Why are they so low?

Kaplan: I'll say this now as a former businessperson. I'll tell you what's alarming to me is when you see companies that have such scale and are in so many different products that they can decide to get into an industry. I guess we don't mention individual companies up here. A big firm that got into the grocery store industry very visibly earlier this year, and decided that basically it's clear on some of their products, as I looked, they sell them for no gross margin, because they drive traffic, and their data is so good that they can incrementally make money, and the stock market is rewarding them for it.

To this question, the race to the bottom, I don't know if it's a race to the bottom, but I think it's very difficult. If you're selling one of those products and you're competing with that, I really don't know what you do.

Reinhart: Anyone else have any views on this?

Charles Evans: I have a hard time sort of getting my mind around exactly what markets we're talking about. I think it's always difficult if you're in one of these industries where everybody's learned how to produce this, they figured out how to produce it very cheaply, and it becomes pretty much commodity like. What used to be a very good business where people could earn healthy profits in your local community, get a country club membership and everything that goes with that, then it becomes a commodity, and you're just like everybody else, or you come up and move on to another sort of product or service provider.

I think that that's part of the natural evolution in any capitalist marketplace. I definitely think that technology, computers, our ability to take standard OLS regression techniques and drive it to your phone, and how that gets used when I walk into a store. That is extremely powerful and it's going to change things. Yeah.

Kaplan: The surprise to me has been there've been industries, and I can give an example, because I take...any of you take taxis? Taxi cab industry used to be lots of people, made a living, middle class. Now a new entrant, technology-enabled, logistics business basically, where the people that work for that new entrant, they actually can't make a living either, but they normally do the job, statistics show, as a second job. For those who were doing it as a profession, they can't make a living either.

The speed at which it happened was basically in the blink of an eye. That's the thing that strikes me. I know, speaking to businesspeople, and as a businessperson, it's having a little bit of a chilling effect on businesspeople when you see just in three or four years that type of thing happening to your entire industry, and capital investment that was made, say the medallion taxi that was worth a million and a quarter just four years ago is now worth maybe 200 grand. I see it as much more widespread than I remember any time in my lifetime.

Reinhart: Yeah, to follow up, there's a question, ride share business model sounds like a natural monopoly. Is it the case that regulation is just behind? How much are we supposed to be embracing it, or how much are we supposed to be controlling it?

Kaplan: I don't know the answer. That's the issue.

Carolyn Evans: Yeah, to me, I talked about this new business model where essentially a platform economy, it has some flavor of a barrier to entry, right? Because you have a company that's sort of already set up this network, and because it has that network, it's really hard to break in. Right now we have Uber and Lyft essentially in ride sharing. The regulation question though, that's not clear to me, because I always hesitate about regulating a technology. Then what happens? What's the downstream effect? I wouldn't comment on that.

Charles Evans: I'm not quite sure I understand the premise of the question exactly. One reason why you've had the competition of taxis is because they were highly regulated. There was a barrier to entry. You had the Medallions, and so all of a sudden, as is usually the case when you have regulation and somebody's making pretty good money, rents and some sort, somebody's going to try to whittle that away. Now you can do it by trying to change the regulations where you have an advantage, or you just sort of go in, and you wait to litigate on how the regulations will be enforced. I think that some of the Uber and Lyft has been along those lines.

Now natural monopoly makes me think that the question poser is thinking about congestion effects in a city where you've got all these vehicles on the road because of the competitive environment. There might be a reason for wanting to do that. There's an externality, but in terms of competition, it's certainly true that Uber and Lyft have a lot of data and a lot of stickiness there, but smart people could combine this with another type. The technology is there. That's easy enough to do. Combine that with another product, and all of a sudden you could get a different loyalty of sorts. It's hard to predict things like that.

Kaplan: Thirty, 40 years ago when I was growing up, my parents, I was sitting in the back seat of the car, and they'd be driving. They'd say, "Get a profession, because we don't want this to happen to you." Right? Today, that's probably still good advice, but I guess my point is, and this is as a former professor, I think the advice has never been as true. You're going to have to learn. If you're a worker today, or you're coming out of school, or you're starting your career or in the middle of your career, the need to be resilient emotionally and adaptable, and the risk of or probability of a surprise that will radically change your company, your job, and your career prospects strikes me has never been higher.

I think the antidote to that, I think is higher skills and higher education, but I'm not sure, but I think it's having a profound impact on human capital and the workforce in this country. I think maybe there's a lag, as the professor said, but we're going to have to find ways to catch up and help human capital become more adaptable. I don't think we've done it yet.

Carolyn Evans: Yeah, and I would say traditionally, I think an undergraduate degree ideally teaches you how to learn, right? That ultimately is the lesson I think is you have to know how to learn, and you have to be willing to go out and learn, because I think one of the questions from the earlier session, things are changing so quickly, and I learned something, but then it's obsolete. That's kind of the way it is now. If you know how to learn and how to teach yourself, then you'll be fine. You'll just go out and learn the next thing.

Reinhart: As opposed to the three panelists, there are a number of people in the audience who have a government solution to this. Indeed, there are five votes, if technology enables some workers to consume more leisure, increasing their welfare, but creates unemployment for others, should not there be some type of leisure tax? I was wondering what the group might think about that?

Kaplan: An Amelia Island tax.

Reinhart: I think Charlie is speechless.

Charles Evans: Well, it looks like a micro exam question. I'd prefer a written exam where I can sit and think for three minutes before the oral. Go ahead.

Reinhart: Anyone else want to weigh in on this?

Carolyn Evans: I'll pass on that one.

Kaplan: I don't see how this will ever be politically palatable to do something like that.

Charles Evans: There are euphemisms at work in this, right? Enable some workers to consume more leisure, increasing their...I think that means wealth and not having to work as much, and then putting...I don't know.

Reinhart: I think I read it as wealth tax.

Charles Evans: Something more direct would be...yes.

Reinhart: Yeah. Okay. Why don't we switch to the macroeconomy for a little bit before we go back to some welfare questions associated with the microeconomy. High up on the list is trying to put together what the view...two parts. One is productivity growth is higher. Doesn't that say something about r-star? Market power is going up. Doesn't that say something about your policy's goals? Let's go to the Federal Reserve goals of policy. Oops, sorry. If it is the case that lags explain low productivity growth, and it's coming back, doesn't that mean r-star is being currently held down but will rise over time? Perhaps you two on the bookends have had discussions of r-star eight times a year. How do you see this as interacting?

Charles Evans: My colleague John Williams likes to mention r-star about 15 or 32 times a year, because that's because of his research, which is fabulous. The premise of this is entirely plausible. I think that equilibrium, real interest rates, the short-run variety have been low. I think longer term, r-star has also been low. What's the stance of monetary policy? We've had nominal interest rates low for a while. If all of a sudden, r-star starts taking off, if we get this productivity acceleration the way the Greenspan Fed had to deal with in the late '90s and early 2000s, then all of a sudden this low-interest-rate stance would be extraordinarily accommodative.

I like that type of analysis, because it suggests that inflation ought to be picking up relatively quickly, if not quite confidently at least. When things work in the same direction, I think that helps us set monetary policy in a world where things are changing, and you might kind of go, "I have no idea what monetary policy should do, because velocity is moving, or all these kinds of things are moving around, but when the implications are it should be showing us to higher inflation, and that's one thing we've been concerned about, then I think that the way would be relatively straightforward.

Kaplan: I've said this, but I hope I'm wrong, I'm skeptical about growth in the medium term: aging demographics, sluggish productivity. I'm more skeptical about whether that will improve, mainly because I don't see us making the investments in human capital, skill levels, and education levels. Maybe there'll be an innovation in AI that will expedite it. That would be great. Then thirdly, I'm also concerned while increases in government debt to GDP has been a tailwind and is a tailwind right now, if we ever get to the point which we're bound to where we're going to have to start moderating debt growth or even de-leveraging, I think that's another headwind for economic growth. I do believe that a lot of what you're seeing in the yield curve and lower r-star has to do with skepticism about medium- term, sluggish medium-term growth.

If there was something that caused productivity growth to pick up or an innovation that did, that would be welcome. I'd be thrilled. I don't see it yet, but I was listening carefully to Carolyn. It sounds like you think AI maybe if it's widely usable, and allow people with less level of retraining and education to use it, that could be really helpful. I'm a little encouraged by that. As a policy maker, I'm encouraged by that. Right now, I'm concerned.

Reinhart: So Carolyn, you said you're not a macroeconomist. As opposed to me, you don't play one on TV. Do you want to weigh in, or do we give you a pass?

Carolyn Evans: I did work at the Fed. It's in the water at the Fed. I agree with what you said. I'm optimistic about that, but I don't know how long it will take, in terms of how much sort of these new technologies will enable the more widespread adoption of AI. I also watched the previous paper with a great deal of interest. I haven't decided for sure yet sort of what that timeline is. So I would agree with what you said.

Reinhart: The next logical follow-up is to talk about inflation determination and this sort of world, as Charlie started talking about it. If low inflation is more persistent because of technology, should that change how the Fed normalizes rates? Is that already embedded in r-star? I'd also like to follow up on President Evans's remarks in that we've been talking about pricing power and the old models of rules versus discretion, if there's more pricing power, that actually argues for the Federal Reserve to have a higher inflation goal, if, in fact, what we're observing is less pricing power and more lower search cost and it's flattening the Phillips curve. Maybe it makes you a little less zealous about your inflation goal, because you can pursue full employment a little more actively. Where are you, both in terms of what the right level of inflation should be, and how you respond to inflation and unemployment deviations? Jump ball, whoever wants to go first.

Charles Evans: That was a good question. Those are important issues. I'm reminded of sort of the early 2000s, maybe the mid-2000s, and at Jackson Hole. And it was when great researchers, Mark Watson and Jim Stock, were evaluating the Great Moderation while it lasted, and sort of what were the causes of the Great Moderation? Was it good monetary policy? Was it good fiscal policy? Was it good technology? Was it just the American system and all of those things?

A lot of people said that productivity was an important part of that. Related to that, I have a vague recollection of Chairman Greenspan sort of weighing in and sort of saying, "Productivity has been so strong, it's hard to get inflation up. Prices are just going to be like that." Ken Rogoff came along and sort of reminded us all, "Look, you're the monetary authority. You're supposed to deliver the inflation rate," which is a nominal phenomenon. You just have to drive harder.

Maybe if you've got a pricing environment that's going to lead to lower inflation on average, then that means that you need to be more accommodating in order to get that up. I think that's sort of my take on all of those issues. I think you kind of snuck in there, what do you think, maybe it should be higher? Which is sort of this alternative monetary framework question. Are we likely to be hitting the zero lower bound more often if we have technology interacting in a way that makes it hard for us to get inflation up? We might need more accommodation.

I think that's a discussion that the Committee, I think pretty much everybody needs to have a discussion to sort of understand how monetary policy works. I think there's good reason to believe that in the moment when you need it most, it's hard to change a framework. It's certainly more difficult to get people around to that idea. Talking about it while it is still sort of an intellectual exercise is important. Part of that could simply be we're going to continue to run with 2 percent as our inflation objective. If that's the decision, what's the best way to ensure that we minimize the risk of hitting the zero lower bound?

Reinhart: It's also hard to have that conversation in flight, because it looks opportunistic, as opposed to structural.

Charles Evans: It does. Yes, that's right.

Reinhart: Yes, sir.

Kaplan: My two cents is, and obviously, this is an issue we debate and re-debate regularly, the short-term forces. We've talked about the headwinds: demographic, sluggish productivity, other issues, but we've got a lot of stimulus in the economy. We're running a very tight, historically tight, at this point, labor force. You've got, in the short run, cyclical forces building, should be creating more wage pressure than we're seeing. I still think we will see it, and should ultimately, you would think, in some of these other policy decisions being made, steel, aluminum, input costs going up. We, at the Dallas Fed, think energy prices, the risk is to the upside on energy in the next three to five years, not to the downside.

Those are short-term forces that should be pushing up inflation. If you go out longer, though, I think a lot of these structural changes, unless there's something positive that emerges, I think I would say I'm concerned they're deflationary forces, including deleveraging, which we haven't really dealt with in our lifetime. This issue of r-starred we're debating regularly, and I'm still a little skeptical, we may see shorter term ... This is the challenge for policy, I think, in that how do you deal with these shorter-term forces now and potentially a significant overshoot on what we would have thought was full employment? Then what do you do when you get over the horizon?

For me, this all translates out into somewhat flatter path of the fed funds rate than we're historically accustomed. It doesn't tell me we should not be removed. It tells me we should be removing accommodation, and we should be doing it deliberately, but it also tells me that where we end, the terminal rate and the pace and the shape of the curve may be flatter than we're historically accustomed to.

Reinhart: Carolyn?

Carolyn Evans: No, I'll pass.

Reinhart: Okay, this one's easy. It's just a yes or no for the three of you. Do you think NARU is lower than 4-1/2 percent?

Charles Evans: I think that there are reasonable arguments for thinking that the natural rate of unemployment could be lower than 4-1/2 percent. My own staff has done research on labor force participation rates for quite a long time, Dan Sullivan, Dan Aaronson. You could go back to the early 2000s, see the demographics, and other people have done this, too, but I think Dan and Dan were on the early end of this. You could sort of see there's a downward drift that was coming from demographics, from different segments of the labor force attachment to the workforce during certain times.

We sort of followed that down during this period, and as recently as last fall, the advice I was getting from them was, "Well, we think the best estimate with big uncertainty is natural rate is going to be 4-1/2 percent in 2020, but it was going to be 4.8, but it was going to go down by five basis points each year.” That's because of the demographic factors taking you down.

We've actually gone down more rapidly with that with the unemployment rate. I haven't seen a lot of signs of inflation. It certainly could come, and I certainly agree with Rob that there's a lot of fiscal stimulus in the pipeline. It's a very different aggregate demand environment than what we were looking at a year ago.

Now having said that, if I could just say one more thing about this, trying to estimate the natural rate of unemployment and being sympathetic to, maybe it's lower. Maybe it's lower. I remember Tom Sargent many years ago in his book The Conquest of American Inflation, and he talked about how...it's really sort of a machine learning in some kind of way. It's like a monetary policy staff econometrician, trying to estimate the slope of the Phillips curve over time with lots of data that keeps coming in, finding it's flat, and sort of being led to think that the natural rate of unemployment is much, much lower, and then getting lost in that and ultimately delivering accommodation that is inconsistent with a 2 percent inflation rate.

It was only by my interpretation of this is sort of I think he might probably have said, "Take that discretion out. Stop searching for it," but I also think anchoring inflation objectives at 2 percent, so you're constantly trying to go, "If we're going to keep going down, but then we find inflation is going up," well, that was sort of what the 1970s central bank wasn't really paying enough attention to. I think there are a lot of lessons in this discussion and with the theme of the conference.

Reinhart: Either of you two want to say yes or no?

Kaplan: Well, listen, the short answer is, as I sit here now, my staff at the Dallas Fed, I think 4-1/2 percent is probably a good point estimate on what we would have said was full employment. It's probably not unusual. Having said that, we're all saying the same thing, we're keeping an open mind. Maybe it is lower. The embedded question related to that is, and a lot of people are raising it, aren't there a lot of people on the sideline that are going to come in? And so that maybe there's more slack than this unemployment rate would suggest.

I'm skeptical. I'd like to believe people on disability, others that are out of the workforce will come in, but we still believe our work at the Dallas Fed suggests that labor participation rate disappointed people, it ticked down to 62.8. Our work at the Dallas Fed suggests over the next 10 years, it's likely to migrate down closely to certainly below 62 percent, maybe closer to 61 because of aging demographics. Second, if you look at U6, which in the lingo—unemployed plus discouraged workers plus people working part-time who would like to work full-time—that's down to 7.8 percent, and I think that now beats the prerecession low, which I think was around 8 percent.

I'm skeptical whether there's a bunch of workers on the sideline, which is why when you see the SEP [Summary of Economic Projections], the dot plot from the Fed, people like me have said that I think the unemployment rate is going to get down to 3.6, I think is the number we submitted, which is sort of a breathtaking number. It basically is a nice way of saying, I don't think you're going to have some new source of labor enter this market. I think we are very tight, and we're getting tighter.

Reinhart: Why don't we transition to, there's a number of questions about the microeconomics of all this, including some more about welfare and also some about protection of data. I guess the main point is the skills necessary to employ machine learning very materially among people and the opportunities to take advantage of it vary considerably across industries and firms. Will income and wealth inequality worsen as those with skills benefit disproportionally? Will the widespread application of ML empower lower-skill workers or marginalize them? Why aren't we better at retraining? I think Rob wants….

Kaplan: There's at least a couple of big problems with retraining. I've been a big supporter of it. I think we need to dramatically ramp it up in this country, but the reason it's so hard is number one, it has to be done more likely than not locally, okay? If we're going to build retraining in the United States, it's got to be done in hundreds of locations, in hundreds of junior colleges locally while worker mobility is historically low. It means a local junior college is going to have to go out, and there's lots of good examples. I interview businesses, and then backward integrate and offer the jobs. I've noticed in the United States, there's some great success stories. Dade Community College is a great example, Dallas Community College is a great success story, El Paso is a great [example], but there aren't enough of them, because it's got to be done. Means the dean or the chancellor has got to be the type of person who thinks in this way. I think it's very uneven in the United States. This is why this is slower than we would have expected.

Secondly, it's easy for me to say you should go out and get retrained when you're 47 years old and you've been disrupted out of your job. I've changed jobs twice. I went from being a banker to a professor and now with the Fed. It's a pain in the neck to get retrained. I did it at a much different level. It is hard emotionally for people to say, "You get to now start over at that age, and you're going to have to sit in a classroom and get retrained." I think that's another psychological reason, if nothing else. This is so difficult. It sounds great, but it's really hard to do in practice.

Carolyn Evans: I think I have three points. First of all, as a former trade economist, I don't know if anybody has ever heard of the Trade Adjustment Assistance program. This is a program where if a group of workers can show that their jobs were displaced by trade or even by investment, by various reasons, they could enter a program where they essentially receive a stipend to go through training. If you look at the results of that, it's not great, in the sense that not everyone got a new job. If they did get a new job, it was probably at a lower wage. From that perspective, a little bit of pessimism.

Second point, though, the private sector has actually generated a lot of sort of data science boot camp type places, which actually have been pretty successful. In some cases, there's so much demand for those skills that employers will sort of pay for people to go through them. I don't know of any sort of other phenomenon like that historically. I also can't give you statistics on sort of how many people they actually go through them, but to me, that's sort of an optimistic thing that for a certain group of people who are displaced, that they would have the ability to get retrained in a reasonable amount of time, because some of these boot camps are just six months, something like that.

Thirdly, and this is kind of a...I don't know. Has anyone heard of Amazon Mechanical Turk? Anybody heard of that? This is essentially a marketplace where when we talk about ML, it's not the sexy part of it, but you have to have a big data set. If you're doing supervised machine learning, you need a whole big data set of what we call label data. If you want to identify a cat picture, you've got to have a whole bunch of pictures of cats and dogs, and somebody has to tell you is it a cat or is it a dog so that you can use that to train your model.

Places like Amazon Mechanical Turk are actually they sort of people go, it's basically a marketplace where people could say, "Oh, I'll do this job," and they'll sort of label pictures cats or dogs. That's one example of other types of jobs that we haven't thought about before that could be created. I don't think it requires a really high level of skill. Again, I don't know how big that will be, but it's just something to keep in mind, in terms of how we're thinking about the effects on labor markets.

Charles Evans: I think examples like that are really important to sort of understand what we're talking about, because I'd like to be open-minded about this and be more optimistic. I tend to think that somebody who's displaced from their current job is probably looking at a lower-paying opportunity after that. Otherwise, as economists, we might kind of go, "Why were we paying that job anyway to begin with?" I think that ends up being the case.

I'm reminded I'm old enough that when President Reagan was either campaigning in Pittsburgh or as a new president in his first year, and the steel mills were being closed, and somebody got a former steel worker a training job with Radio Shack, and it was billed as a big success. Then about six weeks later, the steel mill opened up and the guy left Radio Shack and went to the steel mill, because it pays a lot more. In fact, I know a guy who I went to college with who got one of these football scholarships in western Pennsylvania, and he played football, and he knew that his future was getting into the undergraduate business school and whatnot, and yet the steel mills opened up at about the same time that he went back. I always wonder what happened to that fellow, because it couldn't have lasted very long.

I think we need more examples of what the possibilities are. I think that people have a strong desire to live in the community where they've been for a long time. People really want jobs located where they are. Most of the alternatives, if you're not going to be a steel worker in Pittsburgh, you're going to be somewhere else, and moving is a really hard thing, even though we have a pretty mobile society here, it's not nearly mobile enough.

Reinhart: I feel an obligation to represent the range of questions, and just turning quickly to finance and a very specific one for you, Rob.

Kaplan: Yeah, I'm ready.

Reinhart: Oh, you're good? Go for it. Can you explain why corporate profits are so high in an environment where pricing power is purportedly…?

Kaplan: Yeah, so this is interesting. It will not surprise you. I spent my life looking at industries' and companies' profitability. I look at a number of different things. First of all, gross margin and then bottom line profit margin. I think that's where you see a striking divergence, unlike...you might want to go deeper on this. After-tax profits and earnings per share are very high, but what you're seeing to get there is lots of merger activity, lots of efforts to get more scale, and this is what's driving that. Then on their earnings per share point of view, share repurchase, obviously, and other financial transactions.

If you look at gross margin, and if you could actually segment that out, I think you'd see a different picture. Then again, you have to look by industry, and you'd see a surprising erosion, or at least a real challenge just to keep gross margins stable in a number of industries. Also, if you look then at prices by industry and trends, for example, new car prices, used car prices. Take a look at that.

Car industry is one I like to use, because everybody understands that industry, because everybody goes and buys their car. You'll see new car prices have been under a lot of pressure. Used car prices have been strong. I think if you go industry by industry, you can see this lack of pricing power and pressure on gross margins, although companies disclose it differently. It may not mean that they can't find a way to make greater bottom line, but...this is the job of the CEO. This is why you're seeing so much transactions and other activities, financial engineering, some would say, in order to generate that bottom line.

Reinhart: Anybody else? Okay. I do want to ask one question about monetary policy at the end, because I want to also give everybody a minute or so to hammer home the points you want the audience to leave with. It's in 1980, Robert Lucas wrote, "Our task, as I see it…is to write a FORTRAN program that will accept specific economic policy rules as ‘inputs' and generate ‘output' statistics describing the operating characteristics of the time series we care about."

Now Alpha Go might not have been written in FORTRAN, but the question is, are you actively planning to use machine learning to learn about the state of the economy? Why and why not, given everything you've heard thus far? The outsider, but former alumni, should be able to weigh in there. Why don't we go in this order.

Charles Evans: I thought this morning's session was extremely helpful for me, because I'm not the finest connoisseur of exactly what machine learning encompasses and what it doesn't. As I hear more about it, data reduction is one important aspect of that. We have many efforts that try to synthesize a fairly large amount of data available to us into a smaller number of factors that are important and use that to predict. The Atlanta Fed has been doing that with their Wage Tracker and other things. I find those to be useful, so that's good. In the camp of trying to construct a model that helps us think about alternative monetary policies, I think that's really quite useful. I've done some research on that, and the Board has done a tremendous amount. Those are good things to do.

I think that Bob Lucas, when he was writing in 1980, it was after the rules versus discretion Lucas critique and all of that. I think understanding when you are pursuing achievable monetary policy goals as opposed to things that are unachievable is a really important aspect of hitting our inflation objectives. Learning more about that I think is very, very important. There's some structure to it, but that's really quite a lot.

Carolyn Evans: I would say, sort of echoing what I said earlier, they already are, to the sense that they're using regressing and econometrics to do things. One thing I would add, though, that I think is really interesting that could be done differently and perhaps enable machine learning a little bit more in the past is thinking about some of the new sources of data. If you think about some of the job [sites], Glassdoor or all these different places that have data, which is in a much larger volume than traditional data, that enables new techniques, right? The problem with traditional time-series econometric data is that the frequency is very low. That's just the nature of it, which means that we're limited, in terms of the techniques that we can use, to the extent that you can leverage these other data coming from other sources, there's other issues with those selection issues, things like that. If we can address some of those and try to incorporate those in how we understand the economy, I think that could be really interesting.

Reinhart: That goes back to your point. We don't have that many business cycles.

Charles Evans: We only have so much, but yes. That's right, that's right.

Carolyn Evans: Cross-section versus time series.

Charles Evans: It is. That's right.

Reinhart: But it is the case that people looking at the Phillips curve or people looking at fiscal impetus have benefited from looking at the uneven incidents across states.

Charles Evans: Yes. Oh, absolutely. No, the state Phillips curve has been quite informative. That's true.

Reinhart: Rob?

Kaplan: I don't have anything to add. I agree with the points that were made.

Reinhart: With regard to the issue of what the Federal Reserve should be doing, should you be collecting more data? Where is big data in the scheme of things, recognizing that among other things, the Federal Reserve collects an enormous amount of data, both in terms of balance sheets of institutions, but deposit rates, lending rates, attitudes toward lending. Are you where you think the institution should be, and where do you think the institution will be in five years?

Charles Evans: I think data is very important. I think when the economy is changing as much as we think it is, or even if there's a lot of promise and we find out that it's really not changing that much, that requires a lot of effort. That requires data. That requires looking at different sources. I think it requires a lot of artful interpretation of this. You might be able to just accumulate larger and newer data sets and somehow reduce it down to something and find that it's helpful. But then you're forecasting and you're kind of basically saying, I'm looking at new relationships. I think we're going to have to see how the economy plays out and if the economy is evolving in ways that we don't understand, well, then we probably are going to be experiencing stronger inflation at a time when you don't think policy is set for that, or lower inflation. That's going to be a big part of the cues for that.

Reinhart: Anyone else?

Kaplan: I'll just comment. As an outsider to the Fed, I've only been a Fed [president], what, two and a half years, and it was set up originally in 1913, ratified 1935. I will say as an outsider that 12 Banks, the Regional Banks, and the fact that we, in each of our districts, can really dig in, meet local businesspeople, have lots of conversations, do local surveys, I think the work we're doing at the regional Feds in a period where the economy is changing, I think is enormously valuable. I learn an enormous amount from my colleagues around the table, and we try to add the same insight, but you can't get it sitting on a coast. You have to be sitting in a district. I think that's one of the great features of the Fed that gives me a lot of hope that we'll be able to meet these challenges.

Reinhart: With regard to the 1913 founding of the Fed, the reason we have industrial production going back to 1915 was in part that data collection was some of the instructions given to it by the Congress. All right. We've got only a little bit of time left. What I'd like to do is start from this end to that and say you've got one minute. What do you want this group to leave with?

Kaplan: For me, two points. One, people say technology-enabled disruption or these type of changes have been with us for decades. I think this is actually is different. It's a structural change that's different. I think my only job, or our main job, my job is to be open-minded, to be open to learning, not be rigid, not be predetermined, and to keep learning, because I think this is a structural change that we don't understand. Maybe we won't ever quite fully understand it, but our job in conferences like this is enormously helpful is just to keep learning about it and trying to understand it.

Reinhart: With all due respect, I'm supposed to lean in, just from the family perspective, and say you're never allowed to say this time is different. Carolyn?

Carolyn Evans: I guess in the spirit of Silicon Valley, optimism about the future, though I think that there are challenges as well, and I think this sort of workplace disruption, retraining, that's something that sort of remains to be seen how things evolve.

Charles Evans: One thing I thought that I didn't get around to expressing that's really quite different, but I think the business side of what we're talking about is going to play out in a perfectly fine fashion. I think that people pursuing the best interest of their firms, maximizing profits, and they're going to use the new technologies, and they're going to develop them, and they're going to compete. That will be one track, but the part that I don't understand and I think we need more analysis of is, who are the end users who are going to be paying for all of these? What are the end products that businesses want to produce that somebody is going to pay a good amount for where there is going to be a noncommoditization, a price margin? What is that?

When we talk about sort of reducing existing products down to something more like a commodity, there's an end point to that, I think, but something new where you make a lot of money, and who's going to pay for it? Part of this is I go back to the efficiency wage literature. I think it was Larry Summers who wrote a working paper, I'm not sure I read it, but the title was "Did Henry Ford Pay Efficiency Wages?" Was it the fact that cars were such an expensive proposition that nobody could really afford them at that time when they rolled out? Did he pay their workers more so that they could somehow buy these things? Who is going to end up buying these great, gee-wiz products that are produced? Is it going to be the people who are doing the machine learning language, or is it going to be the people who need to be retrained. There's not as much aggregate demand coming from that segment I wonder about. That's a different economy, and we're going to have to understand it better.

Reinhart: What I'd like to do is express my gratitude to the panel for being so clear.

Charles Evans: Thank you. Thanks, Vince.

Carolyn Evans: Thank you.

Kaplan: Thanks.

Reinhart: To the audience, for asking so many good questions, and I do apologize if I didn't cover nearly half the questions. That's because we were trying to make it a coherent discussion. I'd like to thank the organizers, once again. We return 15 seconds to the schedule.