Driving Revolutionary Ideas into Practice | Development Impact Week 2023

00:00 Opening remarks
- Indermit Gill, Chief Economist of the World Bank Group and Senior Vice President for Development Economics

02:01 Overview of the flagship report: Driving Revolutionary Ideas into Practice
- Arianna Legovini, Head of Development Impact Evaluation (DIME) department at the World Bank Group

19:37 Maximizing the Impact of Development Finance. A conversation with Esther Duflo, Abdul Latif Jameel Professor of Poverty Alleviation and Development Economics, Massachusetts Institute of Technology

46:04 Live Q&A

51:24 Closing remarks

[Indermit Gill]

Revolutionary ideas into practice. Actually, it's called Driving Revolutionary Ideas into Practice, but it sounded like somebody driving a nail through my skull. So, I thought putting it as revolutionary ideas was a softer way to put it. My name is Indermit Gill, and I work for Arianna Legovini here at the World Bank. She'll say that she works for me, but we both know who works for whom. She told me to keep my intervention very brief. And so, I will. My job is just to introduce our two main speakers and then I'm going to let them speak and then ask them questions on your behalf. And then of course you should be asking questions too. So, we're very lucky to have Professor Esther Duflo of the Massachusetts Institute of Technology to join us for the session. She needs no introduction, especially to this crowd. I think that she should be introducing me, not me introducing her. I'm not going to give her one, except to say that she got the John Bates medal for the most promising economist under 40, and the Nobel Prize in the same decade. I think she's the youngest winner of the Nobel Prize. So that I'm sure is the first also to actually get both of those things. It'll be really good to hear from Esther what she's been working on these days. Because we would really like to know what the cutting edge is doing right now. perhaps a bit selfishly then we would also like to know from Esther what we think we, at the World Bank, ought to be working on. So, before we turn to Esther though, I will turn the mic to Ariana, who also needs no introduction. She's going to take about 15 minutes to actually start the conversation up. Do you want to come here? It would be better, I think everybody can see you then.

[Arianna Legovini]

Yeah, thank you. As Indermit said, I'm Arianna Legovini, I’m the Director of Development Impact. We founded this group almost 20 years ago, and we are today launching a new report. It's called “Driving” because it spells DRIP and we have a history of having DRIP reports that get a lot of downloads.

[Indermit Gill]

I get it. I get it now, yeah.

[Arianna Legovini]

Okay, I think we are ready for my presentation to be on. Thank you very much. All right, so this is a report that encapsulates our work for almost two decades. The idea of this report is not only to talk about the issues at hand, but to talk about the underlying technology that allows us to do much better at what we do in development. So, we are going to cover infrastructure for climate change, we're going to cover reducing poverty, and of course we have Esther here so she can tell us more about how much progress we have made in this area in the last 20 years. We have human development on mobile, and disrupting government processes and efficiencies and effectiveness. But really, we are using these chapters to explain how we have worked with operational colleagues and implementing agencies and donor agencies all over the world to change the way we do development. Not anymore in a pre-set deterministic way, but embracing change, embracing innovations, embracing debate, really the dialectics of development, not deciding on day one what we are going to be doing, but actually ingraining innovations in everything that we do. And we think this is the disruptive technology of today, the technology that is cheaper and more effective, and that will replace the way that we do business in development if we can push the inertia forward and get decisions made today for the good of the people that we are trying to serve. Okay, so what is this idea? 

The idea is fairly simple. The idea, we like to call it “trial and adopt,” to communicate that we are here to try out different ideas but also provide real time information to people taking decisions to adopt the better ways of doing things. So we are not going to decide to do a project that does this and turns left, but we are going to implement the project that tests out this and that and then decides whether to turn left or right depending on the evidence, the empirical evidence that gives us the good guidance on how to improve development impact of everything that we do.

So here is my classic example, and it's an example of a project that two years into implementation was not able to meet its targets for the number of trees to be planted. They came to us, they said, “How can we do better?” And we decided with them to test financial incentives, and those financial incentives increased three plantings by three times. The result was incorporated in the project. Later we worked even further with them to optimize those incentives and increase with the same money, the geographical coverage of this project by a factor of two, improving effectiveness by another 70%. So, this is the idea. The idea is that we are going to work with project teams and implementing agencies over time to get it right, not to wait for failure but to secure success. This is what we call the big idea of this report. It is not electric helicopters as somebody said on Twitter, but it's much more than that, I'm sorry, which is that innovation is a critical driver to development. But that innovation is not just there for us to pick, but it is really endogenous to the process of doing development to the gathering of people around the table, discussing what are the problems that we're facing and how we can solve them. Not have solutions being parachuted from outside, but actually a process of thinking and disagreeing. I love disagreement, I love disagreement because it engenders kind of ideas and new ideas and we can test them in the field and figure out what works, empirically and in practice. So, what does this lead to, it leads to solutions that are context specific, that are owned by the people who are implementing these trials and are acted upon to solve the problems that they face. And through a process that we have tested for the last 20 years of co-production with government agencies and others in the field, we can do a few important things. 

One is secure, very good questions for the research that we do. Questions that people on the ground want to have answers to. We can also take the time to increase their capacities for data, data systems, trialing ideas in the field. And third and very importantly, secure adoption of this idea secure actions, policy action. Not just by the government agencies, but also through a process of local diffusion, create the space for donors and others to partake into the knowledge and act upon the knowledge. In a recent interview, Ajay Banga says that we need to take risks that the only trick is to avoid past mistakes. I know Indermit is a big believer of that…

[Indermit Gill]

Same mistake twice.

[Arianna Legovini]

…Not to make the same mistakes twice, yes. Once is okay but twice is not. But how are we to know if we don't measure the impact of what we do, and act upon that knowledge? And so, I think through this process of trialing and evaluating the impact of our interventions, we actually generate a lot of scientific knowledge that can be used by all. So obviously at the operational level we both measure the impact of an investment, but also learn on how to get more out of that investment. Number one. Then kind of trialing across context and across different type of investments, we also learn how to optimize the allocation of our resources in country portfolios and in sector portfolios. Now that part is actually harder to get to, and so this is part of what we need to decide and push. How do we get all the evidence that we have generated, all this stock knowledge that that has been produced in these last years embedded into development programs? 

So how big of an impact can we expect? How big of an impact? So, the whole book is devoted to telling us how big of an impact we can expect from embedding data and trials into the development process. And so, we go through four areas. I want you to imagine a computer with a DOS prompt. Do you remember those, Indermit? Computer with a DOS prompt? Well, of course, it's good hardware, but the DOS prompt didn't help us that much. Particularly now that we don't remember any of those commands. We really like a computer with a good software. We like Windows and even ChatGPT and so forth and so on. We think that will be much more effective. Well, the same thing happens with infrastructure. Here this chapter is devoted specifically to water infrastructure, drains, taps and canals. And the question is, how much does the hard way gets us? And it gets us only halfway. That's the reality of it. As we trial different ways of improving the returns to infrastructure, we find that we can double, even triple the returns to infrastructure by providing information to farmers to use adequate amounts of water to increase productivity and cut water shortages. To develop labor saving technologies for the adoption of irrigation technology that could triple take up of irrigation technology, or pay attention to the fact the 60% of those taps are non-functional for 12 months or more. And that no matter how much we invest in more water points, we still have flat access to water. We can do a lot, and very quickly, to increase functionality and actually transform access to water. 

We move to poverty reduction. We’ve worked on this at least since 1998, with the first large trial conditional cash transfers in Mexico. Since then, we have built a huge literature from academia, from the Bank in understanding what programs, what packages, what ways of delivering actually reduce poverty by a large margin and in a sustainable fashion. We have a lot more work to do to move the resources from the programs that don't work, and move them into, and scaling up the programs that do. We have a lot more work to do.

Let me take you now to an area where we are just scratching the surface, which is the potential of mobile technologies to address the human development crisis. And here I'm going to take one example, but on Thursday we're going to have many more, which is that the world has changed really fast and we are still, and because of COVID-19 but even before COVID-19, seeing systems that are not delivering on basic things. We can put children in school. But in this context, in grade five, only a few children can read just a paragraph. After five years in school, a 5000-dollar price tag for each child. Give children, phones and in a few months, they have learned to read not only them but their siblings as well. Price tag: 50 dollars. This is an area… 

Foreign interpreter.

[speaking in foreign language] 

…Almost everything. So, we need to kind of think about how we scale these technologies in a way that actually contribute to the welfare of the many. 

I'm going to, lastly, cover... We're going to cover this on Friday, another area that has huge potentials. We're doing foundational research in this area, but the scaling of this area could actually transform the way we address the debt crisis, we address fiscal shortcomings. This is the data, this is the area of government analytics using a large, huge, massive amount of data generated by mobile, digital, remote sensing satellite data to understand what the problems are. For example, by identifying a 1% of the road network, that is the responsible for 50% of the deaths all the way to understanding what goes wrong in the process of justice. Where are the inefficiencies? Why things are not being delivered the way they should? The potential for increasing efficiencies in government by 20%, 30%, 40%, 50%, sometimes 60% is what we find working with administrative data, combining with other sources of data to change the way things are done, tasks are assigned, all the way to making government way more efficient and effective. 

So, to recap, innovation, I think of innovation as this process of discovery, which is endogenous to the way we do things, it doesn't come from the outside. It's a process of figuring out what works in practice. And by trialing ideas we can get much better, much faster. So, what are the issues? The issue is that if we were to incorporate just what we have learned in the last 20 years systematically across our portfolios, we could increase our impact by a factor of two. There is so much more that we need to learn, specifically in areas such as climate where we still have a very, very low knowledge base of how to move forward fast and effectively. What we need to overcome, the institutional inertia, we actually need to take decisions and take them now to change the way development is done in a way that incorporates scientific knowledge in a very dynamic way. 

This is our proposal, an action plan for the Bank and its partners. Number one is to use trials systematically across all investments. I'll say, “Ooh, how do we do that?” And the idea here is actually there are things that are already happening and we can jump on that one and change it somewhat but really align to what the institutions are already doing, doing it just slightly differently. The first issue is to move from single investments to multiphase, multi-country adaptive programs. This allows us to do a few things. One is to reduce the cost of design and preparation, move it to integrating early trials in early entry countries, and then use this kind of cross-country knowledge generation to learn much more, much faster about how to make things work and incorporate them... Incorporate learning in the next phase of programming. These programs are already doing these, cross-country workshops at the six or 12 months’ distance that are used to learn a lot together how to move the programs forward. And we'll have Holger [Kray], tomorrow in fact, talking about this program as part of our first session. 

The second, the Bank uses a country engagement program, but we can make that country engagement model even stronger by overlaying it with co-production, a co-production that strengthens local capacities for the generation and use of data, and trials, and helps us coordinate the evidence to action framework in-country that allows all of our partners, development partners, academics and governments to move together in a different way. And of course, we need to finance knowledge, and finance it in the same way we are financing investments, not as an afterthought, not as a small maybe, but actually make sure that knowledge is integrated in everything that we do. Thank you very much. And looking forward to Esther.

[Indermit Gill]

Okay, so does anybody have any questions for Arianna here before we turn to Esther? Anybody? No? Okay. So again, it's all very scripted because Arianna's given me all the questions I'm supposed to ask, but along the way I'm going actually disobey you and I'm going to ask my own question just so you know. So, I can't see Esther. There's Esther. Okay, Esther, so to actually prepare for this, Esther, I actually read your Nobel lecture, which was really nicely written. So those of you who haven't read Esther's Nobel lecture its worth reading. Just so you guys know. So, I will ask one question that Ariana has asked me to ask, and then the other one I'll ask is what I wanted to ask you. So, her first one is, how has experimentation shaped your approach to development economics? And why do you believe that it's actually really critical to incorporate that into the way that we finance projects?

[Esther Duflo]

First of all, thank you very much for having me here. I'm going to take this opportunity to acknowledge the role that the Bank has played in changing the whole ecosystem of development, and in particular in increasing the acceptability of the use of experimentation and learning in developing country governments. And also increase the capacity to do it. The Bank was very early in the game with Arianna at the lead of this effort. Let me thank the Bank as an institution for having been there and playing a very important role. And secondly, let me congratulate Arianna personally. I got a text saying that my volume is low.

[Indermit Gill]

No, it's okay now it's okay now, it's good now.

[Esther Duflo]

I can do something about it, like putting a headphone.

[Indermit Gill]

It's good now.

[Arianna Legovini]

Now we can hear you fine. [Crosstalk]

[Indermit Gill]

Yes.

[Esther Duflo]

I just need to be closer to my computer. And then... I really mean that... This is very important. Now let me answer the question. So, there are actually many things of course I learned in the practice of, now about 20 years, more than 20 years doing this work. J-PAL is done 20 years this year, so it's been a while. Perhaps one of the key findings was exemplified in Arianna's slides about the water, and about the fact that once you have all of the infrastructure of the water, you only have done half the work and you can improve the effectiveness of this investment and the cost-effectiveness by keeping a lot of... By putting software on top of it. And this joins something that... I have called plumbing and it's nice because it's a plumbing example for a plumbing issue, which is that a lot of, in the way that, in economics department, when we do our PhD is and often the way we think in our office we think about problems of development, we think of them at a very conceptual level. That we have the ideas... The general ideas of a program like I don't know, incentives matter or maybe they don't matter, information is important, or local communities and so on. And so that's one big way. And then another way is none of that matters. All that matters is the political economy, and the constraints that are given by the political economy of governments and so on.

One fundamental thing that I've learned in many, many years of doing this work, of working with government to improve the policies that they implement towards their citizens, in particular poor citizens, is that what matters enormously on top of these two things is the details of implementation. And in particular, the details of implementation that nobody could really think about in advance, especially initially, on new programs or new ideas. For example, to take another tap example, I worked with Veolia, and with the city of Tangier had a great program to improve access to drinking water at home. Pascaline Dupas and I evaluated... We were contacted by Veolia because even though they had done all of the engineering work and they also had loans for people, and the government was totally behind, nobody was taking up this program. They were very curious why, and they thought maybe there was something deep about property rights or something like that. We went and started by asking questions to people and we realized that it's the application process was cumbersome, and in particular I had to go several times to the city to get access, to give your documents, etcetera and apply for the programs. We suggested trying with allowing people to apply at home, by sending a fund officer to the house, taking photos and to do the entire application at home. They agreed to that, and the take-up point went from zero to 70%. I'm not mentioning this example, to say “oh this is how clever we are to think about this.” I'm mentioning this example to say this is really somewhere where I think that the intentions were good, most of the work had been done, the engineering was sound, the idea of a loan was sound, but there is always something that doesn't exactly work the way you plan in the actual execution. And in this instance, it was that. Working with government, so this is where Arianna's word of “co-creation” is essential because it's through co-creation that you are learning is what are the questions that your partners are actually asking? What is actually relevant to them at this very moment? And where are they stumbling in achieving the goals that they are setting out for themselves? And I think one of the tensions coming from us coming from Washington or from Boston is that you don't want to replace, or substitute, the democratic process of what's important and what they are thinking about. 

On the other hand, there is so much that people are genuinely confused and uncertain about what's the best way to get at it, that in this space there are enormous demand for experimentation because we are totally in this “I don't know how to best do the thing,” and therefore we can start working together effectively with a variety of partners. So, this can be NGOs, this can be government, this can be private firms, or in this instance, for example, it was a government, a private firm and us working together. And the critical importance of this implementation details, and the back and forth there is between trying something out as in your Ajay Banga quote, sometimes it fails, sometimes it succeeds. But if it fails it's fine. You'll try something else. If it succeeds, you have a lesson that can be shared. And this is how I think one can make so much progress in very diverse environments, very good, very functional environments where there are still a lot of things we don't know exactly how to do. And in fact, even dysfunctional, very difficult environments where there is sometimes still scope for action.

[Indermit Gill]

So, I was going to ask you a question that occurred to me when I was reading your lecture, and it had to do with... As I was reading a lecture, two or three things struck me. One was, I think it was the only lecture where you actually formally give credit to a lot of people who actually helped you with your work, who helped you with your insights and so on, and that was very remarkable. And I think it's worth reading the lecture just for that, because I think it's really good practice to make sure that all the people who have influenced your ideas and whose ideas you influenced who actually made for the success of this movement, that Esther and Abhijit have headed that you actually acknowledge them formally. But I had a question based on the work that you did on microcredit, based on the work that you did on teaching at the right level and things like that. What's the one thing that you feel works very well, but is not widely appreciated? And then the second one is the converse of that, what actually you find doesn't work well at all but continues to be common practice?

[Esther Duflo]

That's a great question. So first of all, I think it's a great question for different reasons, and one of them is to give me the opportunity to highlight how there is not a very direct link between one nice experiment, one shiny policy brief, and boom, policy action follows. I'm giving a very naive way of it, but I think it covers maybe my way of thinking about this when I was just starting. You would go naturally in these steps, and it's not like that. For any number of reasons. First of all because there are legitimate reasons why it's not like that, because you need to accumulate experiments, make sure that your first one is not a fluke. Go from context to context to have more of a sense of what's actually an idea that seems to come back over and over again. And the second reason is less legitimate, it’s that there is a ton of inertia and ideology. We call that three Is, ideology, ignorance, inertia. A lot of programs are conceived in ideology, in ignorance of the reality of the field, but once they're there they have a very enormous amount of staying power. In order to fight that you need to be there for the long game, and that's the beauty of DIME. And I would say that's also the beauty of J-PAL, that we just take our time and we have a very long-run partnership with governments. 

So that was a detour, but let me answer your question with one example of an idea that has an enormous amount of staying power despite evidence to the contrary. And my favorite pet peeve at the moment is the idea that giving money or resources to the poor makes them lazy. That's a good one, because that's actually one that is very widely shared in developing countries, in developed countries. The president of France is one of the biggest proponents of that idea, and a lot of French policy is a direct consequence of that idea. He strongly believes in it. I think he's genuine in his belief of it. But it is striking how much that idea has been disproven over the last many years, actually from the first experiments ever done, which was the Negative Income Tax experiment which showed very little elasticity of people's labor supply to a transfer even though the transfer was taxed away. To the many, many experiments on conditional cash transfers that Arianna was referring to, that have showed no labor supply effect, to more recent work that shows that on the contrary, if you take the ultra-poor, once you give them a bit of security either via an asset or via a social program, it actually tends to increase their labor supply and their productivity while working. So, this is something that so many social systems around the world, rich and poor countries are predicated on this one idea which happens to be counterfactual. It happens to not correspond to the facts. So that would be the example I would take. I don't know, maybe it fits both your questions, it answers both.

[Indermit Gill]

Yes, yes. Very efficient. Very, very efficient. No, I was going to ask… So, I think that what we should do is you all should prepare your questions, because what we'll start to do now is I'll go to Arianna if she has a question and then to one of you if you have a question. So, if you want to come stand at that mic, otherwise... You shouldn't be shy; you should come up. Yeah? All right.

But the thing that we are really worried about these days is climate change, Esther. I know that you worked on things like microcredit, you worked on health, you worked also on issues related to poverty relief as well as teaching. But how can insights from the work that you've done so far, how can they inform the design of effective climate change policies? That's what we'd like to know.

[Esther Duflo]

So, I'm very delighted to see the Bank be interested in climate change, and I'm hoping that the same leadership role that you've played in bringing evidence in the fight against poverty, you can now play in bringing evidence in the fight, not only against climate change, but also in helping poor people cope with the consequence of climate change. So, mitigation, adaptation, and also just coping with the disasters as they occur. I think the world of climate change today is… I'm going to say something that's slightly controversial, so I apologize for that in advance. But I think the world of climate change today is a bit where the world of fight against poverty was before Arianna's good work and J-PAL's good work starting 20 years ago, which is kind of a little bit dominated by magical thinking, or at least a lot of people have stopped denying climate change. That's a good thing. But among the people who believe in climate change, there is still very much a hope that there will be a magical solution to get us out of it. And then the magical solution is often of a technological nature. For some it's like carbon capture or for some it's electric vehicles, and a bit like microcredit was supposed to solve us... To get us out of all of poverty problem at once until we realized that it's going to be part of the equation but not all of it. So, we are a bit in that discourse in climate change. And another thing where we also are a bit in that discourse in climate change, and that’s related is the idea that actually it's possible to do good while making money. We can entrust the private sector with this job and they'll make money and they'll be happy and we will all continue to live our comfortable life ever after, but in a cleaner way. This is related, this love for win-win solutions and magic bullets… I prefer to have this conversation than no conversation, but I think we now need to go to the third stage and have the real conversations, which is it's not going to be a magic bullet, it's going to be any number of silver pellets, a lot of them. Like it is for poverty, and now it's so much more accepted and people are so much more pragmatic that not one thing is going to solve poverty, not international trade, not growth, not getting rid of capitalism. Many, many, things are contributing and have contributed so much in improving the quality of life in the last 30 years. And I think with climate we need to go to the same place. There are several things to consider, particularly if we are concerned, like the Bank is in the welfare of the poorest people in the world. One is of course mitigation, but adaptation and energy access in this world and coping with... And social protection in coping with the problems that are go going to occur for sure, drought and floods and stuff like that. On these four pillars we need innovation. We know nothing, because we are, first, ignoring the problem and now we are trying to solve it with the magic bullet. So, there is so little we know. We don't know nothing, we just know very little compared to what we know in term of various aspects of the fight against poverty. There is so much evidence to accumulate on individual things. If I, for example, paid for ecoservices, ecosystem services, do they work? In what condition? How do people need to be informed, convinced to participate, that type of thing. So, there is some work, for example, great work [Unintelligible] much more needs to be done. In how to... In adaptation… So, this is a mitigation example. In adaptation is they are entire swathes of land in the South that are already severely degraded. How do we put them back in cultivation? So, there is work by Kelsey Jack looking at half-moons in Niger, how do we train people to use them? Does it work, does it not work? Etcetera. I could give you several examples. J-PAL has an initiative, the King Climate Action Initiative, that works on those, but we need to multiply them and we need to all be in the business of generating these examples, projects, one by one by one, slowly but surely. That's the only way to make real progress. So, the same attitude of bringing the innovation, don't make the same mistake twice. I love co-creation. All of that has to be brought to the energy, environment, climate change, fight, and it seems to be doing it all over again. So, your work is cut out for that transition.

[Indermit Gill]

Nobody has any questions. Okay, you have a question. Before you ask that question, I have a pushback on Esther. 

So, you said that we are in the same place now about climate change as we were on poverty 20 years back. So, I'm thinking back 20 years and I'm thinking at that time we had the World Development Report on Poverty 2000, 2001. Before that we had the 1990 World Development Report and so on. So, we actually acknowledged the problem of poverty. It's not similar to the climate change in the sense that we actually thought that it was a thing. We also actually spent quite a bit of time looking at the experiences of countries that had reduced poverty rather rapidly. The 1990 World Development Report actually came up with a very nice 2.5 formula, or 2.5-point strategy which was broadly based growth, investments in human capital and targeted social safety nets. So, in that sense actually, if one fast forwards to now and say when we find that investments in human capital falter, or when we find that broadly growth rates falter, then even really well targeted social safety nets and so on have not actually prevented poverty from actually increasing and so on. There are parallels again to climate change in the sense that we are looking for solutions, these magic bullets which you spoke about, when actually what really is required is for these countries to do all... To come up with that 2.5-point strategy analog for climate action rather than specific things like carbon taxes and so on. It has to be... I guess it has to be a strategy that involves, probably involves broadly based development to actually start with too. I don't know if... I guess what I'm trying to say to this group is that the work that we talk, that we are speaking about over here impact evaluations of interventions, those are not necessarily even the mainstay of an anti-poverty program. Do you believe that? I'm sure you've heard this argument before.

[Esther Duflo]

So first of all, maybe I was too generous to climate. I was kind of assuming that we are at the stage where a lot of people agree that there is a climate problem. So, in that sense I was saying we are 20 years, where we... For sure the World Bank agreed that there was a poverty problem 20 years ago. So that's why I'm thinking we are there in that situation. But I think that's where we were 20 years ago is we were still too focused, in term of the poverty, too focused on finding precisely that formula that was going to help us address the problem fully. What has happened since then is combination of things. Some have to do with us, some don’t have anything to do with us that have led to more pragmatism in many developing country governments as well as the people who advised them and fund them. So, there were the millennium development goals which helped move the goal from, “Oh you've not made any progress if you don't have economic growth” to, “Well here are 10 things that we agree are important.” One of them is poverty, but there is also education, there is also infant mortality and maternal mortality, and these are objective on their own right. And I think this greatly empowered countries to seize some issues that they were going to really go after. You have a country like Malawi for example, where over the last 20, 30 years there has been not that much economic growth, but [it] was able to reduce maternal mortality by a half and it's one... Or more than half, and it's one of the countries that has decreased maternal mortality the most proportionally. They didn't do that as spending a ton of new money, which they didn't have. They did that by using all of the accumulated experience from other countries in order to devise a strategy. I don't think... I don't know if there was a randomized evolution of anything they did, but what is clear is the combination of things produced results. Or another example is malaria, the fight against malaria. Until very recently we still didn't have any magic formula. There were no vaccines, now there might be one coming up. And yet the cases of malaria plummeted simply through muddling through, figuring out what's the right way to distribute large quantity of bed nets and making sure they were used. So, in a way, yes, I still believe this is a combination of interventions. Some of them, many of them have had a randomized evolution behind them, some of them didn't. But the key is not so much that. The key is the change in mindset between there is one goal and we need to find the one formula to achieve that goal, to this goal actually projectively gets broken up into thousands of questions that we can make real progress on. Because a well posed question will bring precise answers. And I think we need to do that in climate now and I'm giving credit for agreeing that there is a problem. Maybe we still need to do that. But I think at the World Bank, hopefully it's done at this point. And then it's okay, so now let's take the country we have. There is a number of problems related to climate change. It's not all to reduce emissions, in fact, not mostly to reduce emissions for the poor countries because they contribute nothing to the problem, essentially. It's about that, but also how do we effectively spend resources to ensure that climate change doesn't undo all of the gains that have been experienced in the fight against poverty, which is really my big worry if I look at the next 20, 30 years.

[Indermit Gill]

Okay, so we have three questions. Yes.

[Speaker 4]

I have a question from our online audience for Arianna. Milana asks, “It is important to know what works but also what is effective to implement and scale up. Where do cost efficiency and effectiveness analysis stand in the approach and the way forward presented today?”

[Indermit Gill]

Okay, you wait and then we take the other two questions too, please.

[Speaker 5]

Hello, my question is for Esther. First of all, thank you so much for joining us. Related to some of the topics that you touched upon about how a lot of policies implemented today have a substantive amount of literature proving that they are not the most effective, or in fact counter effective. Do you believe that resources should still be mostly devoted to innovation and more creative experimentation to find more effective solutions? Or should we be trying to mobilize more resources into trying to implement the proven solutions that we already have? Thank you.

[Indermit Gill]

Okay, one more question, and then I'll give it to you guys to answer.

[Victor Orozco]

Victor Orozco from the DIME team. Question for Esther and for Arianna as well. So, we're working with bureaucrats, right? We write long manuals about how this intervention is going to work. We don't want to take any risks. So how can we accelerate, bring this mindset of experimentation, of risk-taking, going beyond lip service in development? What is needed to accelerate and scale up this real experimentation revolution in bureaucracies? Thank you.

[Indermit Gill]

Are there any other questions? Because we will run out of time. We only have five more minutes left. Anybody else? Yes, one more. Yeah.

[Speaker 4]

Last one from Shahid. “Do you think that all developing countries have the required level of knowledge, capacity and infrastructure required to implement a trial and adopt method in their development programs? What about countries with political instability, corruption and bureaucratic inefficiency?” And this is for both Esther and Arianna. Thank you.

[Indermit Gill]

All right, you guys have two and a half minutes each. Esther, you go first.

[Esther Duflo]

Yes, thank you. So let me start with the last question. No, I don't think there is the enough capacity, and I think that one of the great challenges in the next many years is to create much more of this capacity. I think there is not enough capacity because our way of training these capacities were too limited and we need to find ways to train capacity in the developing countries themselves. The talent is there, I have no doubt about that. But we are extremely bad at picking, at identifying and giving the talent that is there in the developing countries the resources and the means to get at the problems, both the technical skills and then easy access to resources and so on. There is progress there, but I think is, for me, this is a huge project that I really want to undertake of contributing to the identification and development of talent in many countries in the South to take ownership of this important... Seize it from us. Honestly, this needs to be done by others at this point.

 

I did enjoy the questions about should we spend more time getting effective policies to be adopted. I actually think that the two work together in that often the best way to get policies to be adopted, in my experience, is to have a very deep partnership with an institution or government. And that starts by looking at what it is they want to do, and what are the questions that are very important for them, and helping them do better what they want to do. Often a lot of these plumbing questions. And then once you've done that, from there you can move to... So, you get already what they're working on, and make it to work better. From there you might be able to move to “Okay, but now did you consider these other things that many other countries have done and have done successfully?” I think the two often work together, the adoption of innovation. In my experience, the countries are so curious that the adoption of the innovating practice comes before the willingness to then do what other people do and have found to be effective. I think that answers that answers most questions. Maybe Victor Orozco, I haven't fully answered this question but now I forget what it was. Arianna will take care of it.

[Indermit Gill]

We'll actually come back for... We plan to have you come back for a last word. So, Arianna, go ahead.

[Arianna Legovini]

First, I want to recognize Esther, and I'm so happy that you're here with us today. She obviously led a huge movement that motivated everyone to partake in that incredible process. Back when I first opened up two positions for Impact Evaluation Specialists with experience in randomized control trials from the field, I got three applications. Now we get 3000 applications, with PhD students from all over the world working on this amazing agenda of building scientific knowledge bottom up. So, a huge transformation in development economics from theory to practice. We have been partaking of this amazing revolution. We want to do even more. I think we are still working with a relatively small part of development finance, but we want to use every single dollar of development finance for learning. I think this is the idea of development finance, if we use it for learning, then we can influence and inform all the government programs and programs of NGOs and all development partners. This is for me, Esther, a call for us to work even more together and develop those capacities in countries through the co-production with local academics and local governments and kind of really pushing it to a different level. 

One of the questions was regarding how we go from small trials all the way to large government systems. I think in poverty we're doing that, right? You started off with the trials at relatively smaller scale and then bigger and bigger. And then we took it to the governments and kind of continue experimenting with that process. There is no actual end to learning more and more how to get the right packages, the most cost-effective that can be brought to the largest number of people. This is a very exciting process.

On the climate issues, I would say that this work done on poverty will be of extreme support to the climate transition. As we all know, climate change is affecting... Has huge distribution of consequences, as you mentioned Esther. But now we are better skilled at thinking about what to do on the just transition. So that gives us information about the energy transition, and how to get it right. But at least on the social part, we have a lot of evidence that we can build on. This is really, really exciting. I'm very, very glad to have you here.

[Indermit Gill]

Last word to Esther, and then thank you to both these remarkable ladies. Yeah, go ahead Esther.

[Esther Duflo]

Thank you again. So, I'll use my last word to amplify what Arianna said. I think that there are places in the Bank, in the USAID, in the French Bilateral in DFID, where this idea of innovation is taken seriously and there are open windows for people to apply to this project. But this remains so small as a fraction of the total amount of money that is spent by these institutions, even among institutions that have been quite pioneering in this fight. 

If I think about what development assistance should be about, literally I agree with Arianna, all of it should be support for innovation. So that seems a bit wild as a dream to think that the entire budget of the World Bank should have that in mind, and should be structured to the idea that actually why are we there? We are there to do that. But I think it's an important idea, because that is really what is relevant for us. That's the only way to keep development assistance, development financing, to stay pertinent in years to come. As countries become richer and their other actors and so on, is to support innovation for countries to arrive at what are their objectives anyways.

[Indermit Gill]

Very good. So listen, as I was reading your lecture, I saw that you had a very close relationship and partnership with the Indian organization Pratham and with Rukmini Banerji, and that you acknowledged her in your Nobel lecture. I hope that when you win your second Nobel Prize, like Marie Curie, you will actually mention Arianna and the World Bank in the same way. So we have a very similar partnership, and a very effective partnership, Esther, yeah? Thank you very, very much for everything.