Evaluation Guru 2: Country Program Evaluations

Transcript

Animation:

Hello and welcome to Evaluation Guru. Through this series we will share with you how to design, implement and manage evaluations which in turn will help you improve the development effectiveness of your projects, programs and strategies, as well as foster accountability and learning.

Introduction:

Joanne Asquith
Director
Thematic and Country Division
Independent Evaluation Department
Asian Development Bank

Welcome to another episode of Evaluation Guru. Today we’re going to talk about Country Program Evaluations. This is where we look at the development effectiveness and the results of ADB’s projects and other interventions in-country. It’s not about the performance of the government. It’s not about the performance of the country. It’s about how ADB performed. What resources did it use? How did it influence other development partners, worked with other development partners? How did it work with civil society and through the government and other groups, to deliver development results and improve people’s lives.

Country Context:

So how do we do Country Program Evaluations? A really good place to start is the country context. It’s really important that evaluators understand the country context. The Pacific for instance, it’s important to note that these are small countries with limited economic bases. In Bangladesh, it’s important to understand that this is a country that faces the risk of natural hazards. In Sri Lanka, it’s important to understand that this is a country that has a history of conflict. So it's important that we understand the context in which ADB is working, and the characteristics of those countries, and their specific development needs.

Country Partnership Strategy:

So once we have an understanding of the country context, the key important document in all of this is ADB’s Country Partnership Strategy. And that document sets out ADB’s key development objectives. It sets out what results ADB wants to achieve with its interventions.

So far we’ve looked at country context, and we’ve looked at ADB’s Country Partnership Strategy. And in all of this, what we’re trying to get to is how relevant is ADB’s support to the country? How effective has ADB been in achieving results? Not just at project level but also at country level. We're also interested in efficiency. Were resources used to get maximum benefits, maximum development outcomes? We’re interested in the sustainability of ADB’s projects because we want the benefits to be sustained well into the future. And we’re interested in what impact ADB had on people’s lives, on the lives of poor people, on the lives of women and girls. What impacts did ADB have on climate change, and so on. So we used the evaluation criteria as a framework against which we’re going to be able to make a judgement about ADB’s overall performance. Was it successful or not?

Sector Assessments:

So the main practice that we undertake in Country Program Evaluations is to undertake sector assessments. Sector assessments are really the meat of our Country Program Evaluation. And we look at the relevance, effectiveness, efficiency, sustainability, and the impact of ADB’s projects at the sector level. But we don’t stop there. It’s not just a sector assessment. That’s kinda like a vertical assessment. We also then slice it horizontally by looking at how effective ADB was at achieving thematic level results.

So for instance if we’re looking at ADB’s performance in the energy sector, ADB may have increased the supply of electricity but did it reach the poorest? Was it done in a climate friendly way? Did the increase in electricity supply benefit women and improved their lives?

Importance of Communicating Evaluation Results

When we're producing a Country Program Evaluation it’s not enough just to deliver a report and there it is. And we hope that people read it and we hope that it has influence. No, it’s not enough. We have to feedback what we learn, what we find, the judgements that we make. We have to feed that back to ADB and also the government, and other partners, and civil society. We want people to now what worked and what didn’t work. It’s also very important because ADB uses the knowledge that we generate from an evaluation to inform its next Country Partnership Strategy. So the idea is that through having this knowledge that we improve future development effectiveness.

Summary:

So in summarizing everything we’ve said so far, it’s very important that we do Country Program Evaluations so that we can inform future ADB strategies and improve development results. We assess the country context and how ADB works within that context, works with other development partners, works with civil society and the private sector to achieve development results. We look at how ADB works with government in the sectors to improve service delivery. We then look across the sectors at thematic objectives like helping to improve people’s lives.

And in doing that, what we end up with is an overall rating of was the program successful or not. And if it was successful, why is it successful? What worked? What didn’t work? And if it wasn't successful, then what does ADB need to do differently in the future. And that’s the role of an evaluator. Thank you very much for watching and look out for the next evaluation Guru.

Animation:

Thank you for watching Evaluation Guru. We will be back with another episode on how to design, implement and manage effective evaluations. Till then, don’t forget to look up our social media accounts. Bye for now.