TheWorld Bank J u n e PREMnotes 2 0 0 4 n u m b e r 8 7 Public Sector Assessing country readiness for results-based monitoring and evaluation systems Results-based monitoring and evaluation systems can facilitate public sector reform in developing countries. A new World Bank tool--a readiness assessment--helps determine whether countries can support such systems and offers additional lessons for evaluating government performance. Countries across the globe are facing pressures ity and use, to rationalize an antiquated to reform their public sectors. An effective, effi- structure. Assessing cient public sector is vital to sustainable devel- · Egypt's minister of finance, who wants to opment, economic growth, and citizens' link performance data to budget allocations government well-being. Similarly, assessing government per- to support innovations in line ministries. formance is crucial in determining a country's · Senior officials in Romania, who want to performance progress toward its development goals, whether track the performance of public sector defined by the Millennium Development Goals, reforms through M&E systems. is crucial to an a Policy Reduction Strategy Paper, a Country Results-based information systems go Assistance Strategy, or another policy statement. beyond traditional reporting on inputs, activ- effective, efficient It is also important in analyzing a government's ities, and outputs to include outcomes and accountability to its citizens, by providing evi- impacts. Put differently, governments need public sector dence on promised government performance. to develop performance feedback systems Focusing on performance can also be an effec- that complement budget, human resource, tive public sector management tool, informing and audit systems. resource allocation decisions and monitoring Building government capacity for such of whether public initiatives are achieving information systems is a long-term effort. Con- expected results. As a result governments at all sider, for example, longstanding efforts in levels are under pressure to move beyond track- many industrial countries to improve M&E sys- ing inputs--and toward measuring results. tems to assess environmental, health, and edu- cation reforms. Given that such efforts have The need for results-based been challenging in industrial countries, they management and information are even more so in developing countries. A results- or performance-based approach Facing a monumental task and limited to development management requires cred- resources, where should developing countries ible, timely, relevant information. A results- begin in implementing results-based man- based monitoring and evaluation (M&E) agement systems? Readiness assessments can system is an important source of such infor- help countries diagnose their M&E capacity mation. Champions of these systems include: and determine the resources available to sup- · The Kyrgyz Republic's minister of health, port such systems, where in government to who is developing such a system to provide begin, what incentives are in place, and what better information on health system capac- demand exists for such information. In short, from the development economics vice presidency and povert y reduction and economic management network readiness assessments help countries design political undertaking, not a technical one. results-based M&E systems that are more Thus, without political champions, there likely to succeed. This note explains why gov- is no legitimacy to move forward. In the ernment officials should consider conduct- five countries where readiness assessments ing such assessments and how Bank staff and have been conducted, the strongest cham- others can support their efforts. pion has been Egypt's minister of finance. At the opposite end of the spectrum, with Assessing readiness for results- no government champion, is Bangladesh. based monitoring and evaluation The only support there has come from A readiness assessment reviews issues related several large nongovernmental organi- to introducing results-based M&E in a spe- zations (NGOs). A readiness cific country, guiding data collection, analy- · What proposed or existing reforms can be linked sis, and feedback for government officials, to a results-based initiative? Public sector assessment donors, and stakeholders in five core areas reforms are under way in four of the five (see below). The resulting report includes countries, and results-based M&E systems reviews issues recommendations that the government, the are being developed in each. In the Kyr- Bank, and other donors can use to tailor a gyz Republic the Poverty Reduction Strat- related to results-based system to the needs of the egy Paper initiative has played a key role. country's decisionmakers. The goal is to sup- In Egypt the system is tied to Ministry of introducing port government reforms by building sys- Finance efforts to reform the budget tems that track performance. process. In the Philippines the process has results-based Bank staff have conducted readiness been driven by the Department of Budget assessments in Bangladesh, Egypt, the Kyr- and Management's involvement in a pub- monitoring and gyz Republic, the Philippines, and Romania. lic expenditure review. And in Romania it (In addition, South Africa's Public Service has resulted from reform efforts linked to evaluation Commission has undertaken its own assess- EU accession. Bangladesh is again the out- ment, and a Bank-supported assessment is lier, with no reform initiatives identified. under way in Mexico's Ministry of Social · What management framework exists within gov- Development.) ernment to oversee a results-based system? An Experiences suggest the need to pilot M&E system that informs decisionmakers results-based M&E systems before applying and citizens about government perfor- them to the entire government. Pilots can mance must collect, analyze, and dissemi- also illuminate political and technical issues nate information in the public that should be addressed at an early stage, sector--both vertically and horizontally. If clarify supply and demand issues for M&E governments have organizations that information, and suggest patterns of use of already perform some of these functions, that information. Egypt is perhaps the best they are better positioned to ensure that example of using a pilot strategy to assess the information is shared appropriately. capacities and strategies for introducing an For example, Egypt's Cabinet Decision Sup- M&E system in the public sector. Eight pilots port Center collects, analyzes, and dis- have been initiated in various ministries to seminates information to cabinet members learn about design and implementation on policy issues in their areas of responsi- issues prior to expanding the system. bility. The Kyrgyz Republic recently created Evidence from the five country diagnos- a secretariat in the president's office that tic missions is summarized below in the five coordinates information related to per- core areas: formance on development efforts. In the · Are there champions of results-based manage- Philippines the Department of Budget and ment in the government or country? This issue Management is coordinating efforts to is pivotal to whether a country can build build a performance-based budget system. a results-based M&E system. Constructing · Are there any links between budget and resource an M&E system is first and foremost a allocation procedures and performance-based PREMnote 87 June 2004 information? In all the countries assessed capacity in disciplines important to design- except Bangladesh, there is understand- ing and sustaining M&E systems, such as the ing of the need to link resource allocations social sciences, mathematics, and informa- to performance indicators. Although none tion management. But there was little expe- of the other four countries had such links, rience in using performance information to they are working to create them. As noted, guide decisionmaking. Thus it is necessary Egypt has eight performance-based pilots to strengthen both countries' understand- under way. The Philippines has made ing of modern public administration prac- efforts to establish organizational perfor- tices, including performance planning. In mance indicators linked to a medium-term short, a readiness assessment provides the expenditure framework. Romania has basis for an action plan to move forward on introduced performance indicators in its the multiple fronts--political and techni- Designing budget formulation process. And the Kyr- cal--needed for a results-based M&E infor- gyz Republic is constructing an indicator mation system. results-based system within the Poverty Reduction Strat- egy Paper process to link performance to Key lessons monitoring and budget allocations. Four key lessons emerged from the five · Where does capacity exist to support a results- readiness assessments. First, no single blue- evaluation requires based system in fields such as social science, eval- print exists for results-based M&E systems. uation, data management, and public Thus designing such a system requires understanding a management? Though this question may be understanding a country's political and insti- framed too optimistically, some capacity tutional context. For example, the assess- country's political exists to support results-based M&E system ment in Bangladesh showed that few of the in all five countries. In some countries the necessary preconditions were in place. Had and institutional capacity is quite thin (Bangladesh, the Kyr- that assessment not been conducted, a gyz Republic), in others it is more mod- results-based system might have been imple- context erate (Egypt, Romania), and in another it mented prematurely. In Egypt and Roma- is rather strong (the Philippines). An nia the readiness assessments provided vital important contribution of a readiness information about likely entry points for assessment is developing a profile of coun- results-based systems that were advocated try capacity--identifying both strengths by strong champions and that supported and areas needing support. For example, existing reforms. the readiness assessment in Bangladesh Second, sustained demand for perfor- made clear the strength of the National mance information should be encouraged Statistics Office--and the near absence of and supported, letting governments know capacity in most line and sector ministries. that they will have to show that policies and Readiness assessments enable govern- programs are achieving expected results. ments, donors, and other partners to address Important sources of demand include cham- the challenges of constructing results-based pions inside the system--who will seek such M&E systems. These challenges include ascer- information as they promote reforms--and taining political support for performance external sources--such as civil society, the management systems and determining what private sector, NGOs, and the international technical training, organizational capacity community. For example, the recent out- building, and sequencing of efforts is needed break of severe acute respiratory syndrome to develop systems that produce, collect, ana- (SARS) provides an example where inter- lyze, report, and apply M&E information to national pressure for accurate data had to improve government performance. be met by all the countries involved. More- Assessments can also help uncover insti- over, governments need prodding to make tutional and technical gaps in capacity. For reporting on results a routine activity. example, the assessments in the Kyrgyz Third, a results-based M&E system must Republic and Romania revealed significant go well beyond managing data. It must also PREMnote 87 June 2004 have sustained government leadership that not be dismissed as overly complicated, cares about whether effective services are demanding, or sophisticated for a develop- being delivered to citizens. Although it is ing country to initiate. All countries need important to have technically competent good information systems to monitor their program managers overseeing government performance--developing countries perhaps programs and projects, there must also be even more than others, given their resource strong support at the highest levels of gov- constraints. Helping developing countries ernment. Egypt's minister of finance is a develop this capacity merits the time and clear example in this regard, as is the Kyr- attention of the international development gyz Republic's president. Without strong community. champions willing to assume ownership and All countries need agree to transparent, accessible performance Further reading information, a results-based M&E system is Kusek, Jody Zall, and Ray C. Rist. 2000. "Mak- good information unlikely to be built--and even if it is, it will ing M&E Matter--Get the Foundation not be used. Right." Evaluation Insights 2 (2). systems to monitor Fourth, just as there has to be demand, so ------. 2001. "Building a Performance-based too there must be supply--capacity within M&E System: The Challenges Facing their performance-- the country to design, implement, and use Developing Countries." Evaluation Journal a results-based M&E system. Moreover, it is of Australasia 1 (2). developing not enough to develop skills in areas such as ------. 2002. "Building Results-based Moni- social research, public management, statis- toring and Evaluation Systems: Assessing countries perhaps tics, and data management. These skills must Developing Countries Readiness." Zeitschrift be mobilized to contribute to a systematic fuer Evaluation 1. even more process of regularly assessing government ------. 2003. "Readiness Assessment: Towards performance. If such skills are not present Performance Monitoring and Evaluation than others in sufficient quantities, then a concerted in the Kyrgyz Republic." Japanese Journal of capacity-building program is essential. Strate- Evaluation Studies 3 (1). gies for building such capacity might include training, mentoring, secondments, experts This note was written by Jody Zall Kusek (Regional in residence programs, and the like. Here Monitoring and Evaluation Coordinator, Africa again, pilot initiatives should be conducted Region) and Ray C. Rist (Senior Evaluation Offi- to determine which skills are present (and cer, Operations Evaluation Department). Further which are not) before large-scale imple- information and additional publications on the mentation efforts are undertaken. M&E readiness assessment can be obtained from In sum, the challenge of designing and the authors. implementing a results-based M&E system If you are interested in similar topics, consider in a developing country should not be under- joining the Public Expenditure Thematic Group. estimated. Building such a system requires Contact William Dorotinsky (x37189) or Anand champions, dedication, and long-term com- Rajaram (x82359), or click on Thematic Groups mitment to reform. But this effort also should on PREMnet. This note series is intended to summarize good practices and key policy findings on PREM-related topics. The views expressed in the notes are those of the authors and do not necessarily reflect those of the World Bank. PREMnotes are widely dis- tributed to Bank staff and are also available on the PREM Website (http://prem). If you are interested in writing a PREMnote, email your idea to Madjiguene Seck. For additional copies of this PREMnote please contact the PREM Advisory Service at x87736. PREMnotes are edited by Paul Holtz and laid out by Suzanne Luft. Prepared for World Bank staff