Report No: ACS3202 Latin America Strengthening Monitoring and Evaluation (M&E) Networks for Evidence Based Decision Making in the LAC Region April 15, 2013 LCSPP LATIN AMERICA AND CARIBBEAN Page | 1 Standard Disclaimer: This volume is a product of the staff of the International Bank for Reconstruction and Development/ The World Bank. The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Copyright Statement: The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750- 4470, http://www.copyright.com/. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. Page | 2 Effective policy making hinges on information - information that guides governments as to whether they are doing the right things, the right way, and how they might be done better. Strong monitoring and evaluation (M&E) systems provide the means to compile and integrate this valuable information into the policy cycle, and consequently serve as an important contribution to sound governance and accountable public policies. Better evidence and M&E systems are valuable tools to ensure greater policy efficiency and effectiveness in terms of coverage, targeting, and delivery of services, and thus critical for achieving poverty reduction and shared prosperity. M&E systems rely on relevant, timely, reliable and user-friendly information. Progress in developing systematic M&E frameworks and models across the Latin America and Caribbean region (LAC) reflects countries’ institutional capacity, regulatory frameworks, data systems and degree of political leadership and commitment. Most of the countries face a common set of challenges such as strengthening the quality of information, creating and implementing incentives and tools to use this information and increasing capacity for the generation, management and application of M&E information. An M&E Network is Created Over the past fifteen years, the World Bank has accompanied countries in overcoming these challenges, providing tailor-made support around areas such as improving the quality of data; linking demand and supply for increased use of M&E information; supporting institutional development; and, promoting capacity building and knowledge sharing. Since 2005, the World Bank has played a leading role in the creation and advancement of the M&E Network for Latin America and the Caribbean, or REDLACME as it is known among members, to promote the sharing, dissemination, and building capacity on M&E in the region. Jointly with the Inter-American Development Bank (IDB), the World Bank has supported the M&E network as a vehicle to promote regional dialogue among M&E stakeholders, including academics, government officials and M&E practitioners throughout Latin America and the Caribbean. REDLACME now serves as a permanent, well-established forum to share experiences, build capacity and stimulate collaboration. The Network began and now culminates in an annual conference, BOX 1. The SFLAC Grant which has become a premier event among M&E Title: Strengthening Monitoring and Evaluation professionals and practitioners in the region. A constant Networks for Evidence-Based Decision- Making in the LAC Region challenge for a Network with the character of a public TF No: TF096261 good is to have the financial and human resources Period: January 2010 to December 2012 required to maintain and facilitate it. Amount: US$ 350,000 Implemented by: The World Bank (LCSPP) In January 2010, the World Bank was able to secure Donor: The Ministry of Economy and Finance of additional support from the Spanish Fund for Latin Spain, through the Spanish Fund for Latin America and the Caribbean (SFLAC) that served to America and the Caribbean (SFLAC) ensure the continuation of the Network and to intensify the work to promote capacity building and knowledge. The funding aimed to support the development and consolidation of existing experiences of monitoring and evaluation (M&E) systems through knowledge sharing and dissemination, building capacity for effective use of M&E through South-South learning, and facilitating the national dialogue through technical assistance to strengthen country-level M&E systems and framework. Page | 3 This was achieved through five components of work: Co-organized and partly financed the 2010, 2011 and 2012 annual regional M&E 1- Annual Regional M&E conferences, bringing together key players in the development and use of M&E systems Conferences from LAC and beyond. Further developed a public website for the regional M&E network as an easily-accessible 2- M&E Network Website source of information, instruments and exchange among M&E professionals, including documents and publications presented at the annual conferences. 3- Technical Assistance Technical guidance provided regarding the formation of national chapters, development of on M&E National their agenda, dialogue among the various national stakeholders and linkages of the M&E Chapters system with policymakers. 4- M&E Learning Developed reference materials and information to provide guidance on the development Resources and implementation of M&E systems and evaluation methodologies. Connected countries that are more advanced in the development of their M&E systems with 5- South-South Exchanges other countries in the region with lesser developed systems to allow for learning and exposure to innovations. From Network to a True Community of Practice The SFLAC grant amplified the regional participation in the Network and the Conference, which allowed for a quantitative and qualitative step towards a true Community of Practice for M&E in the LAC region. At the same time, national ownership was fostered through the support to national chapters and the creation of joint Conference organization committees with the active participation and co-sponsorship of the hosting country, the IADB and the World Bank. The M&E Network for Latin America and the Caribbean is an important forum to stimulate ideas, learn, and receive feedback to advance in the implementation and strengthening of M&E systems at the national and/or sub-national level. It is regarded by donors, international organizations, and government officials as an effective source for increasing knowledge and capacity on M&E to achieve the goal of good governance and poverty reduction. The LAC Regional M&E Network has been able to extend its reach beyond its annual conference to provide a diverse set of mechanisms and tools to support the implementation of innovative strategies and tools for the effective use of M&E strategies for evidence-based decision-making. The Network is guided by both an online and an offline strategy that feed into each other. In particular, the offline part of the strategy, mainly composed by the Conferences and South-South knowledge exchange, mobilizes stakeholders and generates content that feeds into the online arm of the Network. Over the last few years, the Network has strengthened all the factors that lead to a successful network: building up a critical mass of individuals; making a concerted effort to attract and sustain a plurality of institutions to protect the network from political cycles and influences; and, engaging a group of champions to facilitate and moderate the content of the collaborative platform1. The website for the Network (redlacme.org) is a knowledge platform that functions like a social networking site, where the members have the ability to upload information, post blogs, provide comments and pose questions on the latest development on particular issues related to M&E, allowing it to be a dynamic source for updated knowledge and innovative resources. The SFLAC funds allowed the World Bank to spend more effort on connecting with network members to promote and facilitate their active participation. And since late 2009 the website has developed into a highly interactive platform, largely sustained by the inputs of its members, in 1 Henriques, Pinho, Azevedo & Newman, 2010. “The Brazilian Monitoring and Evaluation Network: A Report on the Creation and Development Process�, presented in The World Bank and the Inter American Development Bank, 2010. Challenges in Monitoring and Evaluation: An Opportunity to Institutionalize M&E Systems. Page | 4 which the World Bank, as administrator, only is required to do minimal maintenance. The knowledge platform includes the following instruments for virtual dialogue and information:  Knowledge platform (community of practice): http://redlacme.org  YouTube channel: http://www.youtube.com/user/redlacme  Dynamic dashboard with network membership (and evaluation): http://redlacme.org/page/dashboard  Conference Web sites: http://go.worldbank.org/34FD2QFA40, http://go.worldbank.org/4INSDCSMC0  LAC M&E Activities Web site (WB External Web site): http://go.worldbank.org/YF7IB59JY0 The LAC region is the only one with a regional knowledge platform for Monitoring and Evaluation, and is a clear example of the development of an innovative and effective community of practice. The team is at forefront of the World Bank experience on the use of Web 2.0 technologies to engage with our external audience and manage knowledge, while giving our clients a leading role in this process. The knowledge platform serves to share and consolidate experiences in M&E to derive best practices and complementing resources for adaptation and replication. The M&E Network was initiated with 40 members in 2005, and the number of members and participants has increased and diversified ever since. Before the SFLAC grant, in January 2010, there were 420 members registered in the M&E Network, and just three years later this figure has increased to almost 7,500 members. About half, 48 percent, of the Network members are affiliated with the public sector, followed by consultants (21%) and private sector (14%). Seven percent are students and 10 percent identified themselves as “other� or do not disclose affiliation. In terms of country coverage, the Network has members from over 60 countries worldwide, including eight African countries (See Figure 1). The World Bank continues to be a principal player in the Network, being the secretariat for the Network as well as a primary technical advisor to its members, in particular to the recent national chapters of the M&E network. The countries with the largest share of members, Brazil, Colombia, Peru and Mexico, are the ones that have the longest standing national chapters of the M&E Network. The Brazilian Chapter initiated in 2008 with the support of the World Bank, and has since then taken on a life of its own, hosted by a consortium of federal, sub-national and NGO institutions, with yearly national conferences and a very active platform for knowledge and information sharing for a total of almost 4,000 members (See Figure 2). Colombia launched their national M&E chapter in 2010 and it is hosted by the National Planning Ministry (DNP), and has over 1,200 members. Mexico followed thereafter and has over 700 members affiliated to their Network. Since 2010, Peru, Paraguay and El Salvador also have launched national chapters. The launches of national chapters have served to focus on country specific M&E issues, and they also have had an extremely positive impact on boosting country membership and active participation in the regional M&E Network. The focus of these country level networks is on sharing knowledge, and promoting learning and capacity building on M&E among national and sub-national entities. With the SFLAC funds, the World Bank has been able to provide technical assistance to engage key actors in transforming the national chapters into effective mechanisms for strategic action and in the development of their action plans and training courses. While the virtual Network and its national chapters have been crucial to provide policy-makers and practitioners with invaluable knowledge to inform their work towards better evidence-based policy-making based on systems that provide timely and reliable information, the annual regional M&E conferences have been a key space to interact and learn from other experiences. The first two regional M&E conferences were held in Washington, DC Page | 5 in 2005 and 2006, the III conference was in Peru, the IV in Brazil and the V one in Colombia. For the last three annual regional M&E conferences, the World Bank was supported with additional funding from SFLAC, which allowed for medium- term planning and a closer engagement with the host countries and leveraged important co- sponsorship as well as national ownership by the host countries. It was not until 2010 that the Conference planning and preparation has been done in a truly collaborative spirit with joint tri- partite organizing committees with the host country, the IADB, and the World Bank coming together at an early stage to identify the conference content and speakers. Each conference physically gathered between 250 and 300 experts, practitioners, policy-makers and representatives from academia from a wide range of countries, in addition to another couple of hundred followers through web streaming and Twitter. The main objective of the conferences were to share their experiences and lessons learned on specific themes such as The Use of M&E Results in Public Policy Cycles (2010), Challenges and Contributions of M&E in the Planning and Public Management Processes (2011) and Strengthening M&E for Inclusive Public Policies (2012). As a key input for the Conferences, and a way of identifying and sharing innovative good practices throughout the region, yearly Calls for Papers have been launched since 2010. In 2012, for example, over 50 papers were received from 15 countries. Based on pre-identified selection criteria, a joint World Bank – IADB – host country government counterpart selection committee identified the six best papers. The authors were invited to participate in the Conference as speakers to share their experiences, particularly to discuss the lessons learned and innovative aspects that could be replicated or scaled up. This was also the first year where the conference participants then had a chance to choose and award among the finalists the most replicable, most innovative and best overall presentation. Another additional element that has evolved over the years is the workshops held in conjunction with the Conferences since 2010. The objective of these workshops is to build technical capacity in identified areas of relevance to the Conference theme and of interest to the participants in a smaller setting that allowed for a deeper interaction and exchange among the practitioners and the trainers. Training has been provided on sector specific issues, new monitoring methods, information and data sharing tools, different models for implementation and use of M&E to promote public policies, among other subjects. The overall evaluation of both the workshops and conference have been quite positive, with 88 percent of the participants in the 2012 Conference evaluated it as “excellent� or “good�. Both the virtual Network and the Conferences have provided valuable learning and knowledge on monitoring and evaluation. The idea of the Network is that it should be more than a content repository, but play a broker role to bring new knowledge on M&E, while maintaining the links with the knowledge providing institutions. All materials produced for the Conferences can be found on the website of the Network. Certain information from the Conferences also has been consolidated into specific knowledge resources, like the publication Challenges in Monitoring and Evaluation: An Opportunity to Institutionalize M&E Systems that compiles challenges and lessons learned from M&E practitioners from around the world. This publication provides an overview of examples and knowledge on the role of monitoring and evaluation in evidence-based decision-making, international and regional experiences of institutional arrangements for M&E, as well as showcasing different impact methodologies. The SFLAC funding functioned to leverage partnerships with other units within the World Bank for the conceptualization, production, translation and dissemination of the Nuts and Bolts of M&E series Page | 6 that aims to increase knowledge about M&E systems through a regular series of papers focusing on the design, implementation and use of M&E information by governments and civil society based on best practice experiences from around the world. Since the series started in 2010, there are more papers from the LAC region than from any other region, and examples from different countries throughout the region are frequently used in the notes of a more general character2. The focus for all the work related to the M&E Network is on South-South exchange. Both the virtual platform and the conferences allow for information sharing and knowledge exchange among peers. These spaces serve not only to identify the kind of knowledge and experience that exist in the region, but also to identify technical assistance required by certain countries and the World Bank has been able to facilitate the matching of supply and demand in several cases. Exchanges have been conducted by bringing experts and practitioners from the region to conferences and workshops in which specific bilateral meetings have been arranged in parallel to spur collaboration on issues of specific country relevance. By taking advantage of the wide conference participation, a larger number of countries have been able to participate in these kinds of exchanges. The SFLAC funds have also been used to leverage South-South exchange within the World Bank across sectors. For example, in 2010 field trips and visits on M&E that were organized in the region, for example Salvadorian government’s visit to Mexico shared their lessons learned through the Network. For the 2011 Conference, Social Protection (Human Development) colleagues brought counterparts and clients for a specific session on the institutionalization of M&E in Social Development Ministries. OECS stakeholders were invited to the Conference and also to meet with Mexico and the Dominican Republic for valuable inputs on the challenges and lessons learned on establishing poverty committees. In conjunction with the 2012 Conference, the Gender Impact Evaluation Initiative facilitated South-South exchange of experiences working on gender impact evaluations based on a presentation of impact evaluation study design and recent gender specific findings from the region with the participation of government ministries, NGOs, think tanks and research institutes from several countries including Costa Rica, Guatemala, Mexico and El Salvador. What Can We Learn from this Experience? Ownership increases participation and sustainability. The Regional M&E Network is soon to turn a decade old, which is quite a long life for a community of practice. The social networking platform (redlacme) is functioning principally on the content and exchange of the regional members, which has greatly contributed to its effectiveness and success. It is expected that as capacity and knowledge grows through the network, this sense of ownership will be further strengthened. It is expected that the World Bank will continue to play a key role as convener and knowledge provider to the Network, as the regional members South-South exchange as a driver of supply and demand. The LAC M&E Network is built largely on interest and demand. Although the World Bank and IDB have been supporting sponsors, the network essentially functions as a public good, similar to a social network, where information is shared and utilized through various media, from web information portals to individual professional exchanges, to the specific needs and benefits of its members. As much of the work of the Network is regional in nature, often involving South-South exchanges, it is important to provide seed funding and facilitate the exchange. The SFLAC resources allowed the Bank to respond to client demand for assistance and be less restricted in limiting the scope of activities. The Conferences and their adjacent workshops have been highly valued for providing in-depth knowledge on specific issues and showcasing innovative and high quality experiences throughout the region that focus on practical solutions. Tying together operational work and the M&E Network creates a virtuous cycle. The M&E Network has shown that as multilateral organizations and development banks there is an opportunity to leverage the M&E agenda 2 The LAC specific Nuts and Bolts publications produced are “The State Results-Based Management System of Minas Gerais� (No. 18, April 2012); “The Mexican Government’s M&E System� (No. 14, September 2011); and, “Chile’s Monitoring and Evaluation System, 1994- 2010� (No. 13, August 2011). Page | 7 through existing operational knowledge and experience, showcasing and sharing lessons learned that could be replicated and/or scaled up. The Network can both serve as dissemination channel for innovative experience exchange and discussion, telling the stories of the operational work, and be strengthened by having the engagement in operations feed into the content and best practices generation. The richness of the Network can be mutually beneficial to the World Bank operations and country dialogue. Knowledge sharing is maximized by providing a cross-sectoral perspective. M&E is by nature a cross-sectoral science. Within the World Bank, the collaboration with colleagues from the IEG, LCSPS, WBI, PREMPR, LCRHD and LCSSO has been crucial to ensure the best identification of experiences and knowledge to share. In particular, the Independent Evaluation Group (IEG) has played a key role in the M&E Network, providing both technical and financial support. M&E experts within IEG have served as key technical advisors to lending operations as well as advisory and analytical activities by bringing together different sectors. The content of the virtual knowledge platform and the conferences is enriched by the fact that they bring together stakeholders representing the public and the private sector, NGOs and academia, individual practitioners and experts. Also, by combining strategically chosen experiences and competitively selected ones, the network is able to have both knowledge gathering and sharing functions. Partnering across institutions allows for making the most of the value added and knowledge of each institution. This work would not have been possible without also a number of partnerships outside the World Bank, starting with the IADB who has been a partner since the very beginning in 2005. It involves building close relationships with government and civil society counterparts in the countries that have created national chapters, such as Brazil, Mexico, Colombia, Peru, El Salvador and Paraguay, and include institutions that are more advanced on the M&E agenda such as Mexico, Colombia, Brazil, Peru, to name a few. Strategic partnerships across institutions allows for an increased pool of knowledge and contacts to provide the most relevant and adequate experiences to be shared in the Network. BOX 3. “The more we can share of what works in our countries the better able we will be to face up to those challenges � Ray Shostak, CEO of Core Assets Group, former Head of the Delivery Unit and Performance Management Department for the Prime Minister of Great Britain. Participated as a speaker in the VI Regional Conference on M&E in the Dominican Republic in 2011. “I have absolutely no doubt that we can all learn from each other […] The Network that has been created is an important and quite fundamental device to ensure that we are sharing best practice and learning from each other. That is equally true for the number of speakers that have come from outside of Latin America, like myself, colleagues from South Africa, Korea and other countries. We are equally able to learn from the practices here and the challenges here. […] The more we can actually share of what works in our countries the better able we will be to face up to those challenges. […] This event has been very, very good in terms of bo th its design and the contributions in raising awareness of the issues that are actually faced by each of our individual countries. I think the real challenge for us all is to ensure that we are taking that awareness and turning it into the development of real skills and new approaches.� Susana Gamez, Technical Director in the Office for the Coordination of Social Policies in the Government of the Dominican Republic. Co-hosted the VI Regional Conference on M&E in the Dominican Republic. “I have thoroughly enjoyed seeing in this seminar the diversity of people from institutions around the Dominican Republic […] Seminars currently offered are very short. In many cases you walk out of them thinking about problems or ideas, but without a clear skill to implement them in a country. These monitoring and evaluation networks provide a space for all countries to come together via video conference and have a real dialogue about a specific subject. We all have common problems in Latin America. We are not that different. […] By the way of these networks, we are able to see the progress or th e setbacks each country face in different areas. […] I believe we all have contributions to make, because not all of us have th e same experience or time involved in this process.� Ignacio Irrarrazaval, Director of the Public Policy Center of the Pontificia Universidad Católica in Chile. Participated in the VIII Regional Conference in Costa Rica 2012. “The dialogue between academia, practitioners and public sector is crucial. […] I think that the challenge is to build the M&E systems step by step, not to overburden the countries by saying that if they do not have all these tools their systems do not work. I think that it is important to convey the message that countries can start step by step. “ Page | 8