Research & Policy Briefs From the World Bank Malaysia Hub No. 8, July 2017 Enhancing the Quality of Public Service Delivery: Insights from Recent Research Michael Woolcock Good policies are necessary but insufficient for delivering high quality public services. Building organizations capable of imple- menting increasingly complex policies at scale is crucial. But most developing countries lack such organizations and – alarmingly – most are not improving. Effective reforms in China, Singapore, and Vietnam, however, and from unlikely service delivery successes elsewhere, suggest that three key factors are essential to building implementation capability: beginning with problems nominated and prioritized by local staff; promoting iterative experimentation to identify a range of plausible alternatives; and sharing emerging solutions through a frontline community of practice. These factors also suggest there is much to be gained from embedding researchers into delivery systems themselves, the better to understand context, to map and explain variation, and to facilitate real-time decision making in response to changing realities. Most development research papers and debates seek to What insights does recent research yield? discern the implications of empirical findings for “policy,” First, the unhappy news. As table 1 shows, the overall state the underlying assumption being that a change in policy of implementation capability around the world – as best will lead to desirable outcomes. But even if a specific shift one can measure such vexing issues – is mostly low in most in policy is clearly articulated, technically sound, and developing countries and mostly declining. Recent analy- broadly supported, it cannot be assumed that the govern- ses suggest that only one major country outside the ment can implement it, especially in low- and middle- Organisation for Economic Co-operation and Development income countries. For example, most governments in such (OECD) – Indonesia – is on track to become a “high capabil- countries have policies to minimize (if not eliminate) ity” state – a state that attains the level of the lowest OECD corruption, but too often such practices continue country – by the end of the twenty-first century. At current unabated. Indeed, one could say that a defining character- rates, many of today’s low- and middle-income countries istic of developing countries is their low capability to imple- will never reach high capability status (see Andrews, Pritch- ment complex policies, and that policy failure may often ett, and Woolcock 2017). stem less from the policy’s design than its weak implemen- tation. The picture is little better viewed from the perspective of national data. In recent years, for instance, successive Beyond policy, issues of implementation can be an impor- cohorts of Indian primary school students have performed tant focus of research: How well do governments around worse on tests of learning (Pritchett 2014). Even returning the world implement their policies? Why are some so misaddressed mail – perhaps the least difficult task that much better at it than others? Why are some improving 157 governments around the world have formally commit- more rapidly than others? Why are some policies – or ted themselves to doing – seems beyond many states, certain aspects of a policy – more difficult to implement including long-established ones (such as Russia and the than others? What can be done to improve capability for Arab Republic of Egypt) (Chong et al. 2014). Anthropolo- implementing public policy? gists studying the roll-out of pioneering social programs to Answers to these questions are beginning to become support poor elderly women lament that noble policy clearer, especially in the field of public service delivery, intentions often fail to be realized because, in the poorest where implementation quality clearly matters: we all areas, seemingly unambiguous personal facts – age, land benefit from clean streets and good schools, we all fear ownership (a proxy for wealth), number of children – are having an illness be misdiagnosed. Overseeing such unable to be administratively verified, leaving final services (if not necessarily delivering them) is a program eligibility decisions in the hands of frontline government’s core responsibility, and the realm in which administrators, too many of whom abuse this discretion, in citizens and states most frequently encounter one another. the process becoming, for these poor elderly women, part How well a government implements key services deter- of the problem rather than part of the solution (Gupta mines whether lives are saved or lost, whether citizens 2012). have a hopeful or bleak future, and whether elections are But the news is not all bad; indeed, there is much to won or lost. There are powerful moral and instrumental celebrate and much to learn from in recent studies of reasons for “getting implementation right” in all countries, effective public service delivery implementation. A defin- but especially in developing ones. ing characteristic of complex policy challenges is that the Affiliation: Development Research Group, the World Bank. Objective and disclaimer: Research & Policy Briefs synthesize existing research and data to shed light on a useful and interesting question for policy debate. Research & Policy Briefs carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions are entirely those of the authors. They do not necessarily represent the views of the World Bank Group, its Executive Directors, or the governments they represent. Global Knowledge & Research Hub in Malaysia Enhancing the Quality of Public Service Delivery: Insights from Recent Research Table 1. The (Unhappy) State of State Capability around the World Rapid negative Slow negative Slow positive Rapid positive Rate of g < -0.05 -0.05 < g < 0 0 < g < 0.05 g > 0.05 change Initial levels of State Capability (SC) Strong BHR, BHS, BRN CHL (0), SGP (0), KOR ARE (0) SC > 6.5 (0), QAT (0) Total: 8 0 3 4 1 Medium MDA, GUY, IRN, PED, EGY, CHN, MEX, LBN, KAZ (10,820), GHA (4,632), 4 < SC < 6.5 PHL, LKA, MNG, VNM, BRA, IND, JAM, SUR, UKR (1,216), ARM (1,062), ZAF, MAR, THA, PAN, CUB, TUN, JOR, OMN, RUS (231), BWA (102), NAM, TTO, ARG, MYS, KWT, ISR IDN (68), COL (56), TUR (55), CRI DZA (55), ALB (42), SAU (28), URY (10), HRV (1) Total: 45 13 18 14 0 Weak GIN, VEN, MDG, LBY, MLI, CMR, MOZ, BFA, UGA (6,001), AGO (2,738), 2.5 < SC < 4 PNG, KEN, NIC, GTM, HND, ECU, BOL, PAK, TZA (371), BGD (244), SYR, DOM, PRY, SEN, MWI, GAB, AZE, SLV ETH(103), ZMB (106) GMB, BLR Total: 32 14 12 6 0 Very weak YEM, ZWE, CIV SOM, HTI, PRK, NGA, SDN (7,270), SLE (333), NER (66), GNB (61), SC < 2.5 COG, TGO, MMR ZAR (230), IRQ (92) LBR (33) Total: 17 3 7 4 3 TOTAL: 102 30 40 28 4 Source: Adapted from Andrews et al, 2017, 20. Calculations come from rescaled indicators of state capability derived from three sources (from 2015): Quality of Government (http://qog.pol.gu.se/); (Failed) Fragile States Index (http://fsi.fundforpeace.org/); and Worldwide Governance Indicators (http://info.worldbank.org/governance/wgi/index.aspx#home). Note: The number in brackets after some countries is the years it will take for that country to have a high capability state (a state at the level of the lowest OECD country). Countries in bold are “historically developing countries.” Only 13 of them are on a path to having a “high capability” state by the end of the twenty-first century. In those countries without a number, state capability is declining. g = rate of change; SC = state capability. quality of their realization almost always varies greatly: parents, community members, and mid-level administra- such policies and programs – even when carefully designed tive officials jointly nominate and prioritize their problems, and adequately supported – are likely to work well in some and work together to identify and implement locally legiti- places and for some people, to have little effect on others, mate solutions (see Brixi, Lust, and Woolcock 2015). while for some they may even make things worse. Thus, In East Asia, the effectiveness with which service delivery the empirical challenge is to map and explain this variation, policies are implemented also varies widely, between and and to use insights gleaned from the analysis to guide within countries. Singapore, for example, has risen to the possible strategies for improvement elsewhere. A recent top of most global measures of state capability despite study of education and health care across the Middle East being a very poor country less than 40 years ago. A recent and North Africa, for example, showed that staff absentee- study of the policy implementation strategies adopted by ism at health clinics in the Republic of Yemen ranged from Singapore argues that while it identified clear aspirational 8 percent to 92 percent. The policy in Yemen, of course, is goals, it nonetheless encouraged frontline officials to that staff should show up for work each day, yet there is experiment and iterate (Goh 2016). Similarly, a study of the clearly wide variation in how well that policy is realized. In reform process in contemporary China during its rise to the highly challenging context of West Bank and Gaza, global economic powerhouse shows that local officials, far some schools are nonetheless able to teach students well from being required to forcefully apply a rigid singular enough that they score at levels comparable to average strategy, were instead given wide autonomy to use their schools in OECD countries. How do they do this? By personal networks and professional skills to promote cultivating a leadership culture that engenders high trust investments, expand economic opportunities, and identify and cooperation. The principal, students, teachers, local solutions to local problems (Ang 2016). 2 Research & Policy Brief No.8 Perhaps the most spectacular success in service delivery Expanding roles for research in enhancing service attainment has been Vietnam’s education performance. In delivery quality standardized international examinations conducted in The process by which research on service delivery effec- 2015, students in Vietnam attained scores higher than tiveness is being conducted is undergoing innovations at those in Switzerland, although Vietnam has a per capita multiple levels, from field experiments (see Banerjee, income level roughly one-tenth the size (see figure 1). Duflo, and Glennerster 2008), to systematic reviews Researchers are still seeking to explain exactly how this has (Snilstveit et al. 2016) and “big data” analyses, to case been attained, but for the moment it seems there is agree- studies (exemplified by the Global Delivery Initiative and ment that the familiar “structural determinants” of such Princeton University’s Innovations for Successful Societ- outcomes elsewhere – levels of spending, class sizes, ies), household surveys, and intensive qualitative investiga- teacher education, demographic characteristics – cannot tions (conducted by research teams embedded within account for it (Dang 2017). However, even if it is likely that development projects). The complex nature of service Vietnam’s current success can be attributed to its own delivery challenges means that, ideally, a diverse but version of the discretionary strategy used in Singapore and integrated array of methods and data is needed to China, caution should be exercised in inferring that particu- enhance the quality of decision making. But a defining lar programmatic details from these countries warrant the characteristic of effective service delivery implementers is status of global “best practices” that can be readily that they proceed through iteration. Frequent feedback adopted elsewhere, with expectations of similarly impres- from frontline staff and citizens (who in turn are given sive results (Bold et al 2013). Such “cut and paste” reform discretion, space, and resources to explore a range of strategies to improve the quality of service delivery – in plausible options) can be used to make real-time adjust- which the visible characteristics of successful approaches ments to design and practice. This process would suggest (legislation, organizational charts, training manuals) rather giving priority to research methods optimally placed to than the context-specific processes and practices that facilitate – and ensure the quality of – such strategies generated them – have a long but inglorious history based on iteration. It would also suggest that, just as there (Andrews 2013). In many instances, adopting such “best has been a revolution in recent years in the sophistication practice” reforms have enabled leaders to superficially of tools to evaluate projects, there needs to be a corre- imbue their actions with legitimacy and thereby claim sponding refinement in the tools used to monitor projects, “success,” despite the fact that so often they have shifting their role from that of compliance to learning. In achieved nothing of the sort – precisely because actual short, broader approaches to both ‘M’ (monitoring) and ‘E’ capability for policy implementation is not (and has never (evaluation) are needed if researchers are to help imple- been) acquired this way (Andrews, Pritchett, and Woolcock menters achieve their goals (see Pritchett, Samji, and 2017). Hammer 2013). Figure 1. Some Countries (like Vietnam) Provide Education Much Better than Others 600 550 Estonia Japan Finland Canada Vietnam Korea United Kingdom Hong Kong SAR, China Slovenia Germany PISA 2015 Score in Science China (B-S-J-G) New Zealand Australia Netherlands Switzerland 500 Poland Portugal Belgium Denmark Ireland France Austria Norway Latvia Spain United States Russian Federation Czech Republic Sweden Hungary Italy Croatia Lithuania Iceland Malta Israel Kazakhstan Slovak Republic 450 Bulgaria Chile Greece Uruguay Malaysia Alb ania Romania United Arab Emirates Moldova Cyprus Argentina Costa Rica Thailand Turkey Trinidad and Tobago Colombia Mexico Georgia Montenegro Jordan 400 Indonesia Brazil Peru Tunisia Lebanon Macedonia Kosovo Algeria 350 0 10000 20000 30000 40000 50000 60000 70000 80000 GDP Per Capita in 2015 or Latest (Constant 2011 International $), World Bank ICP East Asia & Pacific OCED Middle East North Africa Eastern Europe Central Asia Latin America & Caribbean High Income East Asia Source: OECD, data from the 2015 Programme for International Student Assessment (PISA). http://www.oecd.org/pisa/data/. Note: For China, the weighted average of provincial GDP per capita is shown. Acknowledgement: Yilin Pan and Amer Hasan produced the graph. 3 Enhancing the Quality of Public Service Delivery: Insights from Recent Research Rao’s work with the Social Observatory in India exemplifies to accommodate populations that continue to grow in size what such an approach might look like in practice. Drawing and diversity, and whose expectations of their govern- on approaches from across the methodological spectrum, ments – especially in education – will only rise as the research teams work closely with project implementers to demands of an increasingly connected and competitive discern not just “whether” a given intervention is working world becomes apparent (Kim 2017). One reading of the on average, but “how” and “for whom” (see Ananthpur, prevailing evidence would lead to rather pessimistic Malik, and Rao 2014; Joshi and Rao 2017). It is often not at conclusions regarding the likelihood that these expecta- all obvious what changes need to be made over time and tions can be met. But a different reading would suggest across contexts to improve project effectiveness, and most that researchers, practitioners, and policymakers have an frontline staff do not have the luxury of waiting three years opportunity to respond to these challenges in creative for empirical findings from “gold standard” evaluations to ways, forging innovative strategies built on new under- make decisions. Thus, deploying a mixed methods standings of how organizations actually acquire effective research approach to generate “good enough” evidence in implementation capability. real time is likely to be the most useful (Bamberger, Rao, These strategies have an important place for research, and Woolcock 2010). Similar interdisciplinary initiatives to though one centered less on identifying universal “best enhance the quality of service delivery are being practices” (as determined by international experts) that pioneered in Indonesia (Alatas 2017). Dedicated “delivery can be replicated and scaled, and focused more on under- units,” such as those being pioneered in middle-income standing the forms and sources of variation from context countries such as Malaysia, can also play an important role to context. Such insights can help identify how and where in rapidly sharing specific insights on innovative problem- local solutions have emerged in response to local prob- solving strategies across professional communities of lems, and promote the sharing of these insights with practice. others. Doing this well requires trusting mid-level and frontline staff to find and amplify the successes that some- Conclusions, implications, and applications one somewhere somehow has already achieved. Development challenges in the coming decades will require governments to implement increasingly complex Note. This Research & Policy Brief summarizes insights drawn from a research conference on “Enhancing the Quality of Service Delivery,” and (potentially) contentious policy agendas, at scale. hosted in Kuala Lumpur in January 2017 by the World Bank’s Knowl- Administrative systems that are already struggling will have edge and Research Hub in Malaysia. References Alatas, Vivi. 2017. “Service Delivery to Reduce Inequality.” Paper presented Shleifer. 2014. “Letter Grading Government Efficiency.” Journal of the at the “Enhancing the Quality of Service Delivery” conference, World European Economic Association 12 (2): 277–99. Bank Knowledge and Research Hub, Kuala Lumpur, Malaysia, January 16. Dang, Hai-Anh. 2017. “High Quality Education Despite Resource Constraints? Ananthpur, Kripa, Kabir Malik, and Vijayendra Rao. 2014. “The Anatomy of The Case of Vietnam.” Presentation at World Bank Policy Research Failure: An Ethnography of a Randomized Trial to Deepen Democracy Conference on “Enhancing the Quality of Service Delivery,” Kuala in Rural India.” Policy Research Working Paper 6958, World Bank, Lumpur, Malaysia, January 15-16. Washington, DC. Goh, Nigel. 2016. Detours, Dead-Ends and Diversions: Singapore’s Road to Andrews, Matt. 2013. The Limits of Institutional Reform in Development: Development Reconsidered. Singapore: UNDP Global Centre for Public Changing Rules for Realistic Solutions. New York: Cambridge University Service Excellence. Press. Gupta, Akhil. 2012. Red Tape: Bureaucracy, Structural Violence and Poverty in Andrews, Matt, Lant Pritchett, and Michael Woolcock. 2017. Building State India. Durham, NC: Duke University Press. Capability: Evidence, Analysis, Action. New York: Oxford University Press. Joshi, Shareen, and Vijayendra Rao. 2017. “Who Should Be at the Top of Bottom-Up Development? A Case Study of the National Rural Ang, Yuen Yuen. 2016. How China Broke the Poverty Trap. Ithaca, NY: Cornell Livelihoods Mission in Rajasthan, India.” Policy Research Working University Press. Paper 7996, World Bank, Washington, DC. Bamberger, Michael, Vijayendra Rao, and Michael Woolcock. 2010. “Using Mixed Methods in Monitoring and Evaluation: Experiences from Kim, Young Eun. 2017. “The Role of Education in Enhancing Productivity: International Development.” In Handbook of Mixed Methods in Social Some International Comparisons.” Presentation at World Bank Policy and Behavioral Research, 2nd revised edition, edited by Abbas Tashak Research Conference on “Enhancing the Quality of Service Delivery,” kori and Charles Teddlie, 613–41. Thousand Oaks, CA: Sage Publication. Kuala Lumpur, Malaysia, January 15–16 Banerjee, Abhijit V., Esther Duflo, and Rachel Glennerster. 2008. “Putting a Pritchett, Lant. 2014. “The Risks to Education from Design Mismatch and Band-Aid on a Corpse: Incentives for Nurses in the Indian Public Health Global Isomorphism.” WIDER Working Paper 2014/039, United Nations Care System.” Journal of the European Economic Association 6 (2–3): University. 487–500. Pritchett, Lant, Salimah Samji, and Jeffrey S. Hammer. 2013. “It's All about Bold, Tessa, Mwangi Kimenyi, Germano Mwabu, Alice Ng'ang'a, and Justin MeE: Using Structured Experiential Learning ('e') to Crawl the Design Sandefur. 2013. “Scaling Up What Works: Experimental Evidence on Space.” Faculty Research Working Paper 13-012, Harvard Kennedy External Validity in Kenyan Education.” Working Paper 321, Center for School, Cambridge, MA. Global Development, Washington, DC. Snilstveit, Birte, Jennifer Stevenson, Radhika Menon, Daniel Phillips, Emma Brixi, Hana, Ellen Lust, and Michael Woolcock. 2015. Trust, Voice and Gallagher, Maisie Geleen, Hannah Jobse, Tanja Schmidt, and Incentives: Learning from Local Success Stories in Service Delivery in the Emmanuel Jimenez. 2016. “The Impact of Education Programmes on Middle East and North Africa. Washington, DC: World Bank. Learning and School Participation in Low- and Middle-income Countries’ Systematic Review.” Summary 7, International Initiative for Chong, Alberto, Rafael La Porta, Florencio Lopez-de-Silanes, and Andrei Impact Evaluation, New Delhi. 4 Global Knowledge & Research Hub in Malaysia