Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004897 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF0A2524) ON A SMALL GRANT IN THE AMOUNT OF USD2.5 MILLION TO THE Justice Center for Legal Aid FOR Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) September 5, 2019 Governance Global Practice Middle East And North Africa Region Regional Vice President: Ferid Belhaj Country Director: Saroj Kumar Jha Senior Global Practice Director: Practice Manager: Renaud Seligmann Task Team Leader(s): Klaus Decker ICR Main Contributor: Klaus Decker ABBREVIATIONS AND ACRONYMS ADR Alternative dispute resolution CPF Country Partnership Strategy CSO Civil Society Organization DO Development objective progress EA Environmental assessment FPD Family Protection Department FY Fiscal Year GoJ Government of Jordan GRM Grant Reporting and Monitoring Report ICR Implementation Completion and Results Report ISR Implementation Status and Results Report IP Implementation progress JBO Jordan Bar Association JCLA Justice Center for Legal Aid JRP Jordan Response Plan MOJ Ministry of Justice MoSD Ministry of Social Development M&E Monitoring and Evaluation NGO Non-governmental organization PDO Project Development Objective SCD Systematic Country Diagnostic SPF State and Peacebuilding Trust Fund TF Trust Fund UNHCR United Nations High Commissioner for Refugees UNRWA United Nations Relief and Works Agency USD United States Dollar TABLE OF CONTENTS DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 3 II. OUTCOME ...................................................................................................................... 5 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 11 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 12 V. LESSONS LEARNED AND RECOMMENDATIONS .............................................................. 13 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 14 ANNEX 2. PROJECT COST BY COMPONENT ........................................................................... 20 ANNEX 3. ICR STAKEHOLDER WORKSHOP MINUTES ............................................................. 24 ANNEX 4. RECIPIENT COMPLETION REPORT SUMMARY AND COMMENTS ............................ 26 ANNEX 5. SUPPORTING DOCUMENTS .................................................................................. 29 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Piloting Delivery of Justice Sector Services to Poor P157861 Jordanians and Refugees in Host Communities Country Financing Instrument Jordan Investment Project Financing Original EA Category Revised EA Category Not Required (C) Organizations Borrower Implementing Agency Justice Center for Legal Aid Justice Center for Legal Aid Project Development Objective (PDO) Original PDO The development objective is to increase access to legal aid services (information, counseling and legal representation) for poor Jordanians, particularly women, and refugees in host communities. FINANCING FINANCE_TBL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF-A2524 2,500,000 2,500,000 2,500,000 Total 2,500,000 2,500,000 2,500,000 Total Project Cost 2,500,000 2,500,000 2,500,000 Page 1 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) KEY DATES Approval Effectiveness Original Closing Actual Closing 10-May-2016 12-Jun-2016 15-Mar-2019 15-Mar-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 27-Jan-2018 Satisfactory Satisfactory 2.23 ADM STAFF Role At Approval At ICR Regional Vice President: Hafez M. H. Ghanem Ferid Belhaj Country Director: Ferid Belhaj Saroj Kumar Jha Director: James A. Brumby Najy Benhassine Practice Manager: Hisham Ahmed Waly Renaud Seligmann Task Team Leader(s): Paul Scott Prettitore Klaus Decker ICR Contributing Author: Klaus Decker Page 2 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES Context Country and Sector Context at Appraisal 1. Jordan was experiencing the effects of security spillovers from neighboring countries. The security situation in Syria and Iraq was worsening and repercussions on tourism and trade in particular had slowed down Jordan’s economy in 2015. At the same time, the influx of refugees from Syria was squeezing the country’s already strained public services. The courts, for example, had been reporting increased caseloads that were challenging the technical and operational capacity of the judicial system. By time of appraisal, 635,324 Syrian refugees had registered with UNHCR in Jordan and many estimated the actual number of refugees much higher. While Jordan had hosted Palestinian and Iraqi refugees for decades, the new influx of refugees created competition for public services between poor Jordanians and refugees in host communities, leading to increased tensions and social conflict in communities hosting large numbers of refugees. 2. Already prior to the refugee influx, access to legal aid services such as legal counseling and representation was insufficient to meet the needs of poor and vulnerable Jordanians, and women in particular. At the same time, poor and near poor were considerably more likely to experience legal problems than other Jordanians. Since Jordanian law did not guarantee free legal counsel, the complexities of accessing government services and navigating bureaucracy constituted significant access barriers. In terms of legal representation in a court of law, those who could not afford a lawyer were not granted legal aid by the state except in criminal cases where the crime is punishable by execution or life imprisonment. When poor and vulnerable people went to court despite existing access barriers, they were left to represent themselves despite their lack of knowledge and court procedures that can be very technical. 3. Legal needs of incoming refugees considerably exacerbated the situation through the sheer volume and more complex nature of their legal problems. They were often linked to missing or incomplete documentation related to their countries of origin. Refugees also faced the additional barrier of their own distrust in government institutions while being in a foreign country with unfamiliar laws. They did not have the knowledge to navigate the system and could not tap into relevant experience and support of a social network in their host countries. This made them vulnerable to abuse and exploitation. Rationale for World Bank Support 4. The project supported priorities of the Vision Jordan 2025 and the Jordan Response Plan (JRP) to the Syrian refugee crisis. Improving the institutional capacity of the judicial system was a need identified by the Vision. The JRP 2016-2018 in particular aimed at ensuring “quality and prompt access to the justice system for all women, girls, boys, and men in Jordan in governorates affected by the Syria crisis.� 5. The project built on findings of the Systematic Country Diagnostic (SCD). Two of them were key for the design of this project. First, social tensions were rising following the large influx of Syrian refugees and the resulting pressure that this had placed on already stretched services. The SCD found that government services were struggling with meeting the needs of Syrian refugees while poor Jordanians’ pre-existing concerns about deprivation and unequal access to services were exacerbated. Second, the SCD found that inequality based on gender was widely documented and referred to a country gender assessment carried out in 2013, which identified the “ability of women to effectively access justice sector mechanisms, such as courts, is essential for them to challenge social norms and discriminatory legal frameworks.� Page 3 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) 6. The project was consistent with World Bank strategic objectives. Under the outgoing Country Partnership Strategy it was in line with pillar 1 (strengthen fiscal management and increase accountability, including strengthening of public service delivery) and pillar 3 (enhance inclusion), but was really at the center of the priorities of the Country Partnership Framework (CPF) FY17-FY22 that would be adopted shortly after project approval. It identified governance, gender and youth issues addressed by the project as key and established them as cross-cutting themes. Moreover, the project supported the implementation of the CPF’s second pillar aiming to address growing inequalities in provision and an erosion of quality of public services, exacerbated by the Surya crisis. Project Development Objectives (PDOs) 7. The development objective was to increase access to legal aid services (information, counseling and legal representation) for poor Jordanians, particularly women, and refugees in host communities. While this PDO of the legal agreement was identical with the one in the main body of the Project Paper, the PDO in the Project Paper data sheet was slightly different and reads: The development objective is to increase access to justice sector services for poor Jordanians, particularly women, and refugees in host communities through innovative service delivery reforms. Key Expected Outcomes and Outcome Indicators 8. The project aimed at achieving the following PDO level results: (1) Enhanced provision of legal services, measured by the number of legal counseling and representation services provided; (2) increased awareness among beneficiaries about legal rights and procedures for enforcing them, measured by the number of beneficiaries that report an increased knowledge following awareness sessions; and (3) increased effectiveness of beneficiary referral mechanisms involving new partners, measured by an increase in the number of referrals and number of trainings delivered to GoJ partners making referrals. Components 9. The project included the following four components: (total USD2,500.000). 10. Component 1: Enhancing Legal Services to Communities Impacted by the Refugee Crisis (estimated USD1,225,157, actual USD1,562,728). Under sub-component 1.1 the project carried out a program of activities expanding the scope and reach of legal services by (1) piloting a legal aid referral system with the Ministry of Social Development (MoSD) on cases related to juvenile justice and violence against women and girls; (2) increasing the scope of alternative dispute resolution services by setting up an Alternative Dispute Resolution (ADR) facility, hiring staff, developing guidelines and conducting staff training among others; (3) providing legal aid services pertaining to the needs of refugees and vulnerable members of their host communities; and (4) reviewing and assessing priority legal aid issues on a case-by-case basis to ensure that resources are available to the poor and vulnerable in Jordan. Under sub- component 1.2 the project developed partnerships with the legal community by strengthening the connection of the Justice Center for Legal Aid (JCLA) with lawyers and institutions within the legal community to expand the reach of its services to the poor and vulnerable in Jordan. 11. Component 2: Increasing Legal Advocacy and Awareness (estimated USD569,082, actual USD503,037.50). Under sub-component 2.1 the project enhanced JCLA’s legal advocacy by (1) engaging more effectively with the public and the government on issues of institutionalizing legal aid in Jordan through the provision of training to JCLA staff, legal consultants, legal community members and members of the public; (2) further developing its strategy to enhance the provision of sustainable and institutionalized legal aid services to the poor and vulnerable; (3) implementing its strategy through workshops, conferences and meetings with relevant legal aid sector stakeholders such as government entities and the community; (4) publishing assessments and analysis on legal aid topics; (5) establishing and improving platforms Page 4 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) through the use of alternative media; (6) enhancing community awareness and engagement on the concept of legal aid through community organizing and citizen engagement tools; and (7) conducting research and desk reviews on legal aid and legal issues to more effectively engage with stakeholders. 12. Under sub-component 2.2 the project expanded and strengthened legal awareness activities by (1) increasing the number of legal awareness partnerships (2) conducting training sessions on legal education topics in UNRWA school in Jordan; (3) carrying out a targeted child’s rights campaign; and (4) producing legal awareness materials tailored to different audiences. 13. Component 3: Capacity Building (estimated USD469,131, actual USD109,927). .Under sub-component 3.1 the project supported capacity building within JCLA by (1) enhancing intake and eligibility assessment processes, (2) improving the case management system, (3) developing ICT for improving information sharing and long-term knowledge management; and (4) providing continuing legal education for JCLA staff. 14. Under sub-component 3.2 the project supported capacity building of key government partners including to MoSD to enhance the ability of its staff to effectively manage the legal aid system in juvenile detention centers; to the Family Protection Department (FPD) by providing training on domestic violence, developing awareness materials for victims of domestic violence and establishing legal aid desks at FPD centers; and to the Ministry of Justice (MOJ) to establish its Legal Aid Unit. 15. The actual cost of this component is only 26.4% of the estimated cost. The reason for this variation was the challenging relationship with the MOJ, which made it impossible to disburse funds on activities with them. By the end of 2017, JCLA therefore requested a budget reallocation to activities with other partners as well as service provision under component 1. The amount allocated to component 1 is therefore higher than originally estimated. 16. Component 4: Project Management and Monitoring and Evaluation (estimated USD236,630, actual USD324,307.50). This component supported grant management by JCLA, including coordination and supervision of implementation activities, financial management, annual grant audit, and monitoring and evaluation. II. OUTCOME Assessment of Achievement of Each Objective/Outcome Page 5 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) A Relevance of the project development objectives 17. Judged by the extent to which the project’s development objectives are consistent with the CPF at closing, their relevance is High. Increasing access to legal aid services (information, counseling and legal representation) for poor Jordanians, particularly women, and refugees in host communities remains critical for addressing challenges related to governance, gender and youth, which remain cross-cutting themes of the CPF FY17-FY22. These objectives also remain integral part of the CPF’s second pillar aiming to address growing inequalities in provision and an erosion of quality of public services, exacerbated by the Syria crisis. The project aimed not only at enhancing access to legal aid per se, but through this and the awareness campaigns it aimed at achieving more equitable access for refugees and poor Jordanians, particularly women, to other government services. Moreover, while supporting youth was not an explicit objective of the project, they were a target group of the awareness campaigns and in particular the child rights campaign. They also benefited directly from project support to women, for example through legal aid in child alimony cases. B Achievement of the project development objectives (efficacy) 18. Judged by the extent to which the operation’s objectives were achieved and that achievement is attributable to the activities supported by the operation, the efficacy is Substantial. Overall, the right indicators were used throughout implementation to measure the achievement of expected outcomes, although not all of them were identified as PDO level indicators. The project’s objective to increase access to legal aid services for poor Jordanians, particularly women, and refugees had several dimensions. First increased access to legal aid services would have had to result in an increase in services provided. A PDO level indicator therefore measured the number of legal counseling and representation services provided. The data was disaggregated by gender and nationality of the beneficiaries as well as by type of subject matter. Second, increased awareness among beneficiaries about legal rights was a prerequisite for an increased number of people seeking to enforce their rights. A PDO level indicator therefore measured the number of beneficiaries that report an increased knowledge following awareness sessions. However, while not reflected as a PDO level indicator, the project also measured the number of awareness sessions and participants, disaggregated by gender and nationality among others to adequately capture this dimension. Third, an adequate response to the expected increased demand for legal aid services required increased effectiveness of service providers. A PDO level indicator therefore measured the increased effectiveness of beneficiary referral mechanisms by measuring the increase in the number of referrals by public authorities to JCLA as well as the number of trainings provided to Government of Jordan (GoJ) partners. Moreover, an intermediate results indicator measured increased capacity of the Justice Center for Legal Aid through a number of enhanced systems and policies in place at JCLA. 19. When it comes to the increase in the provision of legal aid services, the project overachieved its (revised) target. Overall, JCLA delivered 7,615 legal services (4,539 consultations and 3,076 representations) against a revised target value of 7,500. The initial target value of 6,600 over three years was increased following a budget reallocation at the end of 2017 to prioritize service delivery. The majority of beneficiaries were women (71.6%). In terms of nationality, 79.8% were Jordanian, followed by Syrian (14.2%), which is roughly in line with Jordan’s demographics. Assuming that the Jordanian part of the population overall is better off, this results in a slight imbalance among beneficiaries in favor of poor Jordanians compared to refugees (for a detailed break-down see table below). Nationality Number Percentage Male Female Percentage Percentage Male Female Jordanian 6075 79.8% 1572 4503 25.9% 74.1% Syrian 1079 14.2% 421 658 39.0% 61.0 Palestinian 280 3.7% 98 182 35.0% 65.0% Other 181 2.4% 70 111 38.7% 61.3% Total 7615 100% 2161 5454 28.4% 71.6% Page 6 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) 20. In terms of subject matters, legal aid services focused on Sharia law (63.6% of cases), where the most common topics were divorce, alimony and paternity ratification. Consultations and representation in criminal cases focused on theft. A particular form of legal aid service delivery in criminal cases was the detention hotline available 24/7 for detainees in pre-trial detention. Until project closing it had more than 300 beneficiaries. In civil cases, consultations focused on financial and labor claims, whereas representation focused on birth certificate lawsuits. In administrative matters, legal aid services focused on documentation and administrative detention. For an overview see table below. Jurisdiction and Subject Matter Percentage Male Female Percentage Percentage Male Female Sharia Sharia 63.6% 397 4435 8.22% 91.78% Regular Civil 11.5% 501 371 57.45% 42.55% Criminal 20.4% 1081 465 69.92% 30.08% Administrative 4.0% 166 137 54.79% 45.21% Christian family law 0.6% 8 35 18.60% 81.40% Total 100% 2153 5443 28.34% 71.66% 21. When it comes to the geographic distribution of legal aid services across Jordan, the Central region, which includes the capital Amman, had the biggest share (73% of consultations, 77% of representations), followed by the Northern region (15% consultations, 11% representations) and the Southern region (13% consultations, 12% representations). For a detailed break-down by location see table below. Region Governorate Consultations Consultations Representations Representations Total Grand Total Total Grand Total Central Amman 2587 3292 1551 2382 Zarqa 360 384 Rusaifa 48 41 Al-Balqa 200 319 Madaba 97 87 North Irbid 400 672 152 336 Mafraq 66 16 Jarash / Ajloun 206 168 South Karak 167 575 88 358 Tafelah 294 222 Aqaba 54 17 Maan 60 31 Total 4539 3076 22. The impact of these legal aid services was significant. 90% of respondents reported exercising their rights following a consultation service provided by JCLA and 80 % of surveyed respondents reported they had solved their legal issue as a result of the representation service.1 This indicates a development impact beyond the PDO, which focused on access to legal aid, but did not explicitly target outcomes beyond that. 23. When it comes to increased awareness, the project overachieved its targets. Altogether, JCLA reached 32,327 beneficiaries through 1,151 awareness sessions against a revised target of 1,000 sessions. At the end of 2017 the target was increased from initially 750. In terms of content, JCLA developed 10 awareness programs, encompassing 25 different topics relevant for poor Jordanians (especially women, children and juveniles), and refugees: legal documentation, personal status law, protection of physical integrity, labor, tenancy law, disability law, debt, cybercrimes, child protection and access to justice. The impact of awareness sessions was measured through phone surveys and 99.7% of beneficiaries 1 Based on phone surveys carried out by JCLA, for closed cases only. Page 7 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) reported increase of knowledge following the session, which overachieves the target of 80% set for year three. Among the beneficiaries, 74% were female. In terms of nationality, 79.7% were Jordanian, resulting again in a slight imbalance among beneficiaries in favor of poor Jordanians compared to refugees, see table below. Awareness session data Awareness sessions 1,151 Participants (total) 32,327 Participant characteristics Number Percentage Gender Male 8,394 26.0 Female 23,933 74.0 Nationality Jordanian 25,770 79.7 Syrian 3,310 10.2 Palestinian 2,472 7.6 Iraqi 363 1.1 Other 412 1.3 24. When it comes to age, the awareness sessions targeted all age groups, including children and juveniles. Beneficiaries under the age of 18 represented 43% in addition to 22% of beneficiaries in mixed age groups, see table below. Age Group Percentage of beneficiaries of awareness sessions Under 12 years 22% 12 to 17 years 21% 18-29 years 10% Above 30 25% Mixed age groups 22% 25. Beyond the above numbers about awareness sessions, awareness campaigns with the general public had a significant outreach, especially through social media. They touched upon gender-based violence, children’s rights, anti- human trafficking, cybercrimes, debt, and pre-trial rights. The latter campaign reached 9,000 people on Facebook and the Retweet Reach on Twitter was 2.28 million people. The post launching the debt campaign reached 6,000 people on Facebook and over 24,800 impressions on Twitter. 26. Advocacy with Government also had a significant impact on legal aid in Jordan. JCLA outreach to justice stakeholders such as judges and other practitioners created an environment supportive of legal aid. Following a sustained JCLA campaign, the Criminal Procedures Law was amended in February 2018 to increase the coverage of the right to free legal counsel. In November 2018, the Government finally issued a bylaw to institutionalize a public legal aid system, establishing a Legal Aid Unit within the Ministry of Justice. This means that there is now a basis for public legal aid services to be provided in coordination with those provided by NGOs. 27. When it comes to increased effectiveness of service provision, the project partially achieved its targets. The project indicators aimed at an increase by 25% of GoJ partner referrals and delivery of at least 20 trainings to GoJ partners making referrals. The first indicator was substantially achieved. In 2016, the percentage of referrals of the overall JCLA caseload was of an estimated 3.6% (around 180 cases against 5,043 services). An improved referral system was established in January 2017 to better track and facilitate them. In 2017, the referrals represent about 5.3% (320 cases against 6,090 services), an increase of 77.8%. In 2018 including the first quarter of 2019, the referrals represent about 3% (274 cases against 9,339 services). Prorated for the period of one year (equivalent to 219.2 cases), the increase is about 21.8%. Roughly a year and a half into implementation, the actual value was three times higher than the end target after year 3. The subsequent decrease to just under the target value is due to two reasons. First, the MoSD chose to test the Page 8 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) Juvenile Law by referring cases directly to the courts (courts are obliged by law to appoint a lawyer for juveniles). Second, following the 2018 Legal Aid Bylaw, which does not mention civil society organizations as legal aid providers, GoJ referral partners preferred to test the bylaw by referring cases to the Bar Association. The training target was not achieved. JCLA organized 8 trainings instead of the expected 20. JCLA trained 151 social workers of the MoSD through five workshops on referral mechanisms and legal aid. In partnership with PSD, JCLA held two in-depth trainings on restorative justice under the juvenile law for police officers dealing with juveniles. JCLA also organized a training targeting the PSD women’s Police Unit. Training of MOJ staff was not possible due to a lack of a functioning working relationship and absence of a Legal Aid Unit at the MOJ until late in the project cycle. 28. However, JCLA carried out activities that most probably affected effectiveness of service delivery positively, but whose impact was not captured by these indicators. JCLA capacity building activities such as continuous training of JCLA staff and enhanced policies and systems within JCLA are among those activities. There is an intermediate results indicator measuring the number of enhanced systems and policies within JCLA, evidenced by the number of approved policies and implementation of internal management IT solutions. The cumulative target of 10 was by far overachieved with 26 policies approved and two IT solutions implemented. Among the latter a case management system was newly developed. It mirrors the Jordanian online court system and makes it compatible. After establishment of the MOJ Legal Aid Unit, JCLA shared the source code and provided the MOJ with its eligibility criteria and intake forms, which will be adopted for the MOJ’s own procedures. This enabled JCLA to resume direct cooperation with the MOJ, which had been suspended for two years. As a result, JCLA was tasked with the training of MOJ liaison officers whose task is to receive legal aid applications in court. 29. The efficacy rating as Substantial is the result of overachievements on two outcome dimensions (provision of legal aid services and awareness) and partial achievement on the third outcome dimension of effectiveness of service provision. The intermediate results indicator is also overachieved. However, these overachievements do not fully justify an efficacy rating as high, because the capacity building of Government partners was an important contribution to establishing a system where Government institutions and JCLA would be partners in delivering legal aid, based on a legal aid system where both would complement each other and work together. It is true that it is not attributable to the project that the legal aid system was established late in the project and that it does not recognize civil society organizations as legal aid providers under the publicly funded system. It is also true that the relationship with the MOJ made it much harder, and at times impossible, for JCLA to provide them with training, especially at a time when the Legal Aid Unit was not yet set up. However, the targets could have been revised accordingly or the targeted amount of training could have meaningfully been provided to other GoJ referral partners. C Efficiency in achieving the project development objectives 30. Judged by the extent to which the project has achieved its development objective efficiently, the rating is substantial. The project has proved remarkably cost effective. Although the indirect social and economic benefits cannot be monetized, they exceed by far the direct ones which should be expected of a pilot. Its value added rests essentially in the proof of concept (i.e. the value of legal aid) and the induced institutionalization of legal aid, notwithstanding its present limitations. 31. The project has surpassed its objectives within its initial budget. Most of the project cost was earmarked and effectively incurred by the provision of legal aid services which is adequate since such services entail significant lawyers’ fees. But the unit cost of those services remains remarkably low at USD 205 on average, including overheads, i.e. USD 69 per legal consultation and USD 406 per legal representation. Most of the project budget was allocated to piloted legal aid services which benefitted fewer people than the advocacy and awareness components, but such a cost allocation should Page 9 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) not be deemed ineffective since the provided legal aid services were necessary to establish the proof of concept which allowed the successful implementation of the other two components. The procurement of IT systems for case management has met World Bank standards and should be deemed all the more cost effective since the Case Management System developed by the implementing agency is also benefitting the Legal aid Unit of the ministry of Justice through the sharing of its software for free and should be used on a much larger scale than the pilot project. The leveraging of social medias for the awareness campaign has also allowed the implementing agencies to inform and sensitize large swathes of its targeted beneficiaries (up to over 2 million through retweeting) at hardly any cost. Partnerships struck by the implementing agency with civil society organizations also allowed it to extend the reach of its awareness campaigns at low cost. 32. In terms of outcome efficiency, legal aid services provided under the project have generated more intangible than tangible benefits to targeted beneficiaries and consequently cannot be monetized. They have proved quite successful, since the case was solved in 80 % of cases among surveyed beneficiaries. But far more beneficiaries declare that their psychological welfare improved as a result (68%) than they report a financial return (18%). But monetary benefits were not the only tangible benefit of piloted legal aid services. For example. when they led to a court order revoking the deportation of a Syrian refugee (which was without precedent in Jordan), the life of the beneficiary was arguably at stake. Besides, although financial return has not been quantified, it had to be limited in absolute terms since the project deliberately excluded potentially high return legal cases such as purely financial claims. 33. The social distribution of direct beneficiaries looks both positively and negatively skewed. Most piloted legal aid services benefitted women (72%), i.e. beyond initially envisaged. However, the spatial distribution of the piloted legal services is somewhat distorted, with the central region over-represented in both legal consultations and representations (respectively 73 and 77%), taking into account its share of the overall population and of Syrian refugees (respectively around 62 and 25%) as well as the spatial concentration of poverty (in the North and South regions). The project did not initially consider the spatial distribution of its footprint but could have done so to maximize its effectiveness. 34. The benefits of legal aid also went beyond the benefits to direct beneficiaries. Successfully piloted legal aid services demonstrated the value of legal aid to promote access to justice for the most destitute and vulnerable. They were the necessary proof of concept which facilitated the institutionalization of legal aid in Jordan. Such institutionalization through the adoption of a legal and institutional framework for legal aid (through the enactment of a Legal aid bylaw and the establishment of a Legal aid unit in the MOJ). Despite its limitations, the legal recognition of legal aid services should be deemed a breakthrough in a country where it was not officially provided until now and is still resisted by vested interests. The strengthening of child protection through the revision of the Juvenile Law and related bylaws can also be credited to the advocacy impact of the project. These legal reforms are the most systemic and sustainable benefit of the project. It may also be expected that far reaching awareness campaigns will progressively trigger more demand for legal aid services. Overall Outcome Rating 35. The overall outcome of the project is rated Moderately Satisfactory due to the High relevance, Substantial efficacy, and Substantial efficiency. There were no shortcomings in the operation’s relevance and the shortcomings in efficiency were minor. The shortcomings in efficacy were moderate. Had the training goal been slightly underachieved or substantially achieved, the shortcomings in efficacy could have been considered as minor and the overall rating could have been Satisfactory. But since the project was built on the complementarity of legal aid services provided by CSOs in Page 10 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) cooperation with public authorities whose capacity would be strengthened, the shortcomings in efficacy are assessed to be moderate, resulting in an overall rating as Moderately Satisfactory. Other Outcomes and Impacts 36. Beyond the expansion of access to legal aid services targeted by the PDO, the project has contributed to poor Jordanians (particularly women) and refugees effectively seeing their rights enforced. In a survey among beneficiaries2, 90% of respondents who had benefited from consultation services reported exercising their rights following the consultation, which is very high. Given the high percentage of female beneficiaries of legal consultations (68.0%), there is a tangible gender impact. In the same survey, 80% of respondents who had received representation services reported they had solved their legal issue as a result. 68% reported that their mental health improved as a result of solving their legal issue and 18% stated that their financial situation improved. Given the even higher percentage of female beneficiaries of representation services (76.9%), the impact on gender is significant. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME 37. A key factor for the success of this project was the strong performance of the implementing agency. Reporting was always timely and adequate and engagement with the World Bank focused on achieving the development objectives of the project. JCLA has been able to manage its resources effectively and strategically. It has grown and matured. The number of services provided has increased, while a significant number of systems, policies and procedures have been established to increase effectiveness of service delivery as well as internal processes, e.g. it established “legal teams� to work on cases instead of individual lawyers to capitalize on complementary skill-sets and to allow for learning of more junior lawyers. This led to an increase in the success rate of JCLA cases of 92,8% in 2018. At the same time, JCLA reduced the number of its legal clinics, compacting them in strategic locations and hosted by highly active referral partners. Leveraging a smaller number of clinics and lawyers through this reorganization, JCLA was able to provide 1,350 legal services more in 2018 than in 2017. This was possible due to JCLA’s ability to establish a strong network of partners that has been crucial to access beneficiaries through referrals and awareness activities. It also allowed to build on each other’s strengths through respective specialization. Reporting to and engagement with the World Bank was 38. The project would have benefited from greater GoJ commitment to legal aid: While the Project Paper already mentions plans in 2016 for the establishment of a legal aid unit at the MOJ and efforts of the MOJ to establish a public- private partnership with civil society organizations and the Jordan Bar Association around the delivery of legal aid to all vulnerable people in Jordan, progress was slower and less than anticipated. It took until February 2018 for the Criminal Procedures Code to be amended to grant the right to free legal counsel for defendants whose alleged crimes are punishable by 10 years or more. While this amendment is important, it does not guarantee access to legal aid beyond that, thus excluding Sharia, civil and administrative cases as well as criminal cases that are punishable by less than 10 years of imprisonment or fines. This remains well short of what JCLA had advocated for, including with sustained support of this project, and means the continued reliance for the foreseeable future on civil society organizations to fill this gap and provide legal aid to the poor and vulnerable. Moreover, the Legal Aid Bylaw, was issued in November 2018. JCLA was not included in the drafting process and the text did not incorporate any JCLA inputs. As a consequence, the eligibility criteria for applicants remain vague and there are no clear mechanisms yet to meet the demand. Importantly, the Bylaw only 2 Based on phone surveys carried out by JCLA, for closed cases only. Page 11 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) mandates lawyers from the Jordan Bar Association (JBA) as public legal aid providers, thus excluding civil society legal aid providers from the newly established system, which delivered pro bono legal aid in 280 cases in 2018. 39. Government willingness to engage with the Project was uneven: At the design stage, the level of risk related to the impact of Government changes was underestimated, while the level of commitment of the MOJ itself was overestimated. At certain times during implementation the relationship with the MOJ became antagonistic. As a consequence, JCLA requested a budget reallocation at the end of 2017 to shift its focus from activities with the MOJ to service delivery activities with other government bodies that were eager to cooperate, such as the MoSD and the Public Security Department (PSD), resulting in a strong relationship with them. Towards the end of the project the relationship with the MOJ improved and JCLA was able to reengage accordingly. 40. The conflictual relationship between the Jordan Bar Association and civil society legal aid providers remains an obstacle to creating synergies: The effective coordination between civil society legal aid providers and the Jordan Bar Association as the lone public legal aid provider based on the 2018 Legal Aid Bylaw would greatly contribute to a coherent approach to legal aid in Jordan ensuring that all needs of poor and vulnerable segments of the population are met. The Jordan Bar Association considers that civil society organizations such as JCLA provide legal aid services illegally, negatively affecting their members’ livelihoods. This currently conflictual relationship prevents both sides from engaging in a constructive and meaningful dialogue about the future of legal aid in Jordan. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME 41. Judged by the extent to which services provided by the Bank ensured the operation’s quality at entry and supported effective implementation through appropriate supervision, Bank performance is rated as moderately satisfactory. When it comes to quality at entry, the project design was solid from a technical and financial perspective. Based on previous World Bank engagement with the implementing agency in this area, the analytical basis was sound and project design appropriate. The activity was and remained of high strategic relevance focusing on World Bank priorities in terms of poverty and gender. In hindsight, the assessment of the commitment by the Government turned out somewhat optimistic, while the risk of impact of changes in Government on project activities was somewhat underestimated. The implementation arrangements were strong with a highly performing implementing agency. The M&E framework overall was adequate, with minor weaknesses in details such as the errors in formulation of cumulative targets for PDO indicators 1 and 3 and frequency of measurement for PDO indicator 3. When it comes to quality of supervision, throughout the project the focus was on implementation support and achieving the development impact jointly with the implementing agency and relevant stakeholders. Budget reallocations were carried out to adjust to changing circumstances such as evolving MOJ commitment. Supervision of fiduciary aspects was adequate. Reporting left room for improvement. The first and only Implementation Status and Results report (ISR) is dated January 3, 2018, roughly 18 months into implementation, which is late. The data used to report on progress against the indicators was 3-6 months old and absent for one outcome indicator. While the narrative provided useful information, M&E data could have been explained in the comment section. A Grant Reporting and Monitoring Report (GRM) was provided for the period from 06/01/2017 to 06/30/2018 at the end of 2018, until GRM reporting was waived. A shortcoming was that the mid-term review took place rather late in January 2018, when 80% of the grant proceeds had already been disbursed. However, this did not negatively impact implementation and was used as a hand-over exercise from one TTL to another. Page 12 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) V. LESSONS LEARNED AND RECOMMENDATIONS 42. When Government commitment is uncertain and uneven, working with more than one Government agency combined with flexibility during implementation may allow to focus on working with parts of Government that are sufficiently committed and relevant to achieve development objectives. Government commitment (or lack thereof) is not necessarily monolithic. Had the commitment of the Ministry of Justice been stronger under this project, the implementing NGO would have had the potential for an even bigger impact. However, flexibility during implementation has enabled the project to refocus certain activities on other Government agencies that were sufficiently relevant and where commitment was sufficiently strong to achieve project objectives. In environments with uneven Government commitment it may therefore be advisable to have an engagement with more than just one Government agency to minimize the risk of lacking commitment of one agency to disrupt the project. 43. The establishment of networks and partnerships creates important synergies. These synergies have a significant impact on the quality of service delivery of legal aid, but also on efficiency of resource utilization and sustainability, for example they allow for hosting of legal aid services by partner organizations. This has the additional benefit of a broader range of services being available and therefore an increased ease of referral. Notably it has the benefit of improved access for women to legal aid. For cultural reasons, it is still fairly difficult for women to openly and individually seek legal aid, but if the legal aid provider is hosted in the back office of a community-based partner organization with multiple service providers the women are less exposed. If these partnerships include civil society organizations as well as Government partners, they can further contribute to building trust between stakeholders, minimizing antagonism and thus mitigate conflicts between civil society and Government stakeholders. 44. Social media are a powerful tool to enhance outreach of legal awareness campaigns. Leveraging social media and networking for awareness campaigns proves highly cost and result effective and should be operationalized further to promote access to justice and other public services. 45. Monetary returns for direct beneficiaries of legal aid services should have been measured. Data to assess monetary returns, especially in relative terms (i.e. in percentage of beneficiaries’ income), would have been useful to quantify and substantiate the project’s value for money. This is an important aspect of project impact. 46. During project design it is important to map the spatial distribution of target beneficiaries. The project design could have better targeted Syrian and other refugees as well as host communities by more systematically factoring in their spatial distribution across the country to deploy legal aid services capacity accordingly. 47. Sustainability of donor funded NGO activities remains a challenge. While the implementing agency has taken a number of steps to address sustainability challenges (outreach to Government to create a system where there is a role for various actors, including NGOs, in the delivery of legal aid, establishment of partnerships with other NGOs, specialization, visibility with other donors, and enhanced internal capacity and processes), the NGO will further rely on donor funding, because the Government has so far opted for a system limiting public funding for legal aid to cover a fraction of the existing need and that will be delivered by the Bar Association, while the vast majority of cases will still have to be shouldered by NGOs. 48. Since it is yet partial, the institutionalization of legal aid should be supported further through World Bank operations, including budget support (as a policy action) and social protection programs. Follow up World Bank engagement on the matter, including in policy dialogue, seems called for to promote both the sustainability and mainstreaming of achieved results. . Page 13 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Increase in the provision of legal aid services Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Legal services are enhanced Number 2474.00 6600.00 7500.00 7615.00 through increased counseling, representation and ADR 09-Mar-2016 09-Mar-2016 31-Dec-2017 15-Mar-2019 services to target communities Comments (achievements against targets): Overall, JCLA delivered 7,615 legal services (4,539 consultations and 3,076 representations) against a revised target value of 7,500. The initial target value of 6,600 over three years was increased following a budget reallocation at the end of 2017 to prioritize service delivery. Objective/Outcome: Increased awareness among beneficiaries about legal rights Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Beneficiaries have increased Percentage 0.00 80.00 99.70 awareness about legal rights and procedures for enforcing 09-Mar-2016 09-Mar-2016 15-Mar-2019 Page 14 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) them Comments (achievements against targets): The impact of awareness sessions was measured through phone surveys to assess the percentage of beneficiaries of awareness campaigns whose knowledge increased through the session. 99.7% of beneficiaries reported increase of knowledge following the session, which overachieves the target of 80% set for year three. An additional PDO indicator not listed here was the number of awareness sessions with a cumulative target of 1000 for year three. JCLA reached 32,327 beneficiaries through 1,151 awareness sessions against a revised target of 1,000 sessions. At the end of 2017 the target was increased from initially 750. Objective/Outcome: Increased effectiveness of service provision Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Increased effectiveness of Number 0.00 20.00 8.00 beneficiary referral mechanisms involving new 09-Mar-2016 09-Mar-2016 19-Jan-2018 partners Comments (achievements against targets): This indicator set a target for number of trainings provided to Government partners. This target was not achieved. JCLA organized 8 trainings instead of the expected 20. JCLA trained 151 social workers of the MoSD through five workshops on referral mechanisms and legal aid. In partnership with PSD, JCLA held two in-depth trainings on restorative justice under the juvenile law for police officers dealing with juveniles. JCLA also organized a training targeting the PSD women’s Police Unit. A second indicator not listed here was measuring the in crease in the number of referrals by Government partners. The target was an increase of 25% by the end of year three. The share of referrals among the overall JCLA caseload was of an estimated 3.6% (around 180 cases against 5,043 services). An improved referral system was established in January 2017 to better track and facilitate them. In 2017, the referrals represent about 5.3% (320 cases against 6,090 services), an increase in the number of cases of 77.8% against the absolute baseline number. In 2018 including the first quarter of 2019, the referrals represent about Page 15 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) 3% (274 cases against 9,339 services). Prorated for the period of one year (equivalent to 219.2 cases), the increase is about 21.8%, just under the target value. However, roughly a year and a half into implementation, the actual value was three times higher than the end target after year 3. A.2 Intermediate Results Indicators Component: Component 3: Capacity Building Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Enhanced system for effective Yes/No N Y Y case management 09-Mar-2016 09-Mar-2016 18-Jan-2018 Comments (achievements against targets): The enhanced case management system was successfully established. An additional aspect measured and not listed here was the number of enhanced policies adopted by JCLA. The way it was formulated was not very sound, because it lumped together the number of enhanced systems (including the case-management system) and enhanced policies. It was formulated as measuring the number of enhanced systems and policies within JCLA, evidenced by the number of approved policies and implementation of internal management IT solutions. The cumulative target of 10 was by far overachieved with 26 policies approved and two IT solutions implemented. Page 16 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) A. ORGANIZATION OF THE ASSESSMENT OF THE PDO Objective/Outcome 1: Increased provision of free legal aid for poor Jordanians, particularly women, and refugees in host communities Outcome Indicators Number of legal counseling and representation services provided. 1. 7,615 legal services (4,539 consultations and 3,076 Key Outputs by Component representations). (linked to the achievement of the Objective/Outcome 1) 2. Detention hotline with more than 300 beneficiaries in detention. Objective/Outcome 2: Increased awareness among beneficiaries about legal rights and procedures for enforcing them Percentage of beneficiaries that report an increased knowledge Outcome Indicators following awareness sessions. 1. 1,151 awareness sessions with 32,327 beneficiaries. 2. 10 awareness programs designed, encompassing 25 different topics. 3. Outreach through media and communication staff (social media pages, launch of online campaigns). Key Outputs by Component 4. Stakeholder advocacy activities, including a legal aid campaign, nine (linked to the achievement of the Objective/Outcome 2) activities such as workshops and trainings targeting 182 participants (e.g. lawyers, judges, MOJ employees and local CSOs). 5. Desk reviews and policy papers. 6. Draft memoranda and bylaws. 7. Phone survey on impact of awareness sessions. 8. 14,742 awareness brochures and booklets printed. Objective/Outcome 3: Increased effectiveness of provision of legal aid services to poor Jordanians, particularly women, and refugees in host communities 1. Percentage increase in the number of referrals. Outcome Indicators 2. Number of trainings delivered to GoJ partners making referrals. Page 17 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) Intermediate Results Indicators Number of enhanced systems and policies within JCLA. 1. Five workshops on referral mechanisms and legal aid for 151 social workers of the MoSD. 2. Two in-depth trainings on restorative justice under the juvenile for police officers dealing with juveniles. 3. Training targeting PSD women’s Police Unit. 4. 7 trainings for JCLA lawyers. 5. 1 training for JCLA communication officer. 6. Advocacy strategy. 7. JCLA transportation compensation policy. 8. JCLA Code of Conduct. 9. JCLA employees performance evaluation policy. 10. JCLA apprentices policy. 11. JCLA interns policy. 12. JCLA working hours policy. Key Outputs by Component 13. JCLA internal communication policy. (linked to the achievement of the Objective/Outcome 3) 14. JCLA revised procurement policy. 15. JCLA revised financial policy. 16. JCLA revised internal bylaw. 17. JCLA community facilitator selection and recruitment policy. 18. JCLA case management system. 19. JCLA data protection policy. 20. JCLA eligibility criteria policy. 21. JCLA intake forms. 22. JCLA referral forms. 23. JCLA community facilitator policy. 24. JCLA community facilitator training manual. 25. JCLA detention hotline policy. 26. JCLA duty pawyers policy and compensation scheme. 27. JCLA hotline officers policy. 28. JCLA internal quality control policy. Page 18 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) 29. JCLA legal teas policy and procedures. 30. JCLA success stories documentation policy. 31. JCLA complaints response feedback mechanism policy. 32. JCLA M&E policy. 33. JCLA evaluation procedures. 34. Two volunteer programs. 35. Quarterly review reports. 36. Multi-stakeholder ICR workshop. Page 19 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) . ANNEX 2. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (%) Component 1: Enhancing Legal Services to 1.26 1.56 124.8 Communities Impacted by the Refugee Crisis Component 2: Increasing Legal Awareness and 0.57 0.50 87.7 Information Component 3: Capacity 0.47 0.11 23.4 Building Component 4: Project Management and Monitoring 0.27 0.32 118.5 and Evaluation Total 2.50 2.50 100.00 The following table provides a more detailed break-down of the costs: OUTPUT RESOURCES COST IN USD DETAILS Component 1: Enhancing Legal Services to Communities Impacted by the Refugee Crisis Provision of 7,615 Legal Service provision cost 1,098,683.00 The cost includes the Aid Services, salaries of lawyers and specifically 4,539 apprentices, case fees, consultations and case administrative 3,076 representations costs, communication costs. Legal Services Quality 220,031.00 The cost includes Control salaries of Legal Services Director, as well as Legal Services Supervisor and Quality Assurance staff, responsible for cases quality control processes, of cases, supervision and guidance of lawyers, receiving and handling complaints. Administrative support 152,875.00 The cost includes salaries of administrative Page 20 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) assistants, responsible for taking beneficiaries' appointments and distributing work to lawyers, as well as compiling monthly reports. It also includes the salary of data entry staff, tasked with entering cases' information on the online system. It also includes office assistants, consultants and intake officers who offer support to the Legal Services Department Office supplies, 29,559.00 The cost includes furniture and stationary, office equipment equipment such as desks, as well as IT devices such as printers, laptops, desktop, hard drives, and IT maintenance, for new lawyers and clinics, as well as administrative staff Rent, Utilities, 61,580.00 Transportation and Office Maintenance Subtotal 1,562,728.00 Component 2: Increasing Legal Advocacy and Awareness Provision of 1,151 Direct activity expenses 238,704.00 The cost includes awareness sessions administrative expenses (such as refreshments), salaries of awareness coordinators, outreach coordinators, speakers Awareness Department 72,614.00 The cost includes the Management salaries of Awareness Director and Awareness Page 21 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) Administrative Assistant Development of Consultant 15,232.00 The cost covers legal awareness materials consultants who provided assistance in developing awareness materials. Brochures 14,418.00 JCLA printed 14,742 brochures and booklets on awareness topics. Outreach Media and 46,092.00 The cost includes fees Communication staff of Media Consultant company, responsible for JCLA’s branding and PR. It also includes the Communication Officer’s salary, responsible to manage JCLA's Social Media pages, launch online campaigns, etc. Capacity Building and Transportation for 628.50 JCLA offers Advocacy for Legal Aid University Students transportation remuneration for its volunteers. Driver 10,557.00 Advocacy Management 104,792.00 The cost covers salaries of Advocacy Director, Assistant and Researcher. They are responsible for organizing stakeholder advocacy activities, conducting desk reviews, issuing policy papers, drafting memoranda and bylaws. Subtotal 503,037.50 Component 3: Capacity Building Building Government Workshops 32,610.00 The cost includes Capacity venue, catering, per diem, transportation and trainers for 14 workshops, targeting Page 22 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) government employees, including MoSD, MoJ, judges, PSD, as well as local CSOs. Consultant 18,377.00 The cost includes expenses for consultants assisting JCLA with development of advocacy strategy. Publications 46.00 Building JCLA Capacity Staff Continuous 8,401.00 The cost covers 7 Training trainings targeting JCLA lawyers, 1 training targeting JCLA communication officer Institutional 50,493.00 The consultant was Development responsible with Consultant supporting JCLA in developing policies and revising SoPs. Subtotal 109,927.00 Component 4: Project Management and Monitoring & Evaluation Ensuring effective Finance 122,414.00 implementation of Procurement 56,128.00 activities and Human Resources 18,559.00 operational support Audit 18,023.00 Other Administrative 12,869.00 Costs Project Manager 94,266.00 Subtotal 322,259.00 TOTAL 2,497,951.50 Page 23 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) ANNEX 3. ICR STAKEHOLDER WORKSHOP MINUTES On April 22, 2019, JCLA held a joint ICR stakeholder workshop where feedback was provided on project activities, achievements, challenges and lessons learned. The agenda was as follows: 1. Presentation of Project activities and achievements; 2. Discussion with stakeholders of project achievements, lessons learned and sustainability; and 3. AOB. A total of 25 participants took part in the meeting. They represented the Public Security Directorate (Family Protection Department, Juvenile Police Department, Transparency and Human Rights Department), the Ministry of Social Development, the Ministry of Justice, the Jordan River Foundation, the Jordanian Women's Union, Ruwwad, the National Council for Family Affairs, the Center for Women's Studies/University of Jordan, the UN Refugee Agency (UNHCR), the Norwegian Refugee Council (NRC), the World Bank, and the JCLA team. Part 1: In the first part, JCLA Executive Director Ms. Hadeel Abdelaziz gave a small presentation on JCLA’s legal empowerment program with a specific focus on its achievements under this project. The main shared points were as follows: • An overview of the total beneficiaries reached throughout this project; in legal awareness activities and legal services (legal representations and consultations). • A quick summary on JCLA’s strategy and efforts in building and strengthening strategic partnerships and in boosting its advocacy efforts. • The study that was conducted in collaboration with the World Bank on the need of legal aid and the revealing results of how almost 68% of people in Jordan go to court unrepresented every year. • The study that was carried out in collaboration with the Department of Statistics in which 10,000 Jordanian households were surveyed that revealed the percentage of people who refrain from going to court due to distrust in court procedures as well as lack of financial means to hire a lawyer when faced with a legal problem. Part 2: Ms. Hadeel Abdelaziz used various questions for dialogue facilitation in the following areas: 1. Assessment of the impact of project activities 2. Lessons learned 3. Sustainability When it comes to an assessment of the impact of project activities, the Juvenile Police Department confirmed that previously available resources were reallocated to deal with the impact of the Syrian refugee crisis. The project has encouraged the establishment of strong Page 24 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) partnerships. JCLA was seen having been key for building their partners’ capacity through conducting specialized trainings, sharing expertise and producing studies. The National Council for Family Affairs acknowledged JCLA’s efforts in organizing and delivering capacity building workshops and trainings as well as awareness activities. They also appreciated J CLA’s active participation in specialized committees. The Jordan River Foundation found that JCLA’s legal services had empowered women beneficiaries and helped them to enforce their rights. The Family Protection Department found that JCLA’s efforts were ultimately reflected in a discussion paper launched by the King. UNHCR found JCLA activities complementary and emphasized their contribution to setting a precedent in deportation cases and obtaining exemptions in documentation cases. The Jordanian Women’s Union pleaded for JCLA to revise their eligibility criteria to include monetary claims. The MoSD saw its capacity increased due to the project. They also appreciated JCLA’s contributions to committee work. On sustainability and lessons learned, some participants felt that legal aid and legal awareness requires an institutionalized approach beyond just one project. A change of mindset is required, as the Juvenile Police representative mentioned. The NRC highlighted the need for a long term dialogue approach with the Ministry of Justice and CSOs as well as continuation of long term efforts to have a well-functioning public legal aid system. There was acknowledgement that social media could play an even bigger role in the future, especially given the need to reach a bigger number of people at low cost. There should be further improvements to the way project impact is measured, including the financial impact, and a stronger focus on measuring more systematically to what extent beneficiaries realized their rights. JCLA found that the MoJ and the JBA should be more active and responsive in providing legal aid services in order to reach sustainability through the establishment of well-functioning public legal aid system with adequate quality control and shared responsibilities between the MOJ, the JBA and CSOs. They recommended the development of a comprehensive and clear fee structure for legal aid cases. Page 25 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) ANNEX 4. RECIPIENT COMPLETION REPORT SUMMARY AND COMMENTS The completion report submitted by JCLA is structured as follows: (1) Project design, (2) Indicators, (3) Assessment of outputs, (4) Assessment of governmental agencies’ performance and cooperation, (5) Assessment of JCLA’s performance, (6) Value for money: what the project has bought, (7) Lessons learned, (8) Bank performance, and (9) Sustainability. (1) In terms of project design, the project was structed based on JCLA’s comparative advantage, i. e. legal services, awareness and advocacy. So the project was ready for implementation at the onset. (2) When it comes to indicators, the completion report presents the three outcome indicators as well the intermediate results indicator and provides data to substantiate the achievements and shortcomings. Outcome indicator one (legal services are enhanced through increased counseling, representation and ADR services to target communities, evidenced by number of legal counseling and representation services) has been overachieved with 7,615 legal aid services against a target of 7,000 services. Outcome indicator two (beneficiaries have increased awareness about legal rights and procedures for enforcing them, evidenced by number and percentage of beneficiaries that report an increased knowledge following awareness sessions) was also overachieved with 99.7% of respondents to a survey reporting that the information they learnt in the awareness session was useful and increased their knowledge, against a target of 80%. The extent of achievement of outcome indicator three (increased effectiveness of beneficiary referral mechanisms involving new partners, evidenced by percentage in crease in referrals) is not clear according to the completion report. The target was an increase of 25% of partner referrals and 20 trainings for Government on referral mechanisms. The number of trainings was not achieved. Instead of delivering 20 trainings, JCLA trained 151 MoSD social workers through 5 workshops on referral mechanisms and legal aid, according to the completion report. In terms of percentage of partner referrals, the completion report does not provide a baseline, so such an increase is not measured. Instead, the completion report mentions that referrals from 2017 to 2019 constitute 28% of the overall caseload. The intermediate results indicator (enhanced systems and policies within JCLA, evidenced by number of approved policies and implementation of internal management IT solutions) was achieved with 28 policies approved and IT solutions implemented. (3) Regarding the assessment of outputs, the completion report provides further detailed information about the activities and how they contributed to the realization of the Project Development Objective. The data is disaggregated by gender, location, type of service etc. The report provides additional information about the achievement of the outcome indicator three (capacity building) and lists the trainings provided under the project. The report also elaborates on the monitoring and evaluation efforts under the project. (4) In terms of assessment of governmental agencies’ performance and cooperation, the completion report acknowledges Jordan’s progress by passing amendments to the Criminal Procedure Code, adopting the Legal Aid Bylaw and establishing the Legal Aid Unit at the MOJ, while deploring the shrinking space for civil society. The completion report mentions a constructive albeit challenging relationship at times with the MoSD. The relationship with the MOJ was more conflictual and JCLA was not included in the drafting process for the Legal Aid Bylaw. The relationship with the JBA was antagonistic. These difficulties in the relationship with relevant Government stakeholders and the bar negatively impacted the project. (5) When it comes to the assessment of JCLA’s performance, the completion report mentions that JCLA has been Page 26 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) able to manage its resources effectively to reach the objectives of the project. The volume of services and the success rate of cases has increased. At the same time, JCLA has grown as an organization and significantly improved its approach to monitoring and evaluation, including surveys. It has built strong operational teams and a comprehensive network of partners, which will positively impact sustainability. It has improved ways of reaching clients, especially refugees, through engagement of Community Facilitators from refugee communities. (6) Value for money The completion report outlines JCLA efforts in terms of financial management and procurement. It then provides a detailed overview of the costs of project activities. (7) Lessons learned The completion report presents a number of lessons learned. It emphasizes the success of JCLA’s partnership model. It highlights the importance of nurturing and maintaining partners, as they are key to accessing beneficiaries through referrals and awareness activities. This allows for specialization and the creation of a holistic service system where beneficiaries are able to receive specialized high-quality services thanks to a diverse range of specialized service providers and a flexible and functional referral system. Another lesson revolves around the use of empirical data as a tool for management and learning. Data on needs is important to ensure those needs will be met. Using of data for management purposes also means that service providers can manage themselves more effectively. The management of these data requires sound processes and systems. A lesson with respect to legal awareness is that campaigns should not only spread knowledge, but focus on prevention of root causes that would otherwise initiate a cycle of crimes. Social media are a powerful tool to reach a high number of people. An important lesson on advocacy for change is that while it is important to advocate for changes in the law, outreach to officials in charge of implementing policies and the establishment of a sound working relationship with them are ways to achieve significant changes in the actual practice, albeit under the same legal framework. (8) Bank performance The completion report mentions several aspects of working with the World Bank. World Bank engagement on legal aid ensured that the topic was on the policy agenda. Other donors and ultimately the authorities engaged on it. While being flexible, the World Bank was also felt as imposing professional working standards, which was beneficial and challenging at the same time. The report mentions that awareness of this activity among other World Bank projects was too limited, which prevented JCLA from seeking synergies with other World Bank projects. (9) Sustainability Sustainability remains a challenge, although JCLA took important steps to address this challenge: JCLA has advocated for national legal aid system. While the established system falls short of what JCLA was advocating Page 27 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) for, it should alleviate the pressure on JCLA to deliver services in the area of criminal justice, once the new publicly funded system is fully functional. JCLA’s specialization and presence throughout the country improved their visibility, so other donors approach them for legal aid and legal awareness activities. The increased efficiency of operations within JCLA and mobilization of apprentices and volunteers complementing the work of the lawyers enables JCLA to make better use of its resources, thus increasing the likelihood of sustainability. Also, the investment in advocacy within local communities was useful to create demand for a functional legal aid system to be in place. Finally, there is a continued need for support and improvements to the newly established legal aid system in Jordan. Page 28 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) ANNEX 5. SUPPORTING DOCUMENTS (Under the respective headings the documents are arranged in chronological order) Strategy and Reference Documents: Country Partnership Strategy for the Hashemite Kingdom of Jordan for the Period FY12-FY15, Report No. 58114-JO (February 2012) Jordan Country Gender Assessment: Economic Participation, Agency and Access to Justice in Jordan, Report No. ACS5158 (July 2013) Jordan 2025, A National Vision and Strategy (March 2014) Hashemite Kingdom of Jordan, Promoting Poverty Reduction and Shared Prosperity, Systematic Country Diagnostic, Report No. 103433-JO (February 2016) Country Partnership Framework for the Hashemite Kingdom of Jordan for the Period FY17-FY22, Report No. 102746-JO (June 2016) Project Documents: Project Paper for a Small RETF Grant Requesting Funding from the State and Peace-Building Fund (US$2.5 million equivalent) to the Justice Center for Leal Aid of the Hashemite Kingdom of Jordan (March 2016) Letter Agreement, SPF Grant No.TF0A2524 Enhancing Delivery of Judicial Services to Poor Jordanians and Refugees in Host Communities Project (June 2016) Implementing Agency Documents: Quarterly Progress Report (June 2016-December 2016) Audit Report 2016 Quarterly Progress Report (January 2017-March 2017) Quarterly Progress Report (April 2017-June 2017) Quarterly Progress Report (July 2017-September 2017) Quarterly Progress Report (October 2017-December 2017) Audit Report 2017 Quarterly Progress Report (January 2018-March 2018) Quarterly Progress Report (April 2018-June 2018) Quarterly Progress Report (July 2018-September 2018) Quarterly Progress Report (October 2018-December 2018) Completion Report (March 1, 2016-March 31, 2019) ICR Stakeholder Workshop Minutes (April 2019) Audit Report 2018 World Bank Implementation Support and Reporting Documents: Mission Aide-Memoire (December 2016) Financial Management Supervision Report (February 2017) Page 29 of 30 The World Bank Piloting Delivery of Justice Sector Services to Poor Jordanians and Refugees in Host Communities (P157861) ISR No.1 (January 2018) GRM for the period from June 1, 2017-June 30, 2018 Mid-Term Review Report (January 2018) Procurement Post Review (May 2018) Financial Management Supervision Report (October 2018) Draft GRM for the period from July 1, 2018-March 31, 2019 (waived) ICR Mission Aide-Memoire (April 2019) Page 30 of 30