Document of The World Bank Report No: ICR2561 INTERNATIONAL DEVELOPMENT ASSOCIATION IMPLEMENTATION COMPLETION AND RESULTS REPORT (Grant No. H414-TP) ON A GRANT IN THE AMOUNT OF SDR 1.3 MILLION (US$ 2.1 MILLION EQUIVALENT) TO THE DEMOCRATIC REPUBLIC OF TIMOR-LESTE FOR A YOUTH DEVELOPMENT PROJECT 19 OF MARCH 2013 Social, Environment and Rural Development Unit Papua New Guinea, Pacific Islands, Timor-Leste Country Department East Asia and Pacific Region Vice President: Axel van Trotsenburg Country Director: Franz Drees-Gross Sector Manager: Michel Kerf Project Team Leader: Ingo Wiederhofer ICR Team Leader: Patricia Fernandes 2 ABBREVIATIONS AND ACRONYMS Aldeia Hamlet AusAID Australian Agency for International Development BA Beneficiary Assessment CAS Country Assistance Strategy CD Country Director CDD Community Driven Development CEP Community Empowerment Program CMU Country Management Unit GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit (German International Cooperation Enterprise) GDP Gross Domestic Product ETC Extended Term Consultant EPP Emergency Project Paper FM Financial Management GP Government Program GoTL Government of Timor-Leste ICR Implementation Completion Report IE Impact Evaluation IFR Interim Unaudited Financial Reports ILO International Labour Organisation IS Implementation Support ISN Interim Strategy Note ISR Implementation Support Report KPI Key Performance Indicator LDP Local Development Program MAG Martial Arts Gangs M&E Monitoring and Evaluation MIS Management and Information System MSTAM Ministry of State Administration and Territorial Management NOL No Objection Letter NYF National Youth Fund NYP National Youth Policy PDD Programa Descentralizado de Desenvolvimento PDO Project Development Objective PNDS Programa Nacional de Desenvolvimento do Suco RF Results Framework RVP Regional Vice President SSYS Secretariat for Youth and Sports Suco Village TF Trust Fund UNCDF United Nations Capital Development Fund UNDP United Nations Development Programme UNICEF United National Children’s Fund WB World Bank YC Youth Centers Y-CIC Youth – Community Implementation Committee YCMC Youth Center Management Committee YDP Youth Development Project YIT Youth Implementation Team YPER Youth Public Expenditure Review 3 Democratic Republic of Timor-Leste Youth Development Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph Table of Contents 1. Project Context, Development Objectives and Design ........................................................................ 7 2. Key Factors Affecting Implementation and Outcomes ..................................................................... 14 3. Assessment of Outcomes ................................................................................................................... 24 4. Assessment of Risk to Development Outcome ................................................................................. 31 5. Assessment of Bank and Borrower Performance .............................................................................. 31 6. Lessons Learned ................................................................................................................................ 34 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ................................... 37 Annex 1. Project Costs and Financing................................................................................................ 38 Annex 2. Outputs by Component ....................................................................................................... 39 Annex 4. Bank Lending and Implementation Support/Supervision Processes .................................. 43 Annex 5. Beneficiary Survey Results ................................................................................................. 45 Annex 6. Stakeholder Workshop Report and Results ........................................................................ 49 Annex 7. Summary of Borrower’s ICR and/or Comments on Draft ICR .......................................... 50 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders ........................................... 51 Annex 9. List of Supporting Documents ............................................................................................ 52 MAP 4 A. Basic Information Timor-Leste - Youth Country: Timor-Leste Project Name: Development Project Project ID: P106220 L/C/TF Number(s): IDA-H4140 ICR Date: 03/19/2013 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: ERL Borrower: TIMOR-LESTE Original Total XDR 1.30M Disbursed Amount: XDR 1.30M Commitment: Revised Amount: XDR 1.30M Environmental Category: C Implementing Agencies: Ministry of State Administration & Territorial Management (MSATM) Secretariat of State for Youth and Sports (SSYS) Co-financiers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 09/28/2007 Effectiveness: 12/08/2008 03/12/2009 Appraisal: 05/14/2008 Restructuring(s): 12/15/2011 Approval: 07/01/2008 Mid-term Review: 12/31/2009 10/28/2010 Closing: 12/31/2011 09/30/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Unsatisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Unsatisfactory Performance: Performance: 1 C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project at Yes Quality at Entry (QEA): None any time (Yes/No): Problem Project at any time Quality of Supervision Yes None (Yes/No): (QSA): DO rating before Moderately Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Other social services 62 62 Public administration- Other social services 38 38 Theme Code (as % of total Bank financing) Conflict prevention and post-conflict reconstruction 15 15 Other rural development 15 15 Participation and civic engagement 20 20 Social Inclusion 50 50 E. Bank Staff Positions At ICR At Approval Vice President: Axel van Trotsenburg James W. Adams Country Director: Franz R. Drees-Gross Nigel Roberts Sector Manager: Michel Kerf Cyprian F. Fisiy Project Team Leader: Ingo Wiederhofer Ingo Wiederhofer ICR Team Leader: Patricia Maria Fernandes ICR Primary Author: Patricia Maria Fernandes 2 F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The overall objective of the Youth Development Project (YDP) was to promote youth empowerment and inclusion in development by expanding the capacities of, and opportunities for, youth groups to initiate and participate in community and local development initiatives. Revised Project Development Objectives (as approved by original approving authority) There were no changes made to the original PDO, however, the following key indicators, were modified at Mid-Term (October 2010): (i) percentage of targeted communities where youth feel that there are more opportunities for youth participation in community and local development; and (ii) percentage of targeted communities that report increased youth participation in local and community development activities. The last key indicator: "Improved Secretariat of State for Youth and Sports performance ratings" remained the same throughout the project life-time. (a) PDO Indicator(s) Original Target Formally Actual Value Achieved Values (from Indicator Baseline Value Revised Target at Completion or approval Values Target Years documents) % of targeted communities where youth feel that there are more opportunities for youth Indicator 1 : participation in community and local development. Value quantitative or 0 60% N/A 100% Qualitative) Date achieved 03/12/2009 12/15/2011 12/15/2011 09/30/2012 Comments Indicator revised at project re-structuring on 15th December 2011. (incl. % Data was collected through the Beneficiary Assessment (BA) implemented in October achievement) 2012. Additional details on methodology are provided in Annex 5. % targeted communities that report increased youth participation in local and community Indicator 2 : development activities. Value quantitative or 0 60% N/A 100% Qualitative) Date achieved 03/12/2009 12/15/2011 12/15/2011 09/30/2012 Comments Indicator revised at project re-structuring on 15th December 2011.Data was collected (incl. % through the Beneficiary Assessment (BA) implemented in October 2012 in six targeted achievement) villages. Indicator 3 : Improved Secretariat of State for Youth and Sports performance ratings. Value Moderately Moderately quantitative or Low Satisfactory Satisfactory Unsatisfactory Qualitative) Date achieved 03/12/2009 03/12/2009 12/15/2011 09/30/2012 Comments (incl. % achievement) 3 (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Achieved Values (from Indicator Baseline Value Revised Target at Completion or approval Values Target Years documents) Indicator 1 : Performance Monitoring System for SSYS designed and functioning. Value Functioning with (quantitative None Functioning N/A shortcomings or Qualitative) Date achieved 03/12/2009 03/12/2009 03/12/2009 09/30/2012 Comments (incl. % Data source: SSYS Capacity Assessment (Final report 2012). achievement) % of Youth Centers Management Committee (YCMC) members from targeted Youth Indicator 2 : Centers that report an increase in leadership, management and planning skills. Value (quantitative 0 60% N/A 100% or Qualitative) Date achieved 03/12/2009 12/15/2011 12/15/2011 09/30/2012 Comments Data collected through Focus Group Discussions and Interviews with YCMC members. (incl. % Indicator reformulated at project re-structuring to focus on training provided to YMCM achievement) members (rather than on satisfaction with overall services provided at youth centers). Indicator 3 : % of youth participating in youth decision-making meetings in targeted communities. Value (quantitative 0 30% N/A 22% or Qualitative) Date achieved 12/15/2011 12/15/2011 12/15/2011 09/30/2012 Comments (incl. % New indicator introduced at project re-structuring achievement) Indicator 4 : Number of YDP subprojects implemented by youth in targeted communities. Value (quantitative 0 200 N/A 289 or Qualitative) Date achieved 12/15/2011 12/15/2011 12/15/2011 Comments New indicator introduced at project re-structuring. (incl. % achievement) % of youth participating in the implementation of youth development projects in targeted Indicator 5 : communities Value (quantitative 0 20% N/A 18% or Qualitative) Date achieved 03/12/2009 12/15/2011 12/15/2011 09/30/2012 Indicator revised at project re-structuring. Data collected through the MSATM MIS. Comments (incl. % achievement) 4 % of youth in FGDs in targeted communities who are satisfied with the goods and Indicator 6 : services delivered through the YDP youth grants. Value (quantitative 0 70% N/A 100% or Qualitative) Date achieved 03/12/2009 03/12/2009 03/12/2009 09/30/2012 Comments Indicator revised at project re-structuring. Data was collected through the Beneficiary (incl. % Assessment (BA) implemented in October 2012 in six targeted villages. achievement) G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 06/27/2009 Satisfactory Satisfactory 0.20 2 06/28/2010 Moderately Unsatisfactory Moderately Unsatisfactory 0.27 3 04/12/2011 Moderately Unsatisfactory Moderately Unsatisfactory 0.95 4 06/27/2011 Moderately Satisfactory Moderately Satisfactory 1.02 5 03/17/2012 Moderately Satisfactory Moderately Satisfactory 1.55 H. Restructuring (if any) ISR Ratings at Amount Restructuring Board Approved Restructuring Disbursed at Reason for Restructuring & Key Date(s) PDO Change Restructuring in Changes Made DO IP USD millions There were no changes to the original PDO. Changes were made to KPIs as 12/15/2011 N MS MS 1.23 outlined in the results framework analysis above. 5 I - Disbursement Profile 6 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Country Context In 1999, the citizens of Timor-Leste voted overwhelmingly for the restoration of independence from Indonesia. After the referendum, exiting forces inflicted a campaign of violence across the country that resulted in widespread destruction and massive displacements of civilians. In the aftermath of this post-independence violence, the Timorese Government made significant steps to foster reconciliation and showed a commitment to promoting growth and reducing poverty with a focus on basic economic and social services. Security was assured through the deployment of a UN peacekeeping mission that focused on peace-keeping, training, and support to the Timorese police force. In 2006, however, Dili was again plunged into violence after a disaffected group of soldiers of the national army (the F-FDTL) went on strike and were subsequently dismissed. In the ensuing violence over 160,000 people were displaced and at least 2,000 houses were torched. The unrest had a devastating impact on the economy: coffee production fell by about 20 percent and real non-oil GDP declined by 5.8 percent in 2006.1 The deployment of a new United Nations peacekeeping mission was necessary to assist with the restoration of security. Insecurity and periodic episodes of violence continued well into early 2008. Improved security following the 2006 civil unrest, higher oil-financed public spending, and a rebound in agriculture from the 2007 drought contributed to non-oil GDP growth averaging 11 percent during 2007/09. In 2010, however, about 40 percent of the population was estimated to still live in poverty with the 2010 Human Development Report ranking Timor-Leste at 120 out of a total of 169 countries. Greater visibility of youth exclusion issues in Timor-Leste With 34 percent of the population aged between 12 and 29 years of age, and a high fertility rate of 7.8 births per woman, meeting the needs and aspirations of young people is a medium to long term development challenge in Timor-Leste. At the time of project preparation the potential risks of growing youth marginalization had been brought to the forefront of policy discussions. Persistent fighting among youth groups, gang and martial arts group violence were key manifestations of the crisis and instability and ongoing problems at appraisal. While youth were not the instigators of the violence, they were used as agitators and also exploited the breakdown in law and order to loot businesses and exact revenge for past grievances. Analytical work supported by the World Bank on Youth at Risk in Timor-Leste (2007)2 highlighted some of the critical, underlying, issues putting young people at risk of continuing violent behavior. The analysis stressed that youth engagement in violence was not new with both the Resistance leadership and Indonesian authorities having encouraged or condoned violence by youth, respectively as a form of resistance or as a means to intimidate the electorate. A complex mix of economic, social and 1 IMF, 2010 Article IV Consultations. 2 World Bank (2007) Timor-Leste’s Youth in Crisis: Situational Analysis and Policy Options 7 political factors in a context where state institutions, formal sector employment and supra-local identities were all relatively new, meant that this risk was maintained if not exacerbated in the post-independence period. Key risk factors were as follows: (i) high youth unemployment rates of 43 percent nationally and up to 59 percent in the capital Dili; (ii) low educational attainment with only 1.7 percent of males and 3.4 percent of females (15 to 29 years old) having completed primary school3; (iii) low literacy rates in the official language at Independence (Portuguese)4; and (iv) weakened social ties with communities and traditional social control mechanisms, particularly in urban areas. This latter factor was linked to growing membership in youth gangs often active in criminal activities, extortion and vigilantism.5 The World Bank supported analysis looked at issues of youth exclusion holistically and recommended a focus on bringing youth into decision-making processes at community level to foster a closer and constructive engagement with traditional authorities and local government. It also recommended strengthening government’s own capacity to reach out to young people and youth organizations at the central level in order to include these groups in the overall nation-building process. Supporting the Government of Timor-Leste’s Stabilization Strategy The Youth Development Project (YDP) sought, therefore, to address key issues of youth social exclusion and perceived alienation from the nation-building process, working simultaneously at the community and local government levels. The project was processed under OP/BP 8.00 6 and responded to a direct government request to operationalize its strategies for addressing the role of youth in the 2006/07 crisis. A new government, formed in 2007, prioritized a number of measures to stabilize Timorese society through the National Priority Program. A National Youth Policy (NYP) was also promulgated in 2007 as part of this overall effort. It addressed a wide range of social, economic and political issues affecting youth. The YDP constituted an important element of a more comprehensive governmental stabilization strategy that also included support for the return and reintegration of displaced populations, generous benefits for veterans, food price stabilization, expansion of public sector employment, and a significant fiscal stimulus. The project was designed to include, on the one hand, activities with short- term impacts and immediate gains for disenfranchised young people. On the other, it aimed, in the medium term, to strengthen government capacity to work more effectively with youth and youth organizations in order to engage them in community life and nation-building. The approach followed by YDP was in line with the World Development Report of 2007: Development and the Next Generation, which highlighted the importance of broadening the opportunities for youth and developing their capacities as decision-makers. It was also consistent with the Country Assistance Strategy 2006/08 (CAS) which stressed the importance of addressing youth 3 Demographic and Health Survey (2003) 4 This was found to be an important element of youth’s perceived disfranchisement and strained relations with government. 5 See also James Scambary, Hippolito Da Gama and João Barreto (2006), A Survey of Gangs and Youth Groups in Dili, Timor-Leste. 6 A blanket authorization was obtained from the then RVP to process all operations in support of the GoTL’s stabilization efforts in the aftermath of 2006 under OP 8.00 (Rapid Response to Crises and Emergencies) to allow for a faster response. OP 8.00 can be accessed at: http://web.worldbank.org/WBSITE/EXTERNAL/PROJECTS/EXTPOLICIES/EXTOPMANUAL/0,,contentMDK:21238942~isCURL:Y~menuPK:6470163 7~pagePK:64709096~piPK:64709108~theSitePK:502184,00.html 8 issues in a cross-cutting manner in all its three pillars: (i) delivering sustainable services; (ii) creating productive employment and; (iii) strengthening governance. 1.2 Original Project Development Objectives (PDO) and Key Indicators The PDO for YDP was to promote youth empowerment and inclusion in development by expanding the capacities and opportunities for youth groups to initiate and participate in community and local development initiatives. The original key indicators were as follows7: (i) percentage change in female and male youth from targeted communities who feel that there are more opportunities for youth participation in community and local development; (ii) percentage change in female and male youth from targeted communities who participate in planning and managing community development activities in urban and rural areas; and (iii) Improved Secretariat of State for Youth and Sports (SSYS) performance ratings.8 1.3 Revised PDO and Key Indicators, and reasons/justification There were no changes to the original PDO, however, key indicators, were modified at Mid- Term (October 2010) through the inclusion of two new indicators: (i) percentage of targeted communities where youth feel that there are more opportunities for youth participation in community and local development; (ii) percentage of targeted communities that report increased youth participation in local and community development activities. The last key indicator - Improved Secretariat of State for Youth and Sports performance ratings - was not changed. The corresponding end of project target was, however, revised down from “Satisfactory� to “Moderately Satisfactory.� In addition, the Intermediate Outcome Indicators were significantly modified to reflect project re-structuring. These changes are outlined in Section F of the data-sheet (Results Framework Analysis) and in Annex 2. The modifications mentioned above were introduced for three key reasons. Firstly, to address the fact that it was not possible to establish a baseline for the original indicators. The focus on “percentage change� of respondents reporting increased opportunities for and participation in project activities was therefore dropped. Secondly, to reflect the changes in the data collection approach adopted by the team as additional trust fund resources expected to support quantitative data collection were not secured. The team shifted its approach to use less costly qualitative methodologies providing an overview of youth and community perspectives on participation. Compiling data at community level, also required that gender disaggregated information be dropped at the Key Performance Indicator level. Gender disaggregated information was, however, collected on participation rates in consultations for the design of sub-projects as well as in actual sub-project implementation through the project’s Management Information System (MIS). In addition, the Beneficiary Assessment (BA) collected qualitative information on the perceptions of male and female youth of the opportunities for participation created at 7 As outlined above there was agreement with the implementing agency to begin monitoring the revised/new indicators after the Mid-Term Review in October 2010, although project re-structuring was completed in December 2011 (additional details are provided in Section 3). 8 Measured through periodic capacity assessments carried out by an independent consultant. 9 community level by YDP. Finally to take into account the scaling back of the project’s Component 1, with several activities originally envisaged being dropped. 1.4 Main Beneficiaries At appraisal, the primary target group for YDP were young people (those between 16 to 30 years of age in line with the NYP) in both urban and rural areas. Following project re-structuring in 2011, YDP focused to a larger extent on rural youth though the project’s Component 2. Activities under Component 1 (including the provision of grants to youth groups in urban areas through the planned NYF and the provision of youth center based services) were considerably scaled back. Following project restructuring, this latter component focused on a more modest set of capacity building targets. (Additional analysis will be provided in sections 1.6 and 3). Under the revised Component 2, small grants were provided to youth groups for the implementation of sub-projects that addressed the needs of young people in targeted villages. These sub- projects covered a wide range of activities with a strong focus on sports and livelihoods. The project benefitted approximately: (i) 20,000 young people (37% female, and 63% male) through participation in decision-making meetings to determine sub-project activities and design. Of these, 13,000 young people (38% female and 62% male) were also actively involved in the implementation of sub-project activities. In line with YDP’s Management and Information System (MIS) approximately 17% of all young people in targeted communities participated in sub-project implementation (against a target of 20%). The Emergency Project Paper (EPP) did not set specific targets for numbers of beneficiaries to be reached. It instead set a target of 30% change in (male and female) youth participation in community- level organizations. This was revised at project Mid-Term with the updated indicator setting an end of project of 20% of youth participating in the implementation of youth development projects in targeted communities (Please see Annex 2 for additional details). Activities at the village (suco) and hamlet (aldeia) level targeted the following five of Timor- Leste’s 13 districts9: Aileu, Bobonaro, Ermera, Lautem and Manufahi. Given the challenging institutional context in which the project was implemented, one of the team’s design choices was to build to some extent on existing Ministry of State Administration and Territorial Management (MSTAM) programs. YDP used the platform/structure (including staff at district level) of the existing GoTL Local Development Project (LDP) to channel small grants for youth and community activities. The initial selection of target districts was therefore based on the presence of LDP10. A second factor, which was critical in the selection of target areas, was the extent to which GoTL had identified the risk of continued violent behavior on the part of young people in certain of these LDP districts. Consultations with MSATM, SSYS and civil society organizations on youth at risk, therefore, also informed the final selection of target areas. While the team attempted to collect additional data on youth violence for targeting purposes limited information was available at district level making these consultations a critical part of the selection of target areas. 9 The project’s initial intention was to scale-up the innovative approach piloted by YDP to all 13 districts should implementation be successful. This will be discussed further under Section 3.1. 10 The LDP had been established with the support of UNCDF and UNDP in 2004. It was active in eight districts at the time of appraisal with a budget of US$3,000,000 planned for execution in 2008. Under LDP, local assemblies were established at the district and sub-district level to foster community involvement in local development planning and oversight. 10 The first districts selected for project implementation were, Bobonaro and Lautem (the most experienced in LDP implementation), followed by Lautem, Manufahi and Ermera.11 In addition, the project also targeted youth organizations through the support it provided to youth centers in the district capitals of Aileu, Ermera, Oecussi and Manufahi. 12 Finally, it is important to highlight that institutional support was extended to both SSYS (albeit scaled back after re-structuring) and MSATM to improve: (i) their capacity to address the needs of young people and; (ii) their ability to reach out to youth groups at central and community level. 1.5 Original Components The project consisted of the following two Components: Component 1: Operationalizing the National Youth Policy (US$520,000). This component was designed to support the SSYS to operationalize the National Youth Policy by: (i) strengthening its institutional capacity to develop, implement and coordinate policy and programs relating to its mandate, including a unit to administer the National Youth Fund (NYF) within SSYS; and, (ii) enhancing its ability to work with youth groups, organizations and youth centers and support youth initiatives. It included two sub-components.  Sub-Component A – Institutional Strengthening: to strengthen the capacity of SSYS to develop, implement and coordinate policy and programing relating to its mandate as identified in the NYP. Specifically, this component aimed to assist SSYS to implement a series of capacity building workshops, conduct annual progress reports of its capacity and undertake an annual expenditure review of the government budget allocation for youth programming (Youth Public Expenditure Review). In addition, it aimed to support the establishment of a unit to manage and administer the NYF, including: (i) technical assistance; (ii) funding for office refurbishment and equipment, and (iii) support for outreach and capacity building.  Sub-Component B - Strengthening Youth Centers: providing funding to strengthen the capacity of SSYS to work with and coordinate the activities of youth centers. The sub-component covered capacity building for staff and funding for management of selected youth centers. Component 2: Youth For Local Development (US$1.6 million). This component supported MSTAM to establish a mechanism to distribute small grants to youth in rural villages and hamlets through the existing LDP. The component targeted five districts with each expected to be provided with a minimum of two cycles of funding13. The three sub-components include: 11 Consideration was also given to achieving some degree of geographical balance between East and West regions of Timor- Leste primarily through the inclusion of Lautem district. 12 These were intended to demonstrate a potential Youth Center model that could be replicated by SSYS. 13 This happened in all districts with the exception of Ermera, where sub-projects ran for one cycle only as the district was added to the project after re-structuring in 2011. 11  Sub-Component A – Youth Grants: providing funds earmarked for youth groups at the village level to identify, prioritize and implement sub-projects – such as livelihood activities, rehabilitation of youth centers, social events, construction of small infrastructure and life skills training. Grant allocations were calculated on a per-capita basis (U$1.8/person) and resulted in allocations of approximately US$2,200 to US$5,000 per village depending on the population. These allocations were based on the Local Development Program (LDP) formula and were initially adopted by the team for purposes of consistency.  Sub-Component B – Youth Facilitators and Capacity Building: providing support to train and staff a corps of youth facilitators to guide implementation of activities at village level. The assistance provided included information dissemination on the project to young people, youth consultation meetings, and capacity building of a Youth Implementation Team (YIT) to oversee and manage sub- project implementation at hamlet/village level.  Sub- Component C – Program Implementation and Technical Support: augmenting project implementation capacity of the Project Management Unit within MSTAM. This included outreach, training, administrative support staff, equipment as well as technical assistance with documentation, training development and Monitoring & Evaluation. The component included communication activities associated with project implementation and technical assistance to ensure that the use of the youth grants was gender inclusive. Table 1: YDP Component 2 Sub-Project Categories (Cycle I & II)* Percentage of Sub-Project Type Total Sub-Projects Sports Field – New 25.0% Livelihoods 19.5% Sports Equipment 12.6% Youth Center – New/Rehabilitate 9.9% Sports Field – Rehabilitate 7.3% Musical Equipment 6.4% Training 5.7% Community Infrastructure 5.3% Computer Equipment 2.3% Agriculture 1.2% Transport 0.9% Other 4.1% Total 100% * Source: MIS of PMU/MSATM as of 30th September 2012 12 1.6 Revised Components There were no changes to overall project components, although the scope of activities under Components 1 and 2 were revised as outlined below. In particular, activities relating to the implementation of the NYF were dropped at project re-structuring. 1.7 Other significant changes (in design, scope and scale, implementation arrangements, schedule, and funding allocation) Adjustments were made to project design following the Mid-Term Review (October 2010) in order to address: (i) delays in overall project implementation; and (ii) technical and capacity challenges with the execution of activities under Component 1. These changes also sought to leverage the good foundation established by MSTAM for the implementation of Component 2. Under Component 1, changes at SSYS senior staff level (discussed further under Section 2.2) caused significant implementation delays and changes in ownership of key elements of this project component. Activities relating to the operationalization of the NYF were dropped due to insufficient capacity by SSYS to carry out the original approach during the project implementation period. Specifically, a draft policy and outline of the corresponding operations manual for the NYF were developed but were found to be of inadequate quality. With one year to expected project closure (original date of December 2011), SSYS continued to experience difficulties in identifying and retaining qualified staff that could provide adequate Technical Assistance to NYF design. In addition, the implementation of the planned Youth Public Expenditure Review (YPER) had not yet been initiated at Mid-Term. In light of the limited time remaining for Project implementation both activities were dropped in the project restructuring.14 This had implications on the extent to which the project was able to reach out to youth in urban areas and in terms of the overall sustainability of the approach. (Additional details will be provided in Section 3). In parallel, activities under Component 2 were scaled-up with Youth Grants provided to an additional district (Ermera). Changes in the operationalization of the sub-grant component were also introduced at mid-term to address bottle-necks in implementation and ensure that grants were more responsive to the needs of youth groups: (i) village block allocations were modified to reflect 2010 national census data; (ii) village block grant allocations were increased from $1.80 to $2.50 per person; (iii) the maximum size of sub-projects was increased to US$5,000 and the minimum set at $800: (iv) the minimum overall village allocation was increased from $1,000 to $2,000 and the maximum from $10,000 to $15,000; and (v) sub-projects that only involved the purchase of goods (e.g., sport uniforms, sports equipment) and were less than $1,500 could be disbursed in one tranche. In line with the changes highlighted above on the scope of Components 1 and 2, funding allocations were also modified (Table 2) and the project closing date extended from December 31 2011 to September 30 2012. 14 Project re-structuring was recommended by the task team following the Mid-Term Review (MTR) in October 2010 but required lengthy discussions/negotiation with SSYS. Final re-structuring was finally processed in December 2011. 13 Table 2: Revised allocation of project funds following project restructuring (December 2011) Project Costs (US$) Components Original After Restructuring Component 1: Operationalizing the National Youth Policy 520,000 290,000 Component 2: Youth for Local Development 1,598,000 1,828,000 Total 2,118,000 2,118,000 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design, and Quality at Entry Three important factors affected YPD preparation, design and quality at entry: first, YDP was an emergency response project whose preparation took place over a short period of time with the task team therefore focusing on essential analytical work only; second, the team sought to follow best practice approaches in post-conflict reconstruction by addressing both short-term risks and longer capacity challenges. Both of these factors contributed to a somewhat ambitious design subsequently simplified during implementation. The third factor was reliance on Trust Fund (TF) resources to be mobilized for the implementation of the project’s monitoring and evaluation strategy. The background analysis carried out by the team was sound and focused on the essential elements required for project design. Project design followed best fit for operations in fragile contexts. However, the team made some design choices for monitoring and evaluation based on anticipated TF resources which were not secured, and this required subsequent adjustments during implementation. a. Short preparation time-frame The preparation time-frame for YDP was compressed. The fragile security situation required a rapid intervention on the part of GoTL and therefore an equally rapid World Bank response in support of GoTLs stabilization plan. Preparation began in March 2008 and appraisal was conducted two months later (May 2008). While this enabled the task team to be responsive to counterpart needs it limited the extent to which additional analysis could be carried out. In terms of quality at entry, the team made the most of the available information and preparation time to secure critical data. The team drew on existing analytical work carried by the World Bank on youth at risk 15and carried out short pieces of targeted analysis. Critical pieces were initiated during preparation and the majority completed before project effectiveness as follows:  A capacity assessment of SSYS to inform the implementation of the NYP and related capacity building activities.  A review of lessons-learned from the implementation of Youth Center (YC) programs, as well as a separate mapping of existing YC in Timor-Leste and the services they offered at the time of appraisal. 15 World Bank (2007) Timor Leste’s Youth in Crisis: Situational Analysis and Policy Options 14  A detailed fiduciary capacity assessment of the implementing agencies (MSTAM and SSYS). Based on the findings the team concluded that overall management of fiduciary matters should remain with MSTAM. This fiduciary arrangement created a requirement for operational cooperation between MSTAM and SSYS which added to project complexity. It was, however, the only viable operational alternative at the time based on the available information on implementing agency capacity. b. Weak institutional context and lessons-learned from other operations One the main challenges faced at entry by YDP was the need to factor into its design the weak institutional context in which the project was being implemented. The initial design choices by the task team dealt with this by following best fit/best practice approaches for post-conflict operations and introducing a capacity building element in project design (while building on on-going programs to generate quick wins for youth). Some of the capacity building elements of YDP were relatively ambitious and brought a greater degree of complexity to the project. The situation analysis of 2007 stressed the importance of a dual strategy to address youth issues in Timor-Leste: (i) obtaining short term gains for youth; and (ii) supporting improvements in government’s capacity to reach out and include young people in the development process. This approach is in line with the World Development Report on Conflict Security and Development (2011), which highlights the role that weak institutions play in perpetuating the risk of conflict and violence in post- conflict settings. The design of Component 1 required SSYS to make rapid/critical changes in a short period of time (from 2009-2011). In particular, it required a shift from having no clear definition of functions or Terms of Reference for core staff or key Director-level staff appointed (given the transition period for the establishment of the new government in 2007) to developing and fully operationalizing a new grant making mechanism (NYF). Component 1 also included the establishment of a set of effectively managed Youth Centers, in practice creating a model that could subsequently be replicated as well as the implementation of the Public Expenditure review of programs aimed at young people. This element of the design required substantial efforts as outlined in the team’s initial assessment. Inherent risks in following this approach were thoroughly discussed both within the task team, with the broader Country Management Unit and most importantly with GoTL. The option of working solely on the outreach to youth at community level (Component 2) was discussed and ultimately discarded primarily given strong GoTL request for capacity building support. The preparation of the project’s Component 2, providing small grants to young people at suco level, drew extensively from the existing LDP implemented by MTSAM. The task team’s analysis focused on how community representation could be further improved in LDP and on the operational adjustments (including additional technical assistance on the part of SSYS) required to adapt the platform to channel grants to youth groups. This included developing a separate process for youth socialization and project identification activities. In addition, the team reviewed options for the coordination of technical support to sub-project preparation between YDP and LDP teams drawing on available district level staff for technical assistance. Importantly, the initial analysis carried out also stressed the importance of training a core of youth facilitators to oversee youth participation and monitor project implementation at community level. Finally, the task team was also able to review an earlier experience of using a Community Driven Development (CDD) approach in a post-conflict setting in 15 Timor-Leste. The Community Empowerment Project (CEP) supported by the World Bank from 2000 to 2003 faced significant implementation challenges in terms of quality of infrastructure and sustainability of investments. Lessons learned from this earlier intervention reiterated the importance of using existing systems with an established outreach structure and mechanisms for technical assistance. c. Basing some elements of design on Trust Fund Resources to be mobilized While the issue of Monitoring & Evaluation will be addressed below (Section 2.2) it is important to mention that the initial design of the M&E strategy relied on additional Trust Fund (TF) resources to be mobilized by the task team.16 It therefore included a strong focus on survey work (including the collection on baseline data), which is relatively costly. Given the level of TF resources available for GoTL through AusAID at the time of appraisal it was not unreasonable for the team to expect that funding would be available for the proposed analysis. However, changes in the way in which AusAID TF resources were allocated at regional level from 2010 onwards meant that WB supported interventions in Timor-Leste were no longer eligible for support. The team subsequently adapted and simplified M&E design, using primarily qualitative methodologies. This was a short-coming at entry which subsequently affected implementation (Additional details are provided in Section 5). 2.2 Implementation YDP faced three main implementation challenges: (i) changes in SSYS staffing and further institutional instability impacted government commitment to the implementation of Component 1; (ii) in 2010 substantial additional resources were channeled through MTSAM for a new local development initiative straining its capacity further; and (iii) challenges in staffing key positions given high levels of competition for human resources both in Dili and at district level. a. Institutional instability at SSYS level Changes in senior level staff at SSYS meant that the World Bank team lost its main counterpart in the institution, which had participated actively in the preparation process. A number of key senior positions remained vacant or in flux (with Director-level positions pending confirmation) during the first two years of project implementation (2008 and 2009). This impacted SSYS ownership of and commitment to the implementation of capacity building activities under Component 1, while more fundamental managerial staffing issues were being resolved. Therefore, if core capacity building activities with SSYS were implemented, the overall scope of Component 1 activities was significantly reduced as the task team took stock of challenges and pace of implementation at Mid-Term. Definition of core functions for senior staff and training programs were implemented as planned and critical support was extended to youth centers as intended albeit at a slower pace. The more ambitious elements of this Component: (i) the proposed YPER and; (ii) the operationalization of the NYF did not, however, take place as initially planned, and associated resources were re-allocated to the project’s Component 2. 16 The WB task team and GoTL agreed on a division on M&E responsibilities to take into account capacity constraints of the implementing agencies. SSYS and MSTAM were responsible for the project’s monitoring function while the WB took responsibility for implementing the project’s Impact Evaluation. 16 b. Launching of a new program by MSTAM (Programa de Desenvolvimento Descentralizado) The weak institutional capacity identified during preparation resulted in difficulties in the management of staff, identification, retention and effective supervision of key consultants. This can be well exemplified by the delays in project effectiveness (declared eight months following Board approval). The situation was, however, aggravated by the introduction of a new program at MSTAM level. From 2009, GoTL efforts to accelerate the delivery of basic services and its outreach to under-served communities generated a dramatic increase in the amounts programed in the State Budget for key Ministries and State Secretariats (MTSAM and SSYS). A new local development program (Programa Descentralizado de Desenvolvimento or PDD) was launched in 2010 to accelerate service provision. The program implemented by MSATM had a budget of US$30 million in 2010 and US$44 million in 2011. The amount of resources invested through PDD dwarfed those of YDP. Given the fragile institutional framework and existing challenges in identifying skilled staff for program management, the establishment of this new program resulted in changes in the overall management team at MSATM. In particular, senior staff left YDP to lead the management and roll-out of PPD. Secondly, key financial management and procurement staff at MSTAM responsible for fiduciary matters under YDP were also made responsible for overseeing PDD implementation and were overwhelmed by the demands of the new program. Thirdly, district level personnel, already thinly stretched to provide technical support to LDP/YDP implementation were also made responsible for a growing level of technical assistance activities at field level under PDD (including the supervision of the technical quality of sub-projects). The additional burden of technical supervision placed on district staff had a negative impact on the quality of the sub-projects developed as more limited support was available to help YDP with: (i) preparing budgets and designs; (ii) monitoring construction in the case of small infrastructure work; and (iii) providing specialized advice and technical assistance for the implementation of livelihood activities. A technical review of the sub-projects carried out under sub-component 2 was completed in January 2012.17 The final report, based on a review of 25 sub-projects in Aileu, Bobonaro and Lautem districts, indicated significant challenges with the quality of the infrastructure put in place with only 20% of the sub-projects reviewed considered to be of adequate quality. The task team systematically followed up on the technical recommendations of the review in subsequent Implementation Support (IS) missions and through additional technical missions providing candid feedback on the shortcomings of the technical support provided at district level. There were, however, limitations in terms of the additional support that could be provided for quality control in MSATM. Issues with the quality of sub-projects continued therefore to be raised in ISR throughout the remainder of project implementation (additional details are provided in Section 2.5). 17 While the task team and implementing agency had intended to finalize this review earlier, the assessment was postponed due to delays in procurement 17 c. Limited availability of qualified human resources Increased public sector spending and GoTL’s focus on accelerating service delivery also resulted in a drive to recruit an additional number of civil servants during the YDP implementation period. Reflecting a boarder issue of limited availability of qualified human resources in Dili, positions were, hard to fill and staff, once brought on board, was difficult to retain (please see Table 3 below). Similar challenges affected the recruitment of staff to support the establishment and maintenance of the project’s Management and Information System (MIS). While the project had initially planned to hire a dedicated Monitoring and Evaluation (M&E) Specialist it proved difficult to identify a suitable candidate and a more junior MIS consultant was ultimately recruited. In addition, further technical support on M&E was, therefore, provided directly by the task team. Table 3 – Overview of staff turn-over (key positions for project coordination) Key staff position (National Coordination) Year Financial Management Procurement Monitoring & Evaluation 2009 N/A N/A Vacant 2010 Vacant for the first six months of the year. Vacant Vacant for nine months Recruitment completed in June 2010 Recruitment completed in September 2010 for an MIS consultant (more junior position) 2011 Vacant in May 2011 Recruitment of procurement MIS consultant in place New recruitment completed in September assistant completed by October 2011 2011. The Procurement assistant position became again vacant after four months. 2012 In place Vacant MIS consultant in place 2.3 Monitoring and Evaluation Design, Implementation and Utilization Design YDP’s initial M&E design included the implementation of a quantitative Impact Evaluation (IE) and of a series of thematic studies including: (i) review of infrastructure, (ii) economic cost/benefit analysis, (iii) socialization review, (iv) gender review, and (v) review of program training and/or facilitation. Finally, M&E design also included the introduction of qualitative methods for the study of youth issues, such as the Participatory Ethnographic Evaluation and Research (PEER) methodology drawing on lessons-learned from youth violence interventions supported by the WB. With the exception of the gender and capacity building reviews, the task team expected to seek external funding for both the thematic studies and impact evaluation. 18 Regular monitoring of project activities was to be carried out through period capacity assessment reports implemented by an independent consultant for capacity building of SSYS and through the establishment of an M&E system at SSYS level for activities related to youth center usage and NYF implementation for Component 1. A simple MIS to: (i) track progress with sub-project implementation, (ii) participation rates by youth in sub-project design and (iii) implementation was planned for Component 2. In addition, the M&E strategy at design included the development of a Grievance Redress System (GRS), the implementation of community score cards to provide beneficiary perspectives on project implementation and the establishment of partnerships with NGOs to independently monitor project progress. The original M&E design covered all aspects of project implementation for both project components and the development of the MIS factored in capacity limitations by MSTAM. Some elements of this original design were ambitious, in particular the large number of thematic reviews to be implemented for a small-scale grant with a short implementation time frame and in a weak institutional context. YDP opted for a split M&E strategy based on the initial assessment of SSYS and MSTAM M&E capacity which was considered low. Government took responsibility for the implementation of monitoring and process evaluation and WB (in close collaboration with GoTL) for the impact evaluation design and implementation. Implementation The scope of the IE strategy planned by the team was significantly reduced to address the lack of complementary TF resources as outlined above. The team shifted to the use qualitative methodologies with a detailed Beneficiary Assessment (BA) carried out in late 2012 to assess the achievement of the PDO. The need to change IE approaches meant that baseline information could not be collected which was factored into BA design (Please see additional details in Annex 5). Similarly, the lack of additional TF resulted in a reduction in the number and topics covered through thematic studies. In order to address these shortcomings, a partnership with the Justice for the Poor program was put in place and a youth lens included in key pieces of analysis: (i) a review of Local Government and Community Development Initiatives in 2011; (ii) Youth Perspectives on Community Trust and Conflict in 2010; and (iii) Trust, Authority and Decision Making also in 2010 (drawing on the Timor-Leste Survey of Living Standards). Of the planned studies, MTSAM carried out a review of the quality of sub-projects and, while a gender review was not carried out, gender disaggregated data was collected on youth participation. Finally, the capacity assessment of SSYS was the third study to be implemented as planned (implemented at project completion).18 Capacity constraints during implementation required MTSAM and SSYS to focus on collecting essential information on project progress in line with the Results Framework. The task team assessed the feasibility of developing a GRS and participatory monitoring and concluded that this would not be feasible given other implementation challenges. 18 Both a baseline assessment (2008) and final report (2012) were produced to generate lessons-learned. 19 A Management Information System was put in place to monitor the progress with sub-project implementation under Component 2, which was operational at project closure. There were initial challenges with the establishment of the system primarily due to difficulties in the recruitment of M&E staff as outlined above but the system generated the key data to monitor project progress and take corrective action. Given project re-structuring and the reduction of activities under Component 1, the accompanying M&E activities (including support to the SSYS M&E unit) were also dropped. Utilization The BA provided key information to understand improvements in youth participation rates at community level (in line with the Results Framework) and generated lessons-learned on the quality of participation, facilitation and community mobilization processes. In spite of its limitations in terms of sample size,19 the qualitative methodology used to collect information yielded rich and detailed information on participation processes. When assessing project results it is important to consider this data together with the information on participation generated by the project’s MIS for a fuller picture of the project’s impact on youth participation. Overall, the evaluation strategy followed generated information at the end of project implementation so that lessons-learned were made available for subsequent interventions by GoTL. The results, shared with GoTL, will be particularly useful given GoTL’s focus on preparing a national community development program in the coming years, and they highlight areas where additional attention may be needed to facilitate youth and community participation. Given the innovative nature of the grant, implementing a process assessment or conducting shorter reviews of implementation quality might have been helpful to complement Implementation Support (IS) and inform corrective action by SSYS and MSATM as needed on the different project components. MIS for Component 2: While capacity to create new data queries was not established at MSTAM level, database maintenance and the production of progress reports was regularly carried out. Data was collected on the number and types of sub-projects and the profile of young people participating in community meetings and sub-project implementation. Candid and timely feedback was provided to the implementing agencies on the basis of the data collected particularly: (i) regarding the need to accelerate progress in selected sub-districts; (ii) the timely identification of potential problem sub-projects. The transfer of the M&E specialist to MSTAM as a civil servant at project closure is likely to result in greater sustainability of the current system. 2.4 Safeguard and Fiduciary Compliance Project compliance with safeguards requirements was overall satisfactory. YDP triggered OP 4.01 and an Environmental and Social Management Framework was developed to guide project implementation. While sub-projects were overall constructed in public land, minor issues were noted with the documentation at district level of Voluntary Land Donations (VLD). These were found not to 19 Data was collected from six treatment and four control villages for the Beneficiary Assessment (i.e ten of 192 project sites). 20 be significant and were satisfactorily addressed through: (i) sharing of templates for VLD with district staff; and (ii) additional training of MSATM to improve recording. Financial Management: The recipient complied with the financial management (FM) conditions outlined in all of the financing agreements and satisfied the Bank's requirements under OP/BP 10.02. Following the initial FM assessment, the FM risk was rated as “Substantial� even after mitigating strategies to address these risks. The project only adopted the use of country systems in two FM areas: project planning/budgeting and external auditing. The full use of country systems for accounting, internal controls, internal audit, treasury, and reporting were not adopted due to a combination of deficiencies identified in the FM assessment as well as the recipient’s preferences. There were significant shortcomings with financial management observed from the onset during project implementation and financial management performance was frequently rated as “Unsatisfactory" throughout the life of the project. Implementation Support Missions reported systematically on delays with audit reports and Interim Unaudited Financial Reports (IFRs), not in compliance with the Financing Agreement. Critical issues recurrently outlined in FM reviews were: (i) inadequate cash book format; (ii) issues of internal controls at the national and district level; (iii) reconciliation of expenditures; and (iv) unreliable accounting with no assurance that all the transferred funds reported as expenditures were documented. Implementation support and technical assistance provided by the team consisted of additional training and temporary mobilization of short-term consultants to address FM issues. In addition, disbursements under Component 2 were temporarily suspended in 2009 based on FM review findings. Implementation support provided follow-up on problem sub-projects. In the exceptional cases where a satisfactory resolution was not reached by MSATM, these were cancelled/ and expenditures considered ineligible. As at December 2012, all required audits of the project’s financial statements were received, although none of them were received by the due date. All audit opinions were unqualified and all audits were accepted by the World Bank. However, the implementing agency (MTSAM) did not address most of the findings and recommendations of the auditors in the management letters, with the same issues therefore being raised each year. In addition, with the exception of the first year audit, there was no implementing agency response to these audit recommendations. This lack of action resulted in continued shortcomings with FM and contributed to the frequent “Unsatisfactory� ratings. The final audit of the projects’ financial statements for the year 2012 will be on June 30 2013. The client has been reminded of their obligation to submit this final audit.20 Procurement: Delays with procurement occurred systematically during project implementation and had a negative impact on overall project performance. Implementation support reviews indicated that procurement procedures were not well understood. To a large extent these difficulties can be attributed to the lack of dedicated project staff for procurement during significant periods of time (as outlined in Section 2.2 above). The Project Coordinator (PC) for Component 2 temporarily took over the task of preparing and processing procurement documentation. In view of his overall role, it was not possible for the PC to effectively attend to procurement activities. Lack of dedicated staff/staff turn-over also limited the usefulness of the training and technical advice provided by the WB team. Recruitment 20 As at December 2012, all required quarterly interim unaudited financial reports (IFRs) were received, only two IFRs having been received by the due date. An IFR for period ended 31 December 2012 was due by 15 February 2013 (and had not yet been received at the time of writing). The client has been reminded of their obligation to submit this final IFR. 21 of temporary staff to support the coordination team, additional training provided by WB team resulted in improved procurement performance. The importance of dedicated staff for procurement functions and of intensive and regular training is a key lesson-learned in YDP. Weak institutional capacity and in particular difficulties on the part of the project coordination team to reach out and communicate with staff at district level required an adaptation of the usual WB model for FM and procurement supervision with more regular “hands-on�/on-the job follow up and training needed by counterparts. 2.5 Post-completion Operation/Next Phase The ICR mission reviewed transition and follow-up arrangements for capacity building activities implemented with SSYS and for sub-project investments implemented with MSATM as follows: a. Capacity building activities with SSYS have been effective in establishing standard roles and functions for senior staff. This improvement in basic management systems has helped to build a foundation for follow-up activities between SSYS and other development partners. While the operationalization of NYF was dropped from YDP following project re-structuring, SSYS is currently in discussions with UNICEF and GIZ to revive the fund. A draft Youth Policy is currently under development and is expected to be finalized in the coming year with UNICEF support. In addition, support for the development of the NYF operational framework and additional capacity building activities is being discussed as part of a partnership with GIZ. SSYS intends to further improve the coordination of the external support received from bi-lateral and multi-lateral partners and seek/develop additional partnerships. b. Sustainability of Youth Center21 Activities supported by SSYS: Capacity building activities for Youth Center Management Committees (YCMC) were highly valued by participants 22 and had a positive impact in supporting the development of follow-up activities. Regular funding by SSYS continues to be provided to cover the centers’ administrative costs (US$500/month). The amount is often received with significant delays and is considered insufficient to cover all basic expenses. Costs associated with the purchase of fuel for generators and maintenance of equipment (both computers and generators provided through YDP) are supported by external partnerships where they exist. Where these are not in place or have a limited scope (Ermera and Aileu) there are no alternative maintenance arrangements, which significantly limits the centers’ activities. Difficulties with the repair of IT equipment were highlighted by YCMC in all sites due to lack of qualified personnel/firms that can do these repairs at district level. 21 The initial mapping of Youth Centers carried out by the Task Team in 2008 indicated that there were approximately 50 youth centers nationwide. 22 Focus Group Discussions carried out with YCMC members during the ICR Mission in Aileu, Ermera, Same and phone interview with YCMC Manager for Oecussi. 22 There is a wide variation in the types of programs implemented by the Youth Centers 23. All organize small-scale language and computer training courses for youth in the district capitals. In addition, the centers are used as a resource to identify beneficiaries of life-skills (UNICEF) or health programs (Mary Stopes International). More ambitious follow-up activities were implemented in Same where the youth center has established strong partnerships with a range of organizations present at district level. Outreach activities by youth centers at district level tend, therefore, to depend on the resourcefulness of specific YCMC and follow the particular needs assessments carried by different donors (and address specific sectors/issues/concerns). A comprehensive plan of activities that addresses the overall needs of young people (covering a broader set of issues, or issues that are not necessarily dealt with by development partners) is not currently in place given the limited core funding provided by SSYS. c. Proposed expansion of the CDD platform with AusAID support: While no further youth-oriented WB operations are planned in Timor-Leste after YDP, MSATM is currently designing a national CDD program with AusAID support (Programa Nacional de Desenvolvimento do Suco). The YDP has helped to reopen the dialogue with GoTL on the potential use of CDD approaches to accelerate service provision in underserved areas, and to demonstrate associated opportunities and challenges. Specifically, YDP demonstrated the usefulness and feasibility of such as an approach following the challenges with the implementation of CDD programs in the early 2000’s (in particular, with CEP). The WB will continue to engage in these policy discussions and may provide analytical and advisory support to the new program. On an operational level, a better understanding of institutional capacity challenges and important lessons learned on community participation are now available through YDP to inform the design of the new operation. As MSATM moves forward with the establishment of the CDD platform and the basic implementation structure is established the World Bank team may wish to engage with MSATM and AusAID on the potential inclusion of a youth focus (or a possible dedicated program window for youth) once the program is more mature. The estimated value of the Programa Nacional de Desenvolvimento do Suco (PNDS) is US$230 million, of which AusAID may contribute up to US$30 million. This reflects a progressive shift away from individual donor funded/externally supported projects and towards support by development partners to establishing GoTL funded programs (drawing on existing resources from the national budget). d. Operation/Maintenance and Sustainability of YDP sub-projects: Implementation Support Missions (ISMs) provided systematic advice on the technical assistance support required to improve the quality of sub-project construction and livelihood activities. Based on supervision findings, certain types of activities and purchase of equipment were excluded because they risked duplicating programs being rolled out by SSYS (such as the purchase of musical instruments for youth groups) or were considered unsustainable (such as the purchase of computer equipment in areas with no reliable access to the electricity network). The sustainability of livelihood sub-projects was the focus of particular attention during ISMs as these interventions required specialized technical support on market and value chain analysis not available at MSATM level. The ISR, BA assessment and the ICR mission all highlighted challenges with: (i) quality of construction with isolated cases on incomplete projects due to difficulties on the part of youth groups to effectively budget sub-project proposals; (ii) issues with maintenance of equipment and lack of viable business plans for 23 It is also important to note that YCMC members are volunteers which limits to some extent the type of activities they can engage in and results in a high degree of turn-over in some of the centers (Aileu in particular). 23 livelihoods projects; (iii) use of sporting grounds where volley balls and nets were no longer available or were built in inadequate locations; (iv) or of youth centers which were found to be too small for the needs of youth groups for example. In spite of these issues with sub-project sustainability, which reflect the challenging implementation context of YDP, ISMs and ICR missions as well as the BA conducted at project closure highlighted the significant gains achieved by YDP in terms of youth participation in community life. Importantly, data collected during the BA indicated that participating youth considered that the skills gained through YDP can be employed in future development projects in their villages. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Objectives As outlined in Section 1.1 YDP was developed in the aftermath of violent clashes in 2006, which brought to the fore key issues of youth marginalization and social exclusion. The key themes raised by the 2007 Youth in Crisis Situation Analysis extensively used for YDP design and the operation’s overall objective remained relevant throughout project implementation as well as in the current context. It is important to note violence on a scale comparable to that of the 2006/7 crisis did not recur. Nevertheless, analytical work 24 on violence in Timor-Leste documented instances of communal conflict in rural areas as well as episodic violence in a number of Dili neighborhoods and persistent community concerns regarding security during project implementation. Youth continue to play a significant role in these violent incidents. A range of factors contributed to persistent insecurity throughout the project implementation period among which: (i) tensions between host and internally displaced communities; and (ii) competition over resources in the context of decentralization. Importantly, the analysis also pointed to the influx into Dili of rural youth looking for employment as an important destabilizing factor. Expectations of finding employment in the city are primarily motivated by a construction boom and commercial development. This latter finding supports earlier analysis on youth and employment25 carried out by the World Bank team and highlighted medium to long term challenges in this area. In particular, concerns raised regarding the aspirations of new job seekers who aim to move away from traditional occupations such as agriculture are particular pertinent. Young people face substantial challenges with large number of youth still reaching adulthood with no education (estimated to be 20% in 2007). This is compounded by the limited absorptive capacity of a slow growing private sector. 24 Robert Muggah and Emile LeBrun, (eds), 2010, Timor-Leste Armed Violence Assessment Final Report 25 World Bank (2007) , Youth Development and the Labor Market 24 Design YDP design remained relevant throughout the project life-span with solid GoTL ownership of the project’s approach to reaching young people through community-based interventions. However, government’s initial focus on addressing issues of youth at risk (as part of its stabilization strategy), gave progressively way to an increased emphasis in GoTL policy on youth employment and on overall basic service delivery to under-served areas. The 2006-2008 Country Assistance Strategy (CAS) was built around the Government’s first National Development Plan. Youth considerations were mainstreamed across its three strategic pillars: (i) Delivering Sustainable Services; (ii) Creating Productive Employment, and (iii) Strengthening Governance. Lessons learned from CAS implementation indicated that this mainstreaming approach may have contributed to less focused attention to youth priorities. A shift in strategy to address youth concerns was therefore adopted in the new Interim Strategy Note (2010-2011), with dedicated activities, including a growing focus on youth employment planned under Strategic Area 3: “Support for Short-Term Stabilization.�26 This followed GoTL’s own shift in emphasis in its approach to youth with a stronger focus towards youth employment once the initial stabilization phase following the 2007 violence was over. YDP focused primarily on bridging the gap between government authorities and traditional leadership on the one hand, and young people on the other. The new ISN acknowledged YDP’s contribution to youth inclusion through participation in local development activities. Its strategy for engagement with youth, however, placed greater emphasis on addressing issues of youth employment. Implementation The project remained relevant to country priorities throughout its implementation period and in line with GoTL stabilization plan and subsequent government program. YDP outcomes in terms of youth participation and youth inclusion in the nation-building process continue to be relevant in a context where the new Government Program (GP) (2012-2017) places renewed importance on: (i) accelerating service provision to under-served areas using the decentralization platform with strong participation by youth; and on (ii) employment generation and skills training for youth. In particular, the GP reintroduces the focus on the operationalization of the National Youth Fund and of Government outreach to Youth Association as a means to complement the focus on employment and education. It explicitly highlights the risk of perceived exclusion on the part of youth: “[…] our young people are aware of the opportunities in the world and many feel they are missing out on these opportunities.�27 However, going forward, and following the initial stabilization phase as outlined above, GoTL’s approach progressively shifted away from dedicated community-based interventions aimed at youth and towards the development of a broader National CDD platform targeting communities overall. It is not expected that operations similar to YDP will be implemented in the near future by GoTL. Lessons- learned from its implementation are none the less important for the development of the proposed PNDS. YDP results and challenges in implementation have highlighted key capacity constraints that have to be 26 The strategic areas of the 2010-2011 ISN are: (i) Strategic Area 1: Support for the Formulation of a National Development Strategy; (ii) Strategic Area 2: The Development of Capacity to Implement the National Economic Strategy; (iii) Strategic Area 3: Support for Short-Term Social Stabilization; and (iv) Strategic Area 4: Towards the Next CAS: Deepening the Foundations for Non-Oil Growth. 27 Program of the Fifth Constitutional Government (2012-2017), p 23 25 taken into account when establishing such community-driven programs and the importance of developing a solid implementation structure (staffing, M&E, FM and procurement management). It also highlights the on-going importance of reaching out to young people and addressing youth exclusion issues. A dedicated youth intervention in rural areas, however, is not likely to be the preferred approach of GoTL going forward. Government focus is expected to be on a CDD operation with overall outreach to communities. 3.2 Achievement of Project Development Objectives The overall objective of YDP was to promote youth empowerment and inclusion in development by expanding the capacities of and opportunities for youth groups to initiate and participate in community and local development initiatives. The project partly achieved its PDO with some shortcomings for the promotion of youth empowerment and inclusion at community level and limited progress made in empowerment and inclusion through the outreach of central government institutions. (Please see Annex 2 for an overview of key indicators). Given the PDO formulation, which focuses on issues of empowerment and inclusion in general terms, the assessment of project achievements presented below is based on an analysis of its two components, also relying on the available data for the project’s Key Performance Indicators and Intermediate Result Indicators. Mixed results towards the achievement of the PDO When considering the extent to which YDP’s PDO was achieved it is important to assess two complementary elements of the project’s objective to promote youth empowerment and inclusion. On the one hand, the project aimed to generate improvements in GoTL’s ability to reach out to youth and youth associations and thus address issues of youth exclusion in the overall nation-building process (Component 1). This was intended to be achieved through the establishment of a more effective central institution (SSYS), and a network of functioning youth centers based on the model developed under YDP. The other element of the project’s objective was to empower young people to participate in community level activities (Component 2). In this regard, the project aimed to create opportunities for youth inclusion and empowerment at the community level. By providing grants to youth groups at village level, the project aimed to enable youth to discuss their own priorities with community leaders, decide on the key activities they would like to see implemented and manage their execution (Component 2). Through this process, the project sought to bridge the gap between traditional leaders/authorities and young people. Both elements aimed to reduce the risk of further violent behavior by young people and contribute to the Government’s overall stabilization program. Reviewing available MIS data, key issues raised during supervision and the main findings of the ICR mission, available information has shown mixed results towards the achievement of the PDO with Component 2 being the more successful element of the project. The scope of Component 1 was significantly reduced at MTR with: (i) the operationalization of the NYF being dropped; and (ii) the scope of SSYS capacity building and technical assistance being reduced. The NYF was intended by GoTL to build the capacity of a broad set of youth groups through the provision of small grants. In that regard, it constituted an important element of YDP’s sustainability strategy and addressed the “supply-side� of the youth inclusion objective by creating a mechanism for central government outreach. As outlined above, weak capacity and institutional instability at SSYS 26 level meant that institutional strengthening could not be carried out as planned. In addition, with improvements in the overall security situation in 2008 and 2009, GoTL’s focus shifted to issues of youth employment and service delivery.28 This may have contributed to less attention/priority being given by SSYS to NYF key tasks under YDP. Therefore, while core capacity building activities were undertaken under Component 1 there were some significant limitations to their overall impact. In summary, the findings of the final Capacity Building Progress Review report29 indicated good progress in establishing key job descriptions and terms of reference for SSYS staff and in the implementation of leadership and management training (albeit through a limited number of sessions). The assessment, however, indicated the following important shortcomings in institutional capacity development: (i) the lack of systematic support for youth centers; and (ii) limited progress on improving data collection and overall management practices. These issues are reiterated in the ISR ratings and periodic assessments of institutional capacity development carried out during ISMs. Regarding Component 2, notwithstanding limitations with the sample size of the independent BA already highlighted, data collected indicates that YDP has had a positive impact on both: (i) generating increased opportunities for youth participation at community level; and (ii) ensuring that young people can effectively take advantage of these opportunities. Importantly, the data indicates that: (i) YDP had a positive impact in reducing the risk of violence in target areas; and (ii) that it resulted in a productive dialogue being established between young people and traditional authorities. As YDP was embedded within MSATM and used LDP as the vehicle for project implementation, it contributed to setting the village leadership in a position to initiate and mediate project activities. The youth interviewed also acknowledged this and explicitly stated that they would participate if called upon by the village leadership. BA findings highlight that YDP reduced the distance between the state and local youth, particularly those in rural areas for whom the state is generally very distant. Additionally, while most assessment participants across treatment and control groups placed a high priority on youth development, participants in YDP locations were more aware of national development agenda. Feedback provided by YDP participants through the BA highlighted some difficulties on the part of youth groups to effectively manage sub-project implementation. For the vast majority this was the first time they were given this opportunity and responsibility, which was very valued. Training was in some instances considered insufficient to manage the more complex aspects of project implementation (in particular financial management).30 Finally, there were important challenges with the implementation of capacity building activities planned for MSTAM. As highlighted in Section 2.2 above, the introduction of the PDD shifted MSTAM management attention away from YDP. Significant staff turn- over and difficulties in retaining staff in key positions, including that of PC, are likely to limit the extent to which lessons-learned from YDP may be introduced and followed-up in other MSTAM programs. 28 This notwithstanding periodic episodes of communal violence and outburst of urban violence in Dili during the project implementation period. 29 Bougas, W (2012), Capacity Building Progress Review Report: Sub-Component A: Institutional Strengthening, Youth Development Project (Secretariat of State for Youth and Sports, Government of the Democratic Republic of Timor-Leste). 30 While important support was sought and provided by village leadership, young people interviewed through the BA describe this support as occasionally “over-bearing.� 27 Overall, targets for intermediate project outcomes under Component 2, regarding youth participation during the design and implementation of sub-projects, were met (with achievements being only marginally less than end of project target in terms of youth participation). Additional analysis on each of the project components is provided under Annex 2. 3.3 Efficiency The nature of this project, which focused primarily on promoting youth empowerment through participation in community level activities, (as well as the compressed preparation time-frame), meant that no economic analysis was initially conducted. The team intended to collect information on the project’s economic impact using TF resources to be mobilized. These were, however, not secured and remaining supervision funding was required for other priority activities. Specifically The team chose to use its limited resources to collect data on youth participation as the key measure of the project’s impact given its focus on youth inclusion. In addition, YDP was a very innovative intervention in the context of Timor-Leste, with its focus on promoting youth participation through a CDD approach and using community sub-projects to: (i) build the capacity of young people; and (ii) improve linkages between youth and traditional authorities. While there are other interventions in Timor-Leste that focus on youth employment at the village level there are no other comparable programs that would allow an appropriate comparative cost-effectiveness analysis. Finally, the team considered comparing the cost of reaching out to youth using a community- based approach with that of conducting similar types of interventions through Youth Center-based activities. However, given the currently limited SSYS capacity to collect and process data on youth center usage, such analysis (that might provide insights/information on whether using existing MTSAM structures and a network of community facilitators might prove more/less cost-effective than building a network of youth centers to conduct outreach activities at district level) was not possible. Based on the currently available information it is, therefore, not possible to assess the efficiency of YDP. 3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory Project objectives remained relevant throughout implementation and in the current context in Timor-Leste. The project was aligned with the earlier GoTL stabilization plan and continues to be in line with the new Government Program (2012-2017) and the World Bank’s new ISN, which focuses specifically on the importance of addressing youth expectations for social stability. However, GoTL’s increasing focus on youth employment and basic service delivery, following the improvement in the overall security situation meant that the YDP experience is unlikely to be scaled-up by MSATM as 28 envisaged at appraisal. Lessons learned from project implementation have played a critical role in informing planned PNDS. At the time of writing it is, however, still unclear whether the new program will include a dedicated youth window once basic systems are well established. In terms of Component 1, the project put in place basic systems and functions at SSYS level and contributed to a greater management capacity on the part of YCMCs. Nevertheless the ability on the part of SSYS to extend the reach of collaboration with youth for development interventions was significantly limited: (i) in terms of the level and quality of the support provided to YCMCs; and (ii) the development of a functioning performance management system. In addition, as outlined above, the NYF support was dropped after the MTR to take into account changing SSYS priorities and capacity constraints. The absence of a GoTL mechanism for the continued delivery of small grants to youth significantly reduces the sustainability of YDP gains. The project’s Component 2 focused primarily on creating opportunities for youth participation in local development. When assessing the outcomes achieved by the project under this Component it is important to take into account that the M&E strategy adopted by the team allowed information to be collected only from a small fraction of the villages targeted by the project (six of 192 project villages). While this makes it difficult to determine whether the results observed in terms of improved participation in these sites are representative of overall project gains, MIS data on youth participation strongly supports the positive findings of the BA (Additional details are provided in Annexes 2 and 5). There were some important shortcomings in the achievement of project objectives under Component 2. While the project generated strong levels of youth participation there were challenges observed in terms of sub-project quality. Livelihood sub-projects, IT training activities and sporting activities (making up 61% of overall sub-projects) were found to be of uneven quality. This is likely to have an impact on future opportunities for youth participation. For example, where suco youth centers built were of poor construction quality, or too small for the needs of youth in the village, a negative impact is likely to also be felt in terms of youth participation in further activities that might have been carried out at these centers. Similarly, the lack of business plans for livelihood activities is likely to limit their “viability� and therefore the sustainability of youth participation. Secondly, as outlined by the BA, youth groups were established at community level for the purposes of the project implementation. These groups have only a temporary character linked to the project cycle and are unlikely to pursue further activities without additional external support. In addition, all Y-CIC members interviewed through the BA found that they would have benefitted from additional training throughout project implementation. Most youth were not skilled enough to undertake the actual construction of the projects they had proposed (typically sports fields) by themselves and were restricted to the role of unskilled labor on the projects. Finally, staff turn-over at MSTAM level and the shift in management attention to PDD implementation is likely to limit the extent to which lessons-learned on youth inclusion and empowerment may be taken forward as part of future CDD or local development interventions. Based on the analysis above, the project’s overall objective was met with some significant shortcomings, particularly in terms of sustainability of outcomes. 29 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development The BA yielded important insights on how the benefits from YDP were “distributed� among young people in the community. First, regarding the gender dimension of participation, in the absence of specific targets for female participation in YDP activities the majority of young people engaged in Y- CIC in treatment areas were men. Where women were involved they tended to work as treasurers for the group. The BA analysis highlighted that the project, overall, worked with established gender roles. Secondly, and as noted in CDD interventions globally, the benefits of participation (in terms of awareness raising, training, “empowerment’) were primarily gained by the group of volunteers directly engaged in sub-project management (Y-CIC). The BA further noted a common trend across treatment sites with more educated youth selected for Y-CIC positions. However, the assessment found no instances where specific groups were explicitly prevented from participation. Rather, the Y-CICs tended to comprise members from different previous youth groups, including the Martial Arts Gangs (MAG), but were not exclusive along those lines. No cases where encountered where the YDP had led to increased levels of conflicts or violence among youth. (b) Institutional Change/Strengthening As outlined above, the project explicitly sought to build institutional capacity for the implementation of the NYP, with modest results. Details on key achievements and shortcoming are provided in Section 3.2 above. (c) Other Unintended Outcomes and Impacts N/A 3.6 Summary of Findings of Beneficiary Survey (please see Annex 6 for additional details) A Beneficiary Assessment was implemented under the guidance of the World Bank team at project closure between September and October 2012. The BA focused on the following four themes: (i) Participation and Development Priorities; (ii) Empowerment and State-Society Relations; (iii) Youth Group Formation and Dynamics; and (iv) Youth Employment. It used qualitative research methodologies (in particular Focus Group Discussions31 and Key Informant Interviews). Information was used to collect information from six treatment32 and four control communities. 31 Focus Group Discussions were held with eight young people in each site (a mix of youth/participants/non-participants and other community members) 32 Treatment areas were Tohumeta, Fahiria, Fatubosa in Aileu and Holsa, Sanirin, Rairobo in Bobonaro while control communities were Maubisse and Liurai in Ainaro and Sau and Lakumesak in Manatutu. 30 Participating youth in the YDP villages showed a stronger understanding of the state, and of the way the state implements local development interventions, than youth from non-YDP villages. Youth (and others) in YDP villages were overall satisfied with YDP sub-projects, though some criticisms did arise, such as the perceived low budget allocation leading to uncompleted projects. The impact of YDP was particularly noticeable in more remote areas. YDP opened up opportunities for youth to genuinely participate in local development (i.e not offered by other government or non-government programs) in spite of the strong influence of traditional leaders. This was recognized and appreciated by both youth and the broader community during the BA. 4. Assessment of Risk to Development Outcome Rating: Substantial Regarding Component 1, gains in terms of SSYS capacity building were considered fairly new and easily reversible in the final capacity assessment conducted. With growing development partner interest in institutional capacity building activities with SSYS (particularly on the operationalization the National Youth Policy and the National Youth Fund where both UNICEF and GIZ are currently active) some of the YDP gains may be consolidated. This will require that SSYS play a stronger coordination role to avoid duplication of efforts. Regarding Component 2, there are significant issues with uneven quality and sustainability of sub-project investments, which are likely to have an impact on future opportunities for youth participation as outlined in section 3.3 above. In addition, there are no other interventions currently ongoing or planned that focus on youth empowerment at community level. There is a significant risk that training and structures put in place by YDP do not lead youth to a sustained engagement in community life and/or with local government. This risk may be to some extent off-set by the future development of the PNDS. , which could potentially integrate a specific youth inclusion dimension. Maintaining dialogue on youth concerns at the level of the Timor-Leste WB country office and in the dialogue with key development partners with a strong in-country presence will be key. 5. Assessment of Bank and Borrower Performance 5.1 Bank (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory In spite of a very compressed preparation time-frame, the task team was able to draw on in depth analysis of the key issues affecting youth in Timor-Leste, carry out consultations with key stakeholders and ensure that targeted pieces of analysis (in particular a capacity assessment of SSYS) were completed before project effectiveness. In addition to this analytical base, the WB was also able to incorporate in YDP design key lessons learned from earlier CDD and community-based program implementation. The design choice of using the existing LDP platform for the implementation of community/youth sub- project was particularly successful. However, as outlined above, the team may have somewhat 31 overestimated SSYS institutional capacity for the implementation of Component 1 resulting in an ambitious design. In addition, the Monitoring and Evaluation plan put in place did not fully take into account capacity constraints and relied on TF resources to be for its implementation. (b) Quality of Supervision Rating: Moderately Satisfactory Given the challenging implementation context and the high risk rating at appraisal the WB rightly identified the need for substantial resources to be allocated to project supervision. Periodic technical missions were carried out to complement semi-annual supervision missions. Key issues and implementation bottlenecks were systematically highlighted and clearly communicated to the implementing agency and to WB management via ISRs. There was thorough follow-up on agreements made, WB team recommendations and proposed next steps. Importantly, the composition of the technical staff (core team members) remained the same throughout project implementation thus ensuring a high degree of continuity and consistency in follow- up. Issues with the quality of investments, challenges with participation and training of field level staff were systematically discussed with the implementing agencies using available MIS data, ISM findings and the findings of short pieces of analysis (including a quality review of sub-project investments). Some of the key concerns raised by the team were: (i) low capacity of the technical officers, and the quality basic designs produced by the Verification and Appraisal Teams; and (ii) highly variable capacity of youth groups to implement infrastructure sub-projects; (iii) need for greater number of technical staff and improved advice on construction techniques to be provided to the Y-CICs. The World Bank team also dealt with M&E challenges by finding alternative solutions to the impact evaluation strategy initially proposed by: (i) adopting a lighter touch qualitative approach; and (ii) conducting targeted analysis on youth participation and local governance through a partnership with the Justice for the Poor Program, and (iii) strengthening the Project’s MIS. While these may not have generated statistically representative data on participation rates in project sites they generated extremely useful/insightful information for project implementation. There were, however, some limitations to the overall supervision model adopted largely due to changes in the availability of TF resources for Timor-Leste and staff-turn over at WB Country Office level. The plans made for implementation support at design took into account existing staff at Country Office level. In-country-staff33 was expected to play an important role in overall supervision on the technical elements of project implementation. Subsequent staff turn-over, the non-replacement of these positions posed some challenges for supervision. Reduction in TF availability had an impact in the World Bank Timor-Leste program more broadly. In the specific case of YDP, limited TF resources and the small size of the grant (and associated supervision budget) limited the options available to the team to provided technical assistance. Resources were not sufficient to mobilize qualified short-term consultants to provide in-country follow-up. In particular, the lack of experienced in-country FM staff that could provide ‘just-in-time� support was a shortcoming of World Bank supervision. Importantly, there was no Timor-Leste based 33 In particular, a Trust Funded Youth Officer Extended Term Consultant expected to provide daily support to project operations. 32 procurement staff that could support YDP before January 2012. In addition, with FM and procurement support provided through the Sydney office and FM staff turn-over at the Timor-Leste Country Office level there were some delays experienced in responding to the implementing agencies’ requests for FM advice as well as in providing NoL to procurement packages. During the project life-time the task team worked with three different procurement specialists and three different FM specialists. This affected the continuity of support provided to the implementing agency. In the Timor-Leste context where institutional capacity is limited this significantly constrained the team’s ability to identify potential challenges early on and work more directly with counterparts on identifying solutions. (c) Justification of Rating for Overall Bank Performance: Rating: Moderately Satisfactory The Bank's performance was moderately satisfactory overall, during project identification, preparation and implementation stages. Information gathered during preparation was effectively incorporated into project design. Initial design choices, however, resulted in a relatively complex design and M&E strategy that posed some difficulties for project implementation. While solid technical supervision was provided there were shortcomings with the support provided, particularly on FM and procurement. 5.2 Borrower Performance (a) Government performance Moderately Satisfactory While dialogue with Government remained extremely productive throughout project implementation significant changes in the ownership of the YDP approach throughout the project life-span. With the loss of key staff at SSYS and MTSAM (in the latter case diverted to other GoTL priorities) there were no alternative arrangements made to address important capacity gaps. Importantly, while challenges with the implementation of Component 1 were highlighted at project Mid-Term in October 2010, there was some reluctance on the part of GoTL to proceed with the proposed changes. This resulted in a delay in project re-structuring which was finally completed only in December 2011. (b) Implementing agency performance Moderately Unsatisfactory In spite of fragile institutional context and capacity challenges at both SSYS and MSATM level, implementing agencies were able to overcome significant operational constraints to implement and monitor YDP. Significant management shortcomings were, however, noted throughout the project’s life- span. Implementation Support Reports (ISRs) systematically highlighted delays in decision-making regarding the hiring and selection of staff and in taking corrective action to the FM and procurement issues emphasized by the WB team. As noted in the fiduciary compliance section both audits and IFR were frequently late. 33 High levels of staff turn-over posed considerable obstacles to project coordination with the Project Coordinator (PC) temporarily taking over FM and procurement functions (please see section 2.2 above). In spite of these operational challenges and persistent delays, SSYS was able to successfully complete essential capacity building activities planned under Component 1 and 289 sub- projects were developed and implemented by young people under Component 2. In addition, in spite of a slow start, functioning systems for project monitoring (MIS) were also established at MSATM level to collect information on sub-project completion and youth participation rates. Justification of Rating for Overall Borrower Performance: Rating: Moderately Unsatisfactory Taking into account: (i) changes in overall government ownership of the YDP approach; (ii) the systematic delays in project implementation and: (iii) limited responsiveness in addressing recurrent procurement and FM issues overall Borrower Performance is rated Moderately Unsatisfactory. 6. Lessons Learned (both project-specific and of wide general application) a. Lessons-learned with general applications There are four key lessons learned from YDP implementation that have wider implications for other operations in Timor-Leste and other operations in post-conflict contexts. The first has to do with the relevance of continuing to work with youth as decision-makers in fragile contexts (i.e. going beyond service provision and a focus on employment and education). As the data from the BA highlights, young people valued YDP in particular because of the leadership roles they were able to play in their communities. The feasibility of using CDD approaches to bridge the gap between local government and disenfranchised youth, and the positive impacts this has in reducing tensions at community level is a particularly important lesson. The BA was able to contrast YDP achievements in terms of empowerment and youth participation with the results obtained by the ILO support youth employment intervention and highlighted YDP’s unique role in promoting the empowerment of young people at community level. The assessment indicated that while the ILO support workfare program was likely to be more inclusive (offering more men and women from villages equal work for equal pay), participants had no decision-making power over project design and implementation. In this case youth were positioned as laborers, in contrast with YDP where they truly considered themselves to be decision-makers (and valued their role as such). The second is the importance of conducting targeted analytical work (possible even in a relatively short period of time) to inform design. The initial analytical work conducted by the team was instrumental in highlighting a broad range of issues putting youth at risk and identifying key capacity constrained. Where data collection took longer, the analysis was initiated and aimed to be completed before project effectiveness. 34 The third, point relates to the importance of a simple design. While the team acknowledged that a number of complementary operations would potentially need to be put in place to address different aspects of youth vulnerability, an effort was made to ensure that YDP itself addressed both short term and medium/longer term needs of young people in rural and urban areas through community level interventions and improvements in center-based services. Similarly, the range of sub-project activities at community level brought another layer of complexity to the project by allowing both small-scale infrastructure and livelihood sub-projects to be implemented. These two types of activities required very different types of technical assistance, placing an increased burden on limited numbers of district level staff. The amount of effort on the part of the implementing agencies and the WB team (in terms of supervision) required to make progress on any of these fronts was significant. While this was highlighted at the time of preparation, discussions with the broader CMU and the counterpart ultimately resulted in this “dual approach� being pursued. In this regard, the WB may have underestimated existing capacity challenges. While project activities were completed and set targets mostly met working of these very different fronts with small-scale funds is likely to have contributed to issues with sub-project quality and sustainability, limitations in the support provided to youth center activities and in the changes brought about in SSYS capacity and improvement in management systems. Concentrating efforts of both the borrower and the WB team on a smaller set of issues/front may have yielded better results by “going deeper� and helped develop a replicable/better documented model either in terms of outreach through Youth Centers or youth engagement in community development. A fourth related point has to do with the supervision model available for small grants of the size of YDP. The weakness in the institutional context in Timor-Leste helps to explain a great deal of the operational challenges during YDP implementation. Challenges in communication with district level staff, language barriers (which played a role in the transmission of recommendations from Aide- Memoires and Mission reports),34difficulties in selection and retention of qualified staff for key positions (FM, Procurement and M&E) required a re-doubling of supervision efforts. While at the time of design, the task team could count on a stronger in-country presence for follow-up (including a dedicated Extended Term Consultant) changes in staffing at CO level and in the availability of TFs meant that the team had to rely on more remote support and communications with the counterpart. The supervision model adopted and considered more successful (based on lessons learned from other WB operations) does rely strongly on additional TF resources, which help to ensure: (i) supervision by a larger CO based team with technical inputs sought from DC staff as needed; (ii) Technical Assistance provided directly to implementing agencies (by local or international consultants supervised by the WB team and often working directly in counterpart agencies). The feasibility of putting in place this type of supervision mechanism will be important to consider during the design phase of future operations. Building partnerships with donors with greater country presence for project supervision may also be a complementary strategy to consider. In the case of YDP closer partnerships with UNICEF and GIZ were pursued. However, given different areas of focus of the different institutions during the project implementation period it was not possible to advance this significantly. For future interventions, where such “overlap� of objectives exists, leveraging in-country resources would be advisable. 34 Where possible the task team worked with Bahasa speaking consultants, which helped to partly address language issues. 35 b. Project specific lessons-learned: capacity building and youth sub-projects The implementation of capacity building activities with SSYS indicated the importance of complementing training events with more regular on the job coaching. YDP’s final capacity assessment report highlighted the importance of a “continued� learning program to ensure greater impact of capacity building activities in SSYS performance. This approach is expected to create a strong connection between skills learned/additional skills needed and the roles/job of key staff members. A parallel can be drawn with the type of training provided to Youth Center Management Committees. While all considered the training provided to have increased their skills level, the extent to which they were able to use this new knowledge was often constrained by the limited set of activities YC run in the absence of additional donor funding. In one case, YC members trained had left to pursue other opportunities. As the YCMC are volunteers this is likely to be a recurring situation, which could be partly addressed by placing greater emphasis on “on the job training.� A number of lessons-learned have been generated through regular implementation support and the BA findings on the design and implementation of Youth Sub-Projects as follows: 1. Engaging youth in the implementation in community level activities brought about a sense of connection with local government and traditional authorities and of responsibility for seeing sub- project implementation through. The youth groups established for sub-project management were temporary but while in place included representation from a broad section of the community, which is likely to have had a positive impact on reducing tension among youth groups. The participatory elements YDP (including the direct management of financial resources) set it apart from other interventions as clearly outlined in the BA. 2. Revising/further increasing the funding allocation formula for future interventions (based on the YDP experience) would be important to improve the quality of future CDD interventions. The initial level of funding was found to be insufficient to implement sub-projects that addressed the expressed needs of youth. Funding levels were therefore revised up from US$1.84 to US$2.5 per capita. Supervision noted, however, some persistent issues with the budgeting of proposals with some instances of unfinished sub-projects observed (due to lack of funding). In addition, implementation experience also highlighted the importance of strengthening technical capacity at district level both for the development of sub-project proposals and the supervision/quality control of implementation. 3. Further strengthening the training of youth facilitators and deepening the initial situation analysis, would be important to address issues of possible elite capture identified through the BA. While youth obtained a significant degree of independence vis-à-vis traditional authorities at village level, the latter’s influence was still strongly felt (particular in steering the implementation of livelihood activities). Ensuring that youth groups are able to benefit from up to three project implementation cycles may also help build capacity and more independent decision-making power. 4. Specialized technical assistance for livelihood activities is particularly relevant in contexts such as Timor-Leste where district level staff may have limited technical expertise. While 36 employment is a key concern of young people the introduction of income generation activities introduced additional complexity in project design. Advice to youth groups on how to conduct market analysis, assess the potential profitability/sustainability of proposed activities and on how to develop a business plan would be particularly important. Additional social accountability measures may also be required to ensure adequate controls are maintained as the use of project funds for private goods generates additional challenges. Finally, YDP generated important lessons learned on Monitoring and Evaluation (M&E). Adopting a more modest, basic M&E strategy using available resources may be more prudent. Small scale interventions of this nature are likely to benefit greatly from targeted studies and process evaluations that can generate results early on and can be very useful to inform timely corrective action. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The completion report prepared by MSATM and SSYS (please see additional details in annex 7) is very appreciative of the overall technical support provided by the task team. It highlights some of the challenges with the support provided by the World Bank team on financial management and procurement issues as well as in meeting WB FM, disbursement and procurement requirements. The World Bank team is in agreement with the assessment made by the implementing agencies in this regard. The report further highlights that while YDP provided important support to some of the challenges faced by youth, a number of outstanding issues and challenges remain, most notably high levels of youth unemployment. The report also stresses the renewed focus by SSYS in the operationalization of the National Youth Policy and National Youth Fund with the support of additional development partners. (b) Co-financiers – N/A (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) – N/A 37 ANNEXES Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ million equivalent) Appraisal Estimate Actual /Latest Estimate Percentage of Appraisal Components (US$ million) (US$ million) 1. Operationalizing the National 55.8% 0.52 0.29 Youth Policy 2.Youth for Local Development 1.598 1.828 114.4% Total Baseline Cost 2.118 2.118 Physical Contingencies 0 0 Price Contingencies 0 0 Total Project Costs 2.118 2.118 Project Preparation Facility (PPF) 0 0 Front-end fee (IBRD only) 0 0 Total Financing Required 2.118 2.118 (b) Financing Appraisal Actual/Latest Percentage of Source of Funds Type of Financing Estimate Estimate Appraisal (US$ million) (US$ million) Borrower 0 0 0 IDA 2.118 2.118 100% 38 Annex 2. Outputs by Component a. Operationalizing the National Youth Policy The scope of this component was pro-actively reduced following the MTR. As outlined in Sections 2.1 and 2.2 above, the initial design was somewhat ambitious particularly regarding longer term capacity building objectives. While the challenges with institutional capacity likely to emerge during implementation may have been under-estimated (particularly with regards to the implementation of the planned Public Expenditure Review) the corrective re-structuring undertaken after the Mid-Year Review allowed the team to focus on essential capacity building activities, taking into account the new reality on the ground. The NYF related activities were dropped taking into account SSYS capacity constraints. YDP, therefore, focused on two key interventions: (i) capacity building for its staff to improve overall management capacity; and (ii) increased management capacity of the Youth Centers Management Committees (YCMC). First, the final capacity assessment report issued in April 201235 highlighted some significant limitations in the establishment of the SSYS performance monitoring system. Importantly, for purposes of assessing overall project performance, the report highlighted the limited impact that capacity building programs had on SSYS performance due to a strong reliance on one-off training events versus the use of a more “hands-on�/on the job training approach. The assessment considered that the latter may have been more effective in ensuring that learning be more directly relevant to SSYS operations. Significantly, additional capacity building work was considered necessary to effectively operationalize SSYS performance monitoring system in particular to produce Key Performance Indicators (KPIs) that effectively measure the institution’s performance. Secondly, all Youth Center Management Committees (interviewed during the ICR mission) reported that the training provided as part of YDP had increased their management and leadership skills. Improvements in Youth Center management at field level were, however, primarily a function of individual YCMC resourcefulness, external partnerships and donor funding with the range of basic activities implemented varying significantly. Basic operational funding provided by SSYS (US$500 monthly) was frequently received with delays and was insufficient to cover basic equipment maintenance or fuel cost (all centers visited during the ICR mission relied to some extent on generators for power supply). Lack of regular contact with SSYS and of a clear understanding of overall priorities and criteria used for the allocation of funding (including that provided by external partners) remained important concerns of all YCMC visited. b. Youth for Local Development Overall targets for community-level youth empowerment activities were met with strong youth participation and high levels of satisfaction regarding the infrastructure put in place. In spite of the limitations of the methodology used to assess overall opportunities for participation by young people 35 Data used here is drawn from the capacity assessment implemented at project closure 2012. The analysis was carried out by an independent consultant who assessed achievements in relation to the original capacity building interventions planned with SSYS. It was based on a review of the implementation of training plans and a series of interviews with key SSYS staff. 39 and actual increases in participation rates in community development the Beneficiary Assessment (BA) provided significant insights into the empowerment process established at the local level. The BA analysis highlighted the specific role of YDP in creating opportunities for genuine youth participation notwithstanding the strong influence of traditional leaders at community level. YDP extended established governmental structures down to aldeia level. While village leadership was found to have strong influence in the overall process and youth were still able to express their will and preferences through the YDP process. Interestingly, while some aspects in project selection and implementation (like the choice of use of paid or voluntary labor, or the choice of location for small scale infrastructure) were influenced by village leaders, other aspects involved a high degree of autonomy, such as procurement and management of projects’ on-going uses. The assessment was also able to contrast YDP achievements in terms of empowerment and youth participation with the results obtained by the ILO support youth employment intervention and highlighted YDP’s unique role in promoting the empowerment of young people at community level. The assessment indicated that while the ILO support workfare program was likely to be more inclusive (offering more men and women from villages equal work for equal pay), participants had no decision- making power over project design and implementation. In this case youth were positioned as laborers, in contrast with YDP where they truly considered themselves to be decision-makers. 40 Results Framework, Overview by Component (following October 2012 Mid-Term review) Key Performance Indicator D=Dropped End of project End of project Source of data C=Continued target achievement N=New R=Revised % of targeted communities where R 60% 100% (data collected from Beneficiary Assessment youth feel that there are more six treatment communities) opportunities for youth participation in community and local development % targeted communities that report R 60% 100% (data collected from Beneficiary Assessment increased youth participation in six treatment communities) local and community development activities % improved Secretariat of State for C Moderately Moderately unsatisfactory Based on Capacity Youth and Sports performance satisfactory Building Progress Review ratings – 2012 Report prepared by an independent consultant Output indicator D=Dropped End of project End of project Source of data C=Continued target achievement N=New R=Revised Component 1 % change in the execution of D programmed state budget for youth programs % of National Youth Fund D disbursed to youth groups Performance Monitoring System1 C System Functioning System Functioning but Final capacity assessment for SSYS designed and functioning implemented with report (SSYS) significant shortcomings % of YCMC members from R 60% 100% FGD and Interviews with targeted Youth Centers that report YCMC undertaken during an increase in leadership, ICR Mission management and planning skills Component 2 % of youth participating in youth N 30% 22% MIS decision-making meetings in targeted communities Number of YDP subprojects N 200 289 MIS implemented by youth in targeted communities % of youth participating in the R 20% 17% MIS implementation of youth development projects in targeted communities % of youth in FGDs in targeted R 70% 100% Beneficiary Assessment communities who are satisfied with the goods and services delivered through the YDP youth Grants % of initiatives funded by Youth D Grants that continue after funding expires 41 Annex 3. Economic and Financial Analysis The nature of this intervention, focused primarily on promoting youth empowerment through participation in community level activities, (as well as the compressed preparation time-frame), meant that no economic analysis was initially conducted. The team intended to collect information on the project’s economic impact using TF resources to be mobilized. These were, however, not secured and remaining supervision funding was required for other priority activities. Specifically, the team chose to use its limited resources to collect data on youth participation as the key measure of the project’s impact given its focus on youth inclusion. While data from CDD operations both in the Region and in Timor- Leste can provide proxy data on cost-benefit of YDP sub-project investments, the nature of these sub- projects (primarily sporting facilities, recreational activities and small-scale livelihood investments) do not lend themselves to an easy comparison with other interventions focused on productive investments. Secondly, YDP is a very innovative intervention in the East Timor context, with its focus on promoting youth participation through a CDD approach and using community sub-project to: (i) build the capacity of young people; and (ii) improve linkages between youth and traditional authorities. While there are other interventions in Timor-Leste that focus on youth employment at the village level there are no other comparable programs that would allow an appropriate cost-effectiveness analysis. Thirdly, the team considered comparing the cost of reaching out to youth using a community- based approach with that of conducting similar types of interventions through Youth Center-based activities. However, given the currently limited SSYS capacity to collect and process data on youth center usage such analysis (that might provide insights/information on whether using existing MTSAM structures and a network of community facilitators might prove more/less cost-effective than building a network of youth centers to conduct outreach activities at district level) was not possible. Based on the currently available information it is, therefore, not possible to assess the efficiency of YDP. 42 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility/ Specialty Lending Ingo Wiederhofer Senior Social Development Specialist EASER Task Team Leader Jose Mousaco Consultant EASHD Youth Samuel Clark Consultant LEGJR Monitoring and Evaluation Angela Nyawira Khaminwa Social Development Specialist EASER Social Development Markus Kostner Social Development Sector Leader EASER Youth at Risk John Butler Lead Social Development Specialist ECSOQ Social Safeguards James Monday Senior Environment Specialist EASTS Environmental Safeguards Cynthia Dharmajaya Program Assistant EASER Overall preparation support Maria Belo Leite Program Assistant EACDF Overall preparation support Evelyn Villatoro Senior Procurement Specialist EASR1 Procurement Supervision Ingo Wiederhofer Senior Social Development Specialist EASTS Task Team Leader Jose Mousaco Consultant EASHD Youth Samuel Clark Consultant EASER Monitoring and Evaluation and overall supervision Angela Nyawira Khaminwa Social Development Specialist EASER Social Development Sladjana Cosic Social Development Specialist EASER Social Development Sonya Woo Senior Social Development Specialist EASNS Social Development Rodrigo Serrano-Berthet Senior Social Development Specialist EASNS Youth at risk Evelyn Villatoro Senior Procurement Specialist EASR1 Procurement Manuela da Cruz Procurement Specialist EASR1 Procurement Jacinta Bernardo Consultant EACDF Financial Management Miriam Witana Procurement Specialist EASR1 Procurement David Whitehead Financial Management Specialist EASFM Financial Management Olivio dos Santos Program Assistant EACDF Financial Management support Cynthia Dharmajaya Program Assistant EASER Overall supervision support Maria Belo Leite Program Assistant EACDF Overall supervision support James Monday Senior Environment Specialist EASTS Environmental Safeguards 43 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle No. of Staff Weeks US$ Thousands (including travel and consultant costs) Lending FY07 6.04 74,943.69 FY08 31.83 286,337.36 TOTAL: 37.87 361,280.05 Supervision/ICR FY09 8.04 113,539.25 FY10 10.28 94,007.91 FY 11 10.81 145,248.74 FY12 16.17 129,306.25 FY13 8.88 84,690.53 TOTAL 54.18 566,792.68 44 Annex 5. Beneficiary Survey Results A Beneficiary Assessment (BA) was implemented under the guidance of the World Bank team at project closure between September and October 2012. The BA focused on the following four themes: (i) Participation and Development Priorities; (ii) Empowerment and State-Society Relations; (iii) Youth Group Formation and Dynamics; and (iv) Youth Employment. It used qualitative research methodologies in particular Focus Group Discussions36; and Key Informant Interviews. Information was used to collect information from six treatment and four control communities. Treatment areas were Tohumeta, Fahiria, Fatubosa (in Aileu) and Holsa, Sanirin, Rairobo (in Bobonaro) while control communities were Maubisse and Liurai (in Ainaro) and Sau and Lakumesak (in Manatutu). Selection of sites took into account high and low participation rate, location (including at least one remote village) and was also aimed at covering a variety of project types (physical construction projects, productive livelihood projects, and projects which involved purchase of ready-made equipment such as sewing machines and computers). Finally, pairing of treatment and control areas was done taking into account underlying levels of tension using information on crime and violence. 37 A total of 39 respondents provided information on the key themes above. In treatment areas the following categories of informants were systematically consulted: (i) village chief (chefe suco); (ii)Y-CIC member; (iii) non- Y-CIC youth; (iv) and non-youth community member. Overview of selected communities for BA District Aileu Historically Low Youth Tensions Suco Participation % Project types Notes Cycle 1 Cycle 2 Cycle 1 Cycle 2 Tohumeta 16.2 N.D. Chicken Husbandry; Sewing machines; Lower participation. Goat Husbandry; Corn milling machine Sub-district Laulara, Sports Equipment close to Dili. Fahiria 33.2 N.D. Youth center Volleyball court Higher participation. Sub-District Aileu, close to Aileu Vila Fatubosa 24.5 13.8 Youth center; Computer training; Remote. Sub-district Volleyball court sports equipment Aileu. 36 Focus Group Discussions were held with eight young people in each site (a mix of youth/participants/non-participants and other community members) 37 Bobonaro and Ainaro are areas where tensions among youth have been relatively high, while they have been low in Aileu and Manatutu. 45 District Bobonaro Historically High Youth Tensions Suco Participation % Project types Location Cycle 1 Cycle 2 Cycle 1 Cycle 2 Holsa 1.5 1.8 5 Volleyball Courts Volleyball Court; Lower participation. Drainage; Youth Sub-district Maliana. Centre Sanirin 4.8 7.3 Fishing Equipment; Fishing Equipment; Remote and low Sports Equipment Water Pump; Corn participation.38 Sub- Milling Machine District Bobonaro.. Rairobo 20.8 22.3 Small Bridge; Sewing Chairs & Generator; Remote and high Machine; Volleyball Small Bridge participation. Sub- Court district Atabae. Control Districts High Tensions Low Tensions Ainaro Manatutu Peri-urban Maubisse Sau Remote Liurai Lakumesak Key findings39 Participating youth in the YDP villages showed a stronger understanding of the state, and of the way the state implements local development interventions, than did youth from non-YDP villages. This greater level of awareness may lead to greater independence in decision-making in the future (although his hypothesis would need to be verified through further analysis). Youth (and others) in YDP villages were overall satisfied with YDP, though some criticisms did arise, such as the perceived low budget allocation leading to uncompleted projects. The impact of YDP was particularly noticeable in more remote areas. In more accessible communities, there was a wider range of development activities and educational opportunities which contribute to an overall higher level of understanding of development projects among the youth. YDP opened up opportunities for youth to genuinely participate in local development (i.e not offered by other government or non-government programs) in spite of the strong influence of traditional leaders. This was recognized and appreciated by both youth and the broader community during the BA. YDP extended established governmental structures down to aldeia level. While village leadership was found to have strong influence in the overall process and youth were still able to express their will and preferences through the YDP process. Many previous studies highlight that village leadership is crucial to the way that local development is carried out in Timor-Leste. In all the six YDP villages visited, the chefe suco and/or chefe aldeia influenced the selection and implementation of projects to some extent. In control villages, other types of local development projects were invariably led or influenced by village leadership. The extent of the influence on the part of village leadership ranged from providing advice and support to the Y-CIC to meet administrative requirements to a 39 The overall positive rating of the YDP should be understood in a context where there are few initiatives for youth in the villages, and no other where the youth were the main implementers. 46 “capture� of the process to meet village leadership priorities (this was observed in Holsa where the focus of the sub-project was the implementation of sporting activities where other higher priorities were expressed by young people). 40 Interestingly, while some aspects in project selection and implementation (like the choice of use of paid or voluntary labor, or the choice of location for small scale infrastructure) were influenced by village leaders, other aspects involved a high degree of autonomy, such as procurement and management of projects’ on-going uses. Finally, “members of Y-CIC, without exception, felt a strong sense of responsibility about their projects. That is, Y-CIC members thought of themselves as decision-makers. And moreover, they thought of themselves as decision-makers in a government program, a program that linked their priorities with the priorities of the state.�41 The beneficiary assessment also yielded important insights on how the benefits from YDP were “distributed� among young people in the community. First, regarding the gender dimension of participation, in the absence of specific targets for female participation in YDP activities the majority of young people engaged in Y-CIC in treatment areas were men. Where women were involved they tended to work as treasurers for the group. Secondly, and as noted in CDD interventions globally, the benefits of participation (in terms of awareness raising, training, “empowerment’) are primarily felt by the group of volunteers directly engaged in sub-project management (Y-CIC). The BA further noted a common trend across treatment sites with more educated youth selected for Y-CIC positions. There were significant variations in how benefits were distributed in small-scale infrastructure sub-projects versus livelihood interventions. Most of the projects which focused on public facilities (volley ball courts for example) facilitated inclusion as they are accessible for all youth in the village. In most of the construction projects, youth beyond the Y-CIC were involved in providing volunteer work, which further facilitated the integration of youth across the village. Livelihood projects reached a smaller number of youth, and tended to be primarily driven by the Y-CIC members. The BA noted a tendency for a certain degree of elite capture of these projects. Typically as the village head placed a close family member in key roles in the Y-CIC. Given the potential to increase personal income, there is a higher premium on being member of a successful Y-CIC of a livelihood project. The assessment found no incident where specific groups were explicitly prevented from participation. Rather, the Y-CICs tended to comprise members from different previous youth groups, including the Martial Arts Gangs (MAG), and were not exclusive along those lines. No cases where encountered where the YDP had led to increased levels of conflicts or violence among youth. In all villages, treatment as well as control sites, village leadership, district authorities and the youth themselves reported on lower level of conflict compared to before the YDP, although this progress was not ascribed primarily to the YDP but rather to a combination of factors as discussed below. YDP was found to have occasionally contributed to reducing tensions at community level. Skills training and capacity building interventions at the local level. Skills were imparted to participating youth in YDP through two main avenues: (i) training on sub-project development and implementation; and (ii) livelihood activities/sub-projects that focused on income generation. 40 A detailed analysis is provided in the YDP Beneficiary Assessment p28. 41 YDP Beneficiary Assessment p 37 47 Skills gained through sub-project development: Importantly, the BA highlighted some difficulties on the part of Y-CIC youth in meeting YDP requirements particularly in terms of financial management. One village leader commented that it took the youth more than one project cycle to understand the requirements of the project proposals. Due to the high mobility of youth and the transitional nature of the Y-CIC, there might have limits to the extent to which youth were able to truly become familiar with YDP requirements for sub-project management. Some Y-CIC members in the second cycle of project implementation explicitly mentioned that they had approached former Y-CIC members from the first cycle to assist in making the proposals and the reports. The process of making a proposal, managing a project and implementing did provide new skills to some the involved youth. The information collected through the BA does not allow the team to make an assessment as to whether young people are able to apply these project management skills in other development activities. Nevertheless, youth who had participated in the Y-CIC groups felt more comfortable in engaging in discussions on the general development of their villages and had a better understanding of development priorities. Skills and employment through livelihood sub-projects: While YDP was not conceptualized as a youth employment project, the BA focused on understanding preoccupation with employment issues at community level. These were found to be central among young people, particularly in communities closer to urban areas where education levels tend to be higher and so do youth expectations. Overall the YDP projects were found in the BA to have been too small to have an employment (sustainable self- employment) effect. Nevertheless, several village leaders though that the YDP had contributed to changing the mindset of the participating youth and matured their involvement in the overall development of the village. 48 Annex 6. Stakeholder Workshop Report and Results Not applicable 49 Annex 7. Summary of Borrower’s ICR and/or Comments on Draft ICR The draft ICR report was shared with GoTL counterparts with a two week period allowed for comments. The implementing agencies provided no detailed comments to the report but expressed their no-objection to its content. A short Borrower Report was equally shared with the World Bank team highlighting the following: i. GoTL’s appreciation for the WB team’s responsiveness to youth at risk issues through YDP and the project’s role in reaching out to youth organizations at grass-roots level and involving young people as decision-makers in community level and youth center interventions; ii. The implementing agencies positive feedback regarding the technical support provided throughout project implementation (including quality of supervision and the sharing of good practices and international experiences that were considered particularly helpful); iii. Limitations in the guidance received from the World Bank team in terms of procurement and financial management which, together with difficulties in the processing of withdrawal applications, were considered to have contributed to significant delays in project implementation; and; iv. The remaining gap in Timor-Leste in terms of interventions that focus directly on youth employment issues. The implementing agencies also highlight the priority to be given to vocational training and employment policy going forward. 50 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not applicable 51 Annex 9. List of Supporting Documents 1. Project Concept Note, September 2007 2. Emergency Project Paper, May 2008 4. Financing Agreement, May 2008 5. Mid-Term Review Report, October 2010 6. Restructuring Paper, December 2011 7. Implementation Support Reports (from June 2009 to February 2012) 8. World Bank (2007), Timor-Leste’s Youth in Crisis: Situational Analysis and Policy Options 9. Timor-Leste, Youth Center Development Lessons-Learned (2009) 10.Youth Development Project: Final Beneficiary Assessment (2013) 11. Institutional Capacity Building Assessment: Secretariat of State of Youth and Sports (2008) 12. Capacity Building Progress Review Report: Sub-Component A: Institutional Strengthening SSYS (April 2012) 13. Youth Development Project – Technical Quality Review (January 2012) 14. Final Report Counterpart – Youth Development Project (October 2012) 15. Timor-Leste, World Bank Country Assistance Strategy (2000-2010) 16. Timor-Leste, World Bank Interim Strategy Note (2010-2011) 17. World Bank (2011) Local Governance and Community Development Initiatives: Contributions for Community Development Programs in Timor-Leste 18. World Bank (2010) Trust, authority, and decision making: findings from the extended Timor-Leste survey of living standards 19.World Bank (2010) Youth Perspectives on Community, Trust and Conflict 20. Government of Timor Leste, National Youth Policy (2007) 52 124º E 125º E 126º E 127º E Wetar Island I N DO N E S I A 10º N 10º N Romang Island TIMOR- LESTE Atauro Biquele Island a i t Alor Atauro S t r Island t a r Berau W e Com Cape Tei DILI Laivai Lautem Mehara Tutuala Rantar Maínal Cape Cutchá Island Baucau Jaco Bucóli Fuiloro Manatuto Laga Island DILI Metinaro Vemasse Afagua Lospalos DILI BAUCAU Luro cal Liquiça Laleia Manufai Sei Cape Maubara Bazartete Quelicai LAUTÉM Venilale Baguia Corimbala LIQUIÇA clo Railaco AILEU La MANATUTO Gleno Aileu Uatucarbau Iliomar Cape de Loré Fatu Beso Ermera Laclubar Ossu Savu S ea Atabae VIQUEQUE Hatolla Letefoho Maubisse Turiscai Uatolari Dilor Aliambata Marko Viqueque ERMERA Hatobuilico en C Sah Batugade BOBONARO Atsabe Tata Mai Lau Haotio Bé Aco iere Maliana (2963 m) Cape Deilubún 9º N Balibo Same Natarbora 9º N To Ainaro Welaluhu Naikliu d Bobonaro MANUFAHI n Mape AINARO Sakato l a Fatolulik Zumalai Hatoudo Pante Macassar s Lolotoe Betano I Cape OECUSSI COVA LIMA Beco Calétec Citrana Baqui r Tilomar Nítibe Bobometo o Suai m To 127º E Passabe i Kuala Lumpur Cape Tafara T Timor Sea T IM OR-L EST E INDON ES I A MAIN CITIES AND TOWNS DISTRICT CAPITALS 0 10 20 30 Kilometers NATIONAL CAPITAL 0 10 20 30 Miles RIVERS 8 MAIN ROADS JANUARY 2011 IBRD 33496R This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information DISTRICT BOUNDARIES shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, or any INTERNATIONAL BOUNDARIES endorsement or acceptance of such boundaries. 124º E 125º E 126º E