Public Disclosure Copy The World Bank Implementation Status & Results Report Capacity Building for MNAO through Establishing a Countinuing Professional Development Strategy and Program (P155305) Capacity Building for MNAO through Establishing a Countinuing Professional Development Strategy and Program (P155305) EAST ASIA AND PACIFIC | Mongolia | Governance Global Practice | Recipient Executed Activities | Investment Project Financing | FY 2016 | Seq No: 5 | ARCHIVED on 13-Jun-2018 | ISR32910 | Implementing Agencies: Mongolia National Audit Office, Ministry of Finance Key Dates Key Project Dates Bank Approval Date:22-Jan-2016 Effectiveness Date:04-Feb-2016 Original Closing Date:30-Aug-2018 Revised Closing Date:30-Aug-2018 Project Development Objectives Project Development Objective (from Project Appraisal Document) The main objective of the proposed project is to strengthen MNAO’s capacity, primarily in financial audit and audit firm contracting and oversight. To achieve this objective a comprehensive and systematic continuing professional development strategy together with a time-bound action plan that is needs-driven, tailored and sustainable will be developed and implemented. For this purpose, the current capacity strengths and gaps in financial audit and related areas will be independently assessed by a peer SAI. In parallel to this assessment, MNAO personnel will be surveyed and participate in focus groups to identify their requests for and perceptions of their professional development needs and alternative means to achieve these objectives. The results of the financial audit peer review and the knowledge and learning needs assessment to be carried out under this proposed project, as well as the 2010 performance audit peer review, will feed into development of a CPD strategy for the institution that will then be developed into a 3-year knowledgeand learning program. Furthermore, as part of the project, tailored training materials will be developed and actual training will be delivered to MNAO staff as per the CPD strategy and the knowledge and learning program. While the specific content and depth of topics to be covered under the CPD program are largely to be identified and tailored as per the assessments to be carried out,an early indication of capacity gaps reveals such areas as MNAO staff’s knowledge and understanding of the relevant standards, i.e. IPSAS and the newly translated ISSAIs, quality control, contracting and quality assurance of private audit firm engagements, and audit of the consolidated financial statements of the Government. Has the Project Development Objective been changed since Board Approval of the Project Objective? No PHRPDODEL Components Name Overall Ratings Name Previous Rating Current Rating Progress towards achievement of PDO  Satisfactory  Satisfactory 6/13/2018 Page 1 of 6 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Capacity Building for MNAO through Establishing a Countinuing Professional Development Strategy and Program (P155305) Overall Implementation Progress (IP)  Satisfactory  Satisfactory Implementation Status and Key Decisions Activity 1 has been fully completed with the MNAO peer review report by the UK NAO finalized and quality reviewed by the International Donor Initiative (IDI) of the International Organization of Supreme Audit Institutions (INTOSAI) that developed the methodology for reviewing performance of Supreme Audit Institutions. The final quality assured peer review report was delivered to the MNAO and has since been published and disseminated to the financial auditors. Since finalization of the report, the MNAO has developed a plan for implementing the key peer review recommendations (actions ranging from revisions to the legal framework to training staff in certain skills development) and the implementation of this plan has been satisfactory with the vast majority of the recommended actions already taken or in progress by the relevant departments of the MNAO. Activity 2 has also been completed. MNAO carried out the training needs assessments at various levels of the organization e.g. central and local branches. Relevant surveys and focus group discussions took place in January and a final report on results of the training needs assessment was submitted to the Bank in March 2017. The findings of the training needs assessment, coupled with the peer review recommendations, have helped shape the Learning and Development Strategy and plans for the MNAO as well as the associated training efforts under the project that are currently ongoing. Activity 3 on developing a strategy of sustainable continuing professional training program has also been completed. Draft Learning and Development documents including a strategy, a program and a detailed plan prepared with support from an international consulting firm was reviewed and commented by the management of MNAO and was subsequently finalized and approved on September 19, 2017 by the order no. A/164 of the Auditor-General of Mongolia. As part of the Learning and Development strategy developed and adopted, MNAO indicated its desire to have a Competency Framework developed that could be used in staff recruitment and performance appraisal. The project had funds available to accommodate this request with savings of prior international consultancies, so proceeded to engaging an international consultancy to help MNAO develop the requested Competency Framework. A draft of the document has now been received and is expected to be finalized by mid-July. Activity 4 on delivering priority training under the Learning and Development Strategy and Action Plan has commenced and efforts are ongoing. The MNAO has been developing a comprehensive annual training plan that has included training events and activities to be covered under the project in the areas identified as priority by the learning and development strategy and plan. Implementation of the training annual training plan and the associated training is currently ongoing. The MNAO established an online training facility with connection to all the 21 aimag branch offices with support of the project and this facility is currently in use for organization of the appropriate training. Efforts to develop training materials in line with the handbooks developed by the INTOSAI IDI are ongoing with the translation of the handbooks for all three types of audits (financial, performance and compliance) close to being concluded. The MNAO will continue focusing on delivering on the planned training to its auditors during the remaining project implementation period. Activity 5 on developing a transparent and documented procedure for outsourcing audits to private audit firms was also completed. With support from a local consultant, MNAO developed a “Regulation on selection, contracting, and evaluation of independent audit entities to conduct audit of financial statements, and monitoring the quality of performed audits” and obtained approval by the order no. A/191 of the Auditor-General on October 26, 2017. The MNAO has successfully carried out selection of audit firms for the FY2017 outsourced audits following this new regulation. In order to help monitor performance of the outsourced audits, MNAO is currently developing simple software under the project where the relevant audit firm and engagement-level information can be collected, stored, easily accessed, reviewed and evaluated by those who are exercising quality control and quality assurance for the outsourced audits from the MNAO’s side. The software is currently being developed and is expected to be in place by August 30, 2018. Risks 6/13/2018 Page 2 of 6 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Capacity Building for MNAO through Establishing a Countinuing Professional Development Strategy and Program (P155305) Overall Risk Rating Risk Category Rating at Approval Previous Rating Current Rating Overall  --  Moderate  Moderate Results Project Development Objective Indicators PHINDPDOTBL MNAO has made demonstrable progress in implementing the targeted improvements which emerge from the financial audit peer review (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 90.00 25.00 Date 11-Jan-2016 26-Jun-2017 06-Jun-2018 31-Aug-2018 Comments MNAO developed a dedicated plan of actions for implementing the peer review recommendations. As per a progress report received from the MNAO as at May 30, 2018, 90% of the recommended actions have now been implemented. PHINDPDOTBL MNAO auditors have been trained in the priority areas identified by the CPD strategy and action plans (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 -- 91.50 50.00 Date 11-Jan-2016 -- 06-Jun-2018 31-Aug-2018 Comments Various training as per the MNAO's Learning and Development Strategy and Plan are ongoing. The regional training series organized under the project in 2017 had the most coverage and already covered 91.5% of all state auditors in Mongolia. PHINDPDOTBL MNAO has utilized a strengthened procedure for selection of private audit firms contracted to conduct audits of government entities (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target 6/13/2018 Page 3 of 6 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Capacity Building for MNAO through Establishing a Countinuing Professional Development Strategy and Program (P155305) Value N N Y Y Date 11-Jan-2016 27-Jun-2017 06-Jun-2018 31-Aug-2018 Overall Comments All the PDO indicators have now been met. Intermediate Results Indicators PHINDIRITBL Apeer review report including recommendations for improvement of MNAO’s financial audit capabilities submitted (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value N Y Y Y Date 11-Jan-2016 27-Jun-2017 06-Jun-2018 31-Jan-2017 PHINDIRITBL Report on MNAO training needs assessment (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value N -- Y Y Date 11-Jan-2016 -- 06-Jun-2018 30-Jun-2017 PHINDIRITBL CPD strategy and an action plan have been developed and adopted (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value N N Y Y Date 11-Jan-2016 28-Jun-2016 19-Sep-2017 31-Aug-2017 6/13/2018 Page 4 of 6 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Capacity Building for MNAO through Establishing a Countinuing Professional Development Strategy and Program (P155305) PHINDIRITBL Materials developed for 50% of the courses included in the CPD action plan (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 50.00 Date 11-Jan-2016 28-Dec-2017 06-Jun-2018 31-Jan-2018 PHINDIRITBL Selection procedure for private audit firms that would perform contracted audits developed (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value N -- Y Y Date 11-Jan-2016 -- 26-Oct-2017 30-Jun-2017 Overall Comments All the intermediate results indicators, except for the one on training materials development, have now been met. The indicator "Materials developed for 50% of the courses included in the CPD action plan" has remained slightly difficult to measure given training materials are not all developed in advance but rather prepared in advance of the specific training events. This practice has found to be satisfactory. Data on Financial Performance Disbursements (by loan) Project Loan/Credit/TF Status Currency Original Revised Cancelled Disbursed Undisbursed Disbursed P155305 TF-A1838 Effective USD 0.38 0.38 0.00 0.27 0.11 72% Key Dates (by loan) Project Loan/Credit/TF Status Approval Date Signing Date Effectiveness Date Orig. Closing Date Rev. Closing Date P155305 TF-A1838 Effective 22-Jan-2016 04-Feb-2016 04-Feb-2016 30-Aug-2018 30-Aug-2018 Cumulative Disbursements 6/13/2018 Page 5 of 6 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Capacity Building for MNAO through Establishing a Countinuing Professional Development Strategy and Program (P155305) Restructuring History There has been no restructuring to date. Related Project(s) There are no related projects. 6/13/2018 Page 6 of 6 Public Disclosure Copy