AgRicuLTuRE & RuRAL DEvELOpmENT NOTES 53341 Tracking Results in Agriculture and Rural Development in Less-Than-ideal conditions iSSuE 46 mARcH 2009 A Sourcebook of Indicators for Monitoring and Evaluation WHy A SOuRcEBOOK cant impacts. The need to empirically demonstrate the Of iNDicATORS? impacts of projects has shifted the focus of M&E from a concentration on inputs and outputs to a concentration The demand for verifiable evidence of results and on outcomes and impacts. impacts of development agricultural programs and proj- ects is growing. However, most of the indicators that The ability to measure and demonstrate outcomes and development practitioners have traditionally used in impacts relies on the use of indicators that are based tracking progress toward achieving projects' objectives on reliable data and on the capacity to systematically focus on the workings of the development operation collect and analyze that information. The conditions in itself. These performance indicators relate mainly to which M&E are carried out vary widely, depending on lower-level inputs and outputs and are used to populate the demand for information, the extent to which it is management information systems. Higher-level indica- used to inform decision-making, and the reliability of tors are used to measure progress in achieving the ulti- the systems that are in place to capture and convey that mate objectives of projects, and in bringing about larger information. Throughout much of the developing world outcomes and impacts. These results indicators have these conditions are "less-than-ideal," and information become increasingly prominent in the wake of recent is irregular and often lacking altogether. In these condi- international resolutions such as the Paris Declaration on tions there is a lack of effective demand for information Aid Effectiveness in 2005 and the Monterrey Consensus on the part of policy makers. The conditions are often on Financing for Development in 2002. While no con- especially pronounced for data related to rural areas, flict exists between performance and results indicators, where the costs of data collection are high and the qual- and while effective monitoring and evaluation (M&E) ity of existing data is particularly low. Building data sys- systems track both--reassuring performance indicators tems and developing and supporting capacity for M&E in do not ensure the achievement of a project's larger these conditions is, therefore, a pressing imperative for goals. A project that is diligently monitored for financial interventions in the agriculture and rural development oversight and compliance with sound management and sector. Strengthening capacity for M&E begins at the performance principles may very well achieve no signifi- national and sub-national levels, where addressing the weaknesses of national statistical systems is a common priority. The data collected and reported within countries must not only be of sufficient quality to inform plan- ning and policy formulation but must also be consistent between countries. Standardizing the information col- lected by global databases facilitates comparisons across countries by international agencies such as the World Bank and the UN Food and Agriculture Organization (FAO) that monitor regional and global trends and reali- ties and track progress toward achieving the Millennium Development Goals. OBjEcTivES The FAO, the Global Donor Platform for Rural Development, and the World Bank, therefore, set out to compile a set of results-based indicators, including a number of core indicators to meet the most basic data requirements of international monitoring. The core indi- Photo: Tran Thi Hoa cators were also selected on the basis of their simplicity THE WORLD BANK in order to accommodate the limited resources and capac- indicators, for no such sequence applies. Results need ity of institutions responsible for M&E at the national and to be tracked throughout the project's implementation sub-national levels. The indicators are introduced in a co- so that corrective action can be taken mid-course--for publication of the three institutions titled Tracking Results instance, identifying intended beneficiaries who are not in Agriculture and Rural Development in Less-than-Ideal being reached and determining why. This tracking of early Conditions: A Sourcebook of Indicators for Monitoring results addresses a traditional weakness in M&E that is and Evaluation, which also presents a series of practical attributable to the time lag between when project out- recommendations based on experience in setting up M&E puts are provided and when higher-level outcomes are, systems where capacity and resources are limited. The or are not, achieved. The Sourcebook focuses on results Sourcebook is intended to serve as a resource for capac- indicators, including early outcome indicators. ity building. Systematically monitoring core indicators will lead to the capacity to monitor more sophisticated indica- THE DATA fRAmEWORK tors as monitoring institutions accumulate experience, as Tracking early outcomes requires data from clients. the information they collect becomes more reliable, and as Employing surveys to determine whether clients are being demand for that information increases. reached by a project's delivery of services and products is equated with market research and its measurement THE ANALyTicAL fRAmEWORK of customer satisfaction. Such surveys can track three Systematically measuring the impact of a development essential indicators that reflect the effectiveness of a program involves the application of an analytic or logical project's delivery: access, use, and satisfaction. What framework (logframe) in which indicators are classified as proportion of intended beneficiaries actually has access performance indicators and results indicators. In results- to the project's products and services? What proportion based systems, relatively greater weight is attached is actively using or adopting those products and services? to indicators that are used to measure outcomes and And among this group, what proportion of users is satis- impacts than to performance indicators, which are com- fied with the products and services? Surveys are optimally paratively cheap and easy to monitor. This represents a carried out by independent authorities rather than the departure from conventional M&E. service providers themselves. The institutions that are responsible for delivering the services can be usefully Performance indicators are used to measure the effec- surveyed to gauge results from the supplier's perspective. tive use of inputs to generate outputs and to compare Beneficiaries, on the other hand, are typically consulted the actual effects of the inputs to their expected effects through community focus groups, household surveys, and on outputs. Inputs are the financial, physical, and human rapid rural assessments. Employing these different instru- resources that are employed by the project to produce out- ments generates feedback from multiple perspectives and puts. Outputs are the project's products--the goods and enables monitoring agencies to cross-reference and trian- services produced by introducing the inputs. Monitoring gulate information from different sources. The surveys are performance by determining how effectively and efficiently used in addition to a variety of traditional data collection inputs are converted into outputs consists largely of book tools, such as population census, agricultural census and keeping and analyzing financial records to produce finan- surveys, household surveys, and community surveys. cial reports and data that are entered into management information systems. This information is used for cost-ben- THE iNSTiTuTiONAL efit analysis, to calculate the costs per unit of output, and fRAmEWORK a variety of input-output ratios that are used for financial M&E is an exacting and costly process that can seriously reporting and in periodic progress reports. tax the limited resources and managerial capacity of pub- lic institutions in low-income countries. Its demands lead Results indicators are generally classified as outcomes to a dilemma in which the countries that most urgently and impacts on beneficiaries or target populations. require M&E are also those that have the fewest resources Outcomes are changes in people's behavior--often available to undertake it. This familiar scenario suggests through their response to incentives--that result from support to capacity building among these monitoring their access or exposure to project outputs and their institutions as a logical priority for official development use. Optimally, these behavioral changes will advance assistance. The starting point for this assistance is most the intended goals or impacts of the project. Impacts are often the country's national statistical system (NSS), to the ultimate effects of the project, whether intended or which sectoral institutions such as agriculture ministries unintended. Monitoring these higher-level effects is sig- provide data. The coordination of M&E between the dif- nificantly more involved than examining the information ferent institutions and agencies that collect data at the internally available in financial and management informa- sector level requires purposeful coordination on the part tion systems, and entails soliciting information from cli- of the NSS and supervision to establish common reporting ents and beneficiaries about how the project has affected practices. In many countries national statistical develop- them. It is important to correct any misapprehension ment strategies are being undertaken to strengthen the that results indicators are monitored after performance NSS, and the Sourcebook identifies this as an opportune 2 entry point for aid that is intended to support capac- ment assistance directly to bear on the capacity for M&E ity building for M&E. The indicators detailed in the at these levels represents an important opportunity to Sourcebook are intended to serve as a basis for a strategic address the weakest links in their information sources. plan to improve agricultural statistics. The plan is to be presented to the United Nations Statistical Commission THE iNDicATORS in February 2010. The Sourcebook presents a list of 86 core indicators which are used to measure early-, medium-, and long- EmERgiNg iSSuES term outcomes. The list includes the core data require- The Sourcebook points to emerging challenges for M&E. ments needed to construct the indicators and the data The first challenge relates to decentralization. In many sources from which the information is derived. The developing countries the process of decentralization is first 20 indicators are sector-wide, followed by a list for underway, in which government roles and responsibili- monitoring agricultural and rural subsectors, including ties are devolving to local authorities. This implies greatly crops, livestock, fisheries and aquaculture, forestry, rural expanded sample sizes and the collection of data at microfinance and small and medium enterprise finance, lower levels of disaggregation in the surveys used to agribusiness, agricultural research and extension, and monitor results indicators ­ and imposes new demands irrigation and drainage. These are followed by a list of on sub-national authorities that often have little existing thematic indicators for community-based rural develop- capacity. It also imposes demands on the national statisti- ment, natural resources management, land policy and cal offices that rely on sub-national authorities for data. administration, and policies and institutions. Few such national offices are likely to see an expansion in the resources available to them commensurate with the Nineteen of the indicators are identified as priority indi- potentially dramatic expansion of sample sizes and num- cators, selected specifically as starting points for M&E bers of surveys. A second challenge relates to community in less-than-ideal conditions, based on their relative participation in M&E, particularly in projects that involve simplicity and the cost-effectiveness with which they can community-driven or community-based rural develop- be gathered. These indicators are also intended to meet ment. Decentralization and projects in which communities the most basic data requirements of international agen- assume responsibility for M&E both represent trends that cies responsible for global-level M&E. They are shown in will require new methods of data collection, analysis, and red in the following list. The complete list of indicators, new capacities to undertake them. From the perspective including data sources, core data requirements, and tech- of international development agencies, bringing develop- nical notes is provided in the Sourcebook itself. A. Sector-Wide Indicators for Agriculture and Rural Development Early outcome 1. Public spending on agriculture as a percentage of GDP from the agriculture sector 2. Public spending on agricultural input subsidies as a percentage of total public spending on agriculture 3. Percentage of underweight children under five years of age in rural areas 4. Percentage of population who consider themselves better off now than 12 months ago Medium-term outcome 5. Food Production Index 6. Annual growth (percentage) in agricultural value-added Long-term outcome 7. Rural poor as a proportion of the total poor population 8. Percentage change in proportion of rural population below US$1 per day or below national poverty line 9. Percentage of the population with access to safe or improved drinking water 10. Consumer Price Index for food items 11. Agricultural exports as a percentage of total value-added in agriculture sector 12. Proportion of under-nourished population 13. Producer Price Index for food items 14. Ratio of arable land area to total land area of the country 15. Percentage change in unit cost of transportation of agricultural products 16. Percentage of rural labor force employed in agriculture 17. Percentage of rural labor force employed in non-farm activities 18. Percentage of the labor force underemployed or unemployed 19. Annual growth rate of household income in rural areas from agricultural activity (percentage) 20. Annual growth rate (percentage) of household income in rural areas from non-agricultural activity B. Specific indicators for Subsectors of Agriculture and Rural Development 1. Crops (inputs and services related to annual and perennial crop production) Early outcome 21. Access, use, and satisfaction with services involving sustainable crop production practices, technologies, and inputs Medium-term outcome 22. Percentage change in yields of major crops of the country Long-term outcome 23. Yield gap between farmers' yields and on-station yields for major crops of the country 24. Percentage of total land area under permanent crops 2. Livestock Early outcome 25. Indicators of access, use, and satisfaction with respect to livestock services Medium-term outcome 26. Annual growth (percentage) in value-added in the livestock sector Long-term outcome 27. Livestock birth rate 28. Percentage increase in yield per livestock unit 29. Percentage change in livestock values Continued on page 4 3 Continued from page 3 3. Fisheries and Aquaculture Early outcome 30. Indicators of access, use, and satisfaction with respect to fisheries/aquaculture services 31. Water use per unit of aquaculture production Long-term outcome 32. Capture fish production as a percentage of fish stock 33. Share of small-scale fishers in the production of fish 34. Percentage of total permitted catch earmarked for local fishing communities as rights 35. Annual percentage change in production from aquaculture farms 4. Forestry Early outcome 36. Indicators of access, use, and satisfaction with respect to the forestry services: 37. Employment in forestry-related activities (full-time equivalents) 38. Value of removals of wood and non-wood forest products 39. Value of services from forests Medium-term outcome 40. Area of forest under sustainable forest management Long-term outcome 41. Percentage of land area covered by forest 42. Annual growth in rural household income from forest-related activities 43. Growing stock per hectare (m3/ha) of forest 44. Percentage rate of deforestation 5. Rural Micro and SME Finance Early outcome 45. Indicators of access, use, and satisfaction with respect to rural finance 46. Percentage of the rural population using financial services of formal banking institutions 47. Percentage of bank branches that are located in rural areas Long-term outcome 48. Percentage of total savings that are mobilized from rural areas 49. Percentage of rural population using non-bank financial services 50. Recovery rate of rural credit 6. Agricultural Research and Extension Early outcome 51. Indicators of access, use, and satisfaction with research and extension advice 52. Public investment in agricultural research as a percentage of GDP from the agriculture sector Long-term outcome 53. Percentage change in yields resulting from improved practices for major crops of the country 54. Change in farmer income as a result of new technologies (by gender) 7. Irrigation and Drainage Early outcome 55. Indicators of access, use, and satisfaction with respect to irrigation and drainage services 56. Irrigated land as a percentage of crop land 57. Percentage of users who report a significant increase in crop yields as a result of irrigation and drainage services 58. Service fees collected as a percentage to total cost of sustainable Water User Association (WUA) activities Long-term outcome 59. Percentage change in average downstream water flows during dry season 60. Percentage change in agricultural value-added created by irrigated agriculture 61. Percentage of irrigation schemes that is financially self-sufficient 62. Percentage increase in cropping intensity 8. Agribusiness (agricultural marketing, trade and agro-industry) Early outcome 63. Indicators of access, use, and satisfaction with respect to agribusiness and market services 64. Percentage change in number and value of activities managed by agro-enterprises 65. Percentage of agro-enterprises adopting improved/certified hygiene/food management system Medium-term outcome 66. Percentage change in sales/turnovers of agro-enterprises Long-term outcome 67. Percentage change in number of agricultural inputs outlets 68. Percentage increase in private sector investments in agriculture 69. Percentage increase in market share of cooperatives/agribusiness enterprises C. Indicators for Thematic Areas Related to Agriculture & Rural Development 1. Community-based Rural Development Early outcome 70. Access, use, and satisfaction with respect to services provided by community-based, rural development organizations 71. Percentage of farmers who are members of community/producer organizations 72. Proportion of community/producer organizations capable of meeting the production and marketing needs of their members 73. Proportion of producer organizations/NGOs with functional internal system of checks and balances 74. Percentage change in number of community associations exercising voting power in local government budget Long-term outcome 75. Percentage increase in number of local enterprises in rural areas 2. Natural Resource Management Medium-term outcome 76. Withdrawal of water for agricultural as a percentage of total freshwater withdrawal 77. Percentage change of land area formally established as protected area 78. Percentage change in soil loss from watersheds Long-term outcome 79. Percentage change of farm land under risk of flood/drought 3. Land Policy and Administration Early outcome 80. Percentage of land area inventoried 81. Percentage of land area for which there is a legally recognized form of land tenure Long-term outcome 82. Percentage change of land over which there are disputes 83. Percentage of agricultural households that have legally recognized rights to land 84. Percentage change in number of formal land transactions (quarterly or yearly basis) 85. Percentage change in land access for women and minority groups 4. Policies and Institutions Long-term outcome 86. Ratio of average income of the richest quintile to the poorest quintile in rural areas This ARD Note was prepared by Gunnar Larson, under the guidance of Sanjiva Cooke and Nwanze Okidegbe, of the Agriculture and Rural Development Department of the World Bank. It is based on Tracking Results in Agriculture and Rural Development in Less-than-Ideal Conditions: A Sourcebook of Indictors for Monitoring and Evaluation prepared by a joint team of staff from the World Bank and FAO for the Global Donor Platform for Rural Development. THE WORLD BANK 1818 H Street. NW Washington, DC 20433 www.worldbank.org/rural