1. Introduction

On January 1, 2016, the world officially began the implementation of the 2030 Agenda for Sustainable Development—a transformative plan of action based on 17 Sustainable Development Goals (SDGs)—to address urgent global challenges over the next 15 years. It seeks not only to eradicate extreme poverty, but also to integrate and balance the three dimensions of sustainable development— economic, social and environmental—in a comprehensive global vision. It is necessary that international development begins with a sense of opportunity and purpose based on an accurate perception of where the world stands now. The SDG agenda was developed based on the Millennium Development Goals (MDGs) to enhance aid effectiveness and provide a robust result-based management with 232 indicators.

Korea has been increasing their ODA budget as an OECD Development Assistant Committee (DAC) member; however they are still in the process of establishing its solid legal and strategic foundation across its development cooperation system to improve aid effectiveness. However, OECD Peer review (2012) pointed out the absence of a clear strategy to measure aid effectiveness, which should be addressed in project programming and implementation to meet the desired outcomes. Therefore, to enhance Korea’s aid effectiveness and consistent policy, measurable project management by utilizing SDG indicators is crucial to have positive impacts from development cooperation projects.

In general, result-based management, which involves the whole process of setting up a project plan such as budgeting and allocating resources focusing on results, can be achieved with periodic monitoring and evaluation (Hong & Yeon, 2014). In the past, while international organizations such as USAID, DFID, and AfDB have set standard indicators for each program, Korea did not have a concrete plan to measure performance. However, with the 232 SDG indicators, it is necessary to utilize these indicators to measure result and investigate the extent to which performance has contributed to SDGs.

There are very few cases on the use of SDG indicators to evaluate past or current ODA projects. However, the Sustainable Development Solutions Network (SDSN), in 2016, proposed a preliminary set of SDG indicators and dash board for all developed and developing countries to measure SDG achievement across 17 Goals. This aims to help stakeholders identify the most urgent priorities in each country and region.

In 2017, the OECD Development Cooperation Directorate (DCD) discussed whether the results of development cooperation contribute to the SDGs or if the SDG targets and indicators can strengthen the dialogue on development cooperation. Also, when the result-based framework incorporates a coherent set of goals, targets and indicators discussed on the 2030 Agenda, it would facilitate a common results focus and monitoring system for development cooperation. Even though most DAC member countries created a system to measure data and results, they still struggle to identify clear objectives and select measurable indicators for comprehensive approaches.

In results management and measurement systems, the challenges most often encountered by donors in a resultbased framework are incomplete, lack of available data and relevant indicators. While each country has its own priority and strategy, in order to build up a result-based management framework for an ODA project with consistency, selecting appropriate indicators and developing measurement methods based on the Sustainable Development targets and indicators is important.

In the case of agriculture and rural development projects, they include a considerable part of cross-cutting elements. Even though there are several indicators, quantitative and measurable indicators are rarely used in these areas.

Therefore, this paper suggested how to measure results and outcomes of ODA projects by using the SDG indicators and identified compatible indicators from international organizations based on a survey of agriculture experts. From these indicators, a result-based management was suggested to measure the contribution to SDGs of a rural development project in Bolivia.

2. Conceptual Framework

2.1. Existing indicators applicable to SDGs targets

Choosing appropriate measurements or indicators for the result-based framework is crucial. Indicators consisted of qualitative or quantitative factors and variables that provide simple and reliable means to measure achievement. They also specify what is to be measured, determine how expected results are measured and what data should be collected. Using similar indicators over a period of time also tends to provide consistent measurement.

In order to identify the compatibility of the indicators used in existing international organizations with the SDG objectives and targets, 25 Korean experts were surveyed. Table 1 shows existing indicators from international organizations such as DFID, USAID, Korea International Cooperation Agency, FAO and World Bank that are possibly linked to SDGs.

For SDG #1, 'the ratio of the population living below the international poverty line', was an applicable index to SDG Target 1.1, which refers to absolute poverty eradication. There are indicators from KOICA and SDSN that can be linked with detailed Target 1.1 such as the indicator ‘absolute poverty level of less than $ 1.25’, which is the ratio of people with income less than $ 1.25 per day (%). In SDG Target 1.2, the definition of the national poverty line is emphasized, and the World Bank and DFID have also used indicators that use the relative concept of the national poverty line rather than the poverty line. Also, the USAID indicator on the number of households in the vulnerable group who directly benefited from donor organizations' support was a similar indicator to SDG Target 1.3. There was also a link between SDG Target 1.4 and existing indicators of international organizations such as DFID and USAID. For example, DFID measures the number of farmers who can benefit from agricultural services and investments, and the proportion of rural population with access to financial services by banking institutions. USAID also made it possible to measure the accessibility of micro-finance and new technology through indicators such as 'the number of SMEs with bank loans' and 'the number of farmers applying new technology or farming'.

For SDG #2, the Inter-Agency and Expert Group (IAEG) indicators such as 'agricultural productivity per unit of labor' and 'average income of small food producers' were found to be related, as well as World Bank’s ‘Crop Production Index and Grain Rank (kg / ha)’ indicator. In addition, KOICA measures increase in income by applying 'farm income index' and 'income growth index' to Project Design Matrix (PDM), which can be linked to SDG target 2.3.

SDG Target 2.4, which refers to the implementation of sustainable food production system and farming methods, existing indicators that found to be compatible with was KOICA's ‘area of s ustainable agriculture farmland area (ha)’. In terms of expansion of investment in establishment of plants and livestock gene bank (detailed SDG Target 2.a) for strengthening international cooperation, indicators found to be similar were USAID indicators such as ‘government budget ratio invested in agriculture’, ‘GDP ratio for agriculture’ or the ‘number of people who have been provided services from the international fund for agricultural development projects (DFID)’.

SDG Target 6.1 and Target 6.2 aims to improve access to and management of drinking water and sanitation facilities. The indicator found to be similar was the ‘population ratio using safe drinking water service in urban/rural areas (SDSN)'. This indicator measures the proportion of urban/ rural population using safe drinking water services as defined by the WHO/UNICEF Joint Monitoring Program.

SDG Target 15.1 included existing indicators such as World Bank's ‘land area change rate’, ‘index of forest area’, or ‘annual change rate of forest and cultivated land (SDSN)’ while Target 15.2 included the ‘ratio of c onservation area by ecosystem in forest conservation area (SDSN)’ and ‘Sustainable Forest Management Area (SDSN)’. Moreover, Target 15.3 matched with the 'Annual Change Indicator of Degraded or Desertified Land (SDSN)', in which FAO defined desertification/degradation as the degradation of land conditions affecting ecosystem goods and services provision including salinization, corrosion, soil nutrient loss, and sand dune erosion.

2.2. Definition of outcomes

The given indicators showed that it is possible to apply existing and SDG indicators to measure the outcomes of selected ODA projects. The outcomes generated through development cooperation projects were shown in Figure 1. To compare outcomes before and after the ODA project, Region A is selected and a Region B, which is similar in various conditions such as population, income, residential environment, and income source, for the purpose of comparison and data collection using the same baseline survey method. At this point, the baseline was selected in consideration of compatibility with SDG indicators, while applying global and existing national indicators. It was assumed that periodic monitoring was implemented in (A) and (B) regions at the same time and changes caused by the ODA project were measured using the same baseline survey method. In this case, outcomes from the ODA project were represented by the gap between (A)′ and (B)′ as shown in Figure 1. Therefore, (A)′- (B)′ was defined as the performance for this study. However, it is very difficult to measure the performance of the project by selecting non-target areas (B) similar to the target area (A) and applying the same measurement method in terms of actual project performance. Therefore, it may be realistic to define the outcomes of the development cooperation project as (A)′- (A) and measure the performance in the form of simplified numbers or percentages <Figure 1>.

The measurement of project outcomes must be simple in terms of metrics and methods and also the way of collecting data only applicable to certain implementing agencies should be avoided. When applying indicators other than SDGs, it requires that: 1) the definition of the indicator should be clear and compatible with SDG indicators; 2) reliable data should be collected at reasonable cost on annual basis; and 3) in case of indicators which are not possible to collect on yearly basis, the estimated rate of change or gap through data collection over a period of two to three years should be considered. In addition, administrative data from the partner country, household surveys, farm surveys and direct monitoring could be utilized.

The data to populate result-based framework may be not necessarily available or reliable. As a consequence, resultbased framework may lack baselines, clear targets and a clear presentation of how the result chain is inter-linked. Low quality data at the initial stage of the program can also weaken performance measurement since the level of performance at the first place of the project intervention or period is not known. The lack of data leads to selecting indicators that do not measure the appropriate link in the chain of results. Therefore, it is necessary to establish reasonable assumptions to measure the result of the ODA project.

3. Utilizing SDG indicators for a rural development project

In general, the ultimate goals of international development projects are to reduce poverty and improve economic and social conditions by raising incomes of family or small-scale farms through value chain creation, better market access, production of alternative crops, improve living environment, infrastructure and training, etc.

It can be said that most details of rural development projects can be managed through SDG indicators. Thus, it is necessary to manage project performance by applying relevant indicators as suggested, mostly for the contents corresponding to SDGs 1 and 2. As initial step, result measurement of the project can be implemented by selecting indicators compatible with SDG indicators and project type characteristics. The main project components can be divided into increase in income, improvement of living condition, and rural infrastructure. In general, the project intended to increase income and creates new income sources through new technologies, installation of a stock farm or greenhouse, introduction of alternative crops and seed improvement.

The SDG indicators, which should be considered in the baseline survey to determine changes in the project content or performance, should include factors which are directly relevant to changes in income, education levels and land ownership while indicators for measuring general status can be selected in a variety of ways. For example, if the rural development project includes detailed areas such as health, education, and infrastructure, it is important to select relevant indicators which can measure the degree of social change in the target area. In addition, indicators other than the SDGs but related to the general situation can be used. However, the more indicators to be measured, the more complicated it is to measure the degree of project contribution in achieving the SDGs. Therefore, indicators should be as simple as possible and compatible with the SDGs <Table 2>.

As seen in Table 2, the activities of income-increase projects generally focus on increasing the overall income of the target area by improving agricultural productivity. In particular, it is a common practice to promote sustainable increase in income through technological cooperation such as construction of demonstration areas and pilot farms and production improvement through facility support. Therefore, indicators for measuring the performance of technical cooperation and facility support should include changes in productivity per unit of labor and changes in the income of small farmers.

In addition, improvement of living environment focuses on supporting drinking water, health, and the environment, which are directly related to the daily lives of the beneficiaries in the target area. Thus, the results of the project can be measured by using SDG indicators such as the number of residents using safe drinking water and who can use basic sanitation facilities. Other outcomes that can be derived from the project could be measured using indicators such as number of sanitation facilities at home, number of basic drinking water, water and sewage facilities, percentage of beneficiaries using hospitals, number of health centers provided with sanitation, and drinking water managed in the areas.

Rural infrastructure development projects generally support access roads to villages, community centers, water irrigation, post-harvest management facilities, agricultural waste and recycling facilities. In particular, post-harvest management facilities minimize post-harvest losses of agricultural products and contribute to an increase in income of the target area through proper storage and processing facilities. However, there are no measurable indicators related to post-harvest management in the SDG indicators, and there are only indicators related to accessibility with other areas including villages and markets. Therefore, projects that have the same basic infrastructure support could be considered as indirect offer to achieve higher goals.

The monitoring of selected indicators to measure project results is based on: 1) utilizing SDG global indicators in the baseline survey; and 2) monitoring conducted periodically. Indicators related to agricultural production should also include sowing and harvesting season of each crop.

4. Empirical approach: The case of an integrated rural development project in Bolivia

4.1. Outline & Assumption

This study looked into an integrated rural development project in Bolivia. This project intended to increase productivity through agricultural mechanization and enhance farmers’ capacity in Bolivia. The project supported sowing, harvesting and post-harvest processing of quinoa in the highlands where many poor people live. This was closely related to Bolivian government's policy development plan, which is one of the four major goals of enhancing productivity in the agricultural industry and attracting investments in infrastructure.

The details of the project included construction of a technology innovation center, support for agricultural equipment, dispatch of experts, and training program. The inputs included Bolivian government's efforts to secure land with a budget of about $2.6 million and conduct of baseline survey by international organizations. The output was building and operating a technology innovation center and processing plant to increase the production of quinoa and income of small-scale producers in Bolivia. Based on the assumptions in Table 3, SDG Indicators 2.3.1 (Agricultural Productivity and Indicators per Unit of Labor) and 2.3.2 (Small-scale producers’ Average income) was used to measure performance to show the results and contribution to SDGs.

This study analyzed the extent to which the achievements of the Bolivian project have contributed to the detailed objectives of the SDGs based on each relevant individual measurement of the indicators. In addition, the degree of contribution was based on the purpose and contents of the project and it assumed that it is reasonable to present quantitative results if it can be objectively understood.

In rural development projects such as the Saemaul Undong, the indicators for the evaluation of the degree of contribution to SDGs can be classified into gender, age, income level, etc., and the contribution should be evaluated accordingly. At this point, measurement of SDG indicators should be based on general assumptions and the statistics needed to quantify the performance of the rural development project should be established at the baseline survey. For example, the basic data on the assumption is applicable to indicators which measures general situation, income growth, improvement of living environment and rural infrastructure, of which most of rural development project consist.

The necessary data for the establishment of basic assumptions should be collected by using statistics of the central, local government and international organizations of the target country. In some cases, the data necessary for the improvement of living environment and the indicators related to infrastructure construction can be collected from the field through a survey.

4.2. Measurement of project outcomes

In the case of the rural development in Bolivia, the purpose was to increase farm income and reduce poverty. The outcomes of the project were estimated as the increase in cultivated area, total production and mechanization, and improvement of unit productivity. Therefore, it is necessary to find the achievements through the SDG indicators. According to Table 3, most of the activities and outcomes of rural development projects were categorized as ‘General situation’, ‘Income growth’, and ‘Improvement of living condition’. In this case, infrastructure and equipment support were categorized under ‘Income growth’ category, with SDG indicators 2.3.1, 2.3.2, 2.a.1, and 2.a.2 to measure the results and quantitative contribution to SDGs. Table 4 shows the outcomes and contributions to SDG of the Bolivian project. The percentage of increase/decrease were measured based on SDG indicators such as volume of production per labor unit, increase of average income or total budget. The contribution rate was calculated based on measurement results. For example, for SDG indicator 2.3.1, the 30% increase in volume of production per labor unit was multiplied by ‘input of labor on the project/total labor input of target country’. As for the ‘general situation’, SDG 2.a.2 was computed as 3.7%, that is the total official flows (official development assistance plus other official flows) to the agriculture sector (i.e. ($2.6M/$70M)× 100).

To be more specific, if the annual output per person increases from 100kg (baseline data) to 130kg after the project, the contribution to SDG was calculated as ‘the rate of increase (30%)’ × ‘input of agricultural labor in target area (32K) / total agriculture labor in target country (2.74M) × 100’, which was 0.35%. In addition, the change in income for small-scale food producers was measured as the rate of increase × (income of total small-scale producer in target are/ total income of small-scale producer in target country) × 100, i.e. 0.18 × ($1.74M/$42.02M) × 100 = 0.75%. Hence, the contribution of the project to SDGs was 0.75%. Table 5

Also, the degree of increase in income of small producers was measured by ‘the rate of income growth’ × ‘target small producer income / total small producer income’ by using SDG indicator 2.3.2. Indicators related to income was measured by the change in the population below the poverty line; i.e. ‘reduction ratio of number of people below the poverty line’ × ‘number of beneficiaries / population below the poverty line in rural areas’. Other relevant indicators included 2.a.1 ‘total public fund flows to agricultural sector’ and 2.a.2 ‘total public fund flow to agricultural sector’. This was to determine the contribution to SDGs by measuring the public sector's share of the total public sector's funds. The specific measurement method is shown in Table 4.

In addition to the indicators presented in Table 4, existing indicators reviewed in Table 1 may be also used. In other words, it is appropriate to apply indicators of productivity and income increase, which are the ultimate goals of the project. In particular, various details of the project such as increase in cultivated area, mechanization, technical education, provision of farm materials, technology innovation center, and establishment of storage and processing plant can be also used as a result of productivity improvement.

5. Conclusion

Among the 17 goals of the SDGs, the main objectives related to agriculture and rural areas include Goal 1 (poverty eradication), Goal 2 (hunger relief and food security, sustainable agriculture development), Goal 4 (education), Goal 6 (Water management) and Goal 15 (sustainable ecosystem). Since not all 17 SDGs and 169 SDG targets are about outcomes and changes on the ground, only the applicable indicators were validated in relation to achieving the SDGs. Based on the identification of existing indicators surveyed from 25 domestic experts, the contribution of the rural development project in Bolivia to SDGs was measured using the data based on the assumptions.

Even though each partner country often sets priorities among goals and targets to reflect its own goals, there is a need to use a consistent result-based framework as a whole by using the SDG indicators with high importance. However, up to certain level of outcomes, impact and change for people, the results chain is extensive to determine whether the input and activity bring benefits to the target group. Therefore, it is important to understand how an ODA project fits into the result chain with the SDG indicators to achieve the ultimate goals for sustainable development.

It also might be crucial to measure the degree of contribution to SDGs by using consistent indicators to enhance aid effectiveness of agricultural and rural development cooperation projects. For example, if the purpose of the project is to increase the income of small producers, the performance should be measured by focusing on SDG indicators related to small producers’ income growth rather than other general status indicators. It should also be noted that the definitions of the SDG performance indicators must be clear with the target country to determine appropriate measurement methods and to collect and accumulate quantified data during baseline survey.

The outcomes of international development cooperation projects in agriculture, rural areas, and other fields can measure the degree of contribution to SDGs. As such, in order to overcome the limited systemic links, the CRS code, which has recorded inputs and activities, can be aligned with SDGs and consistent management framework should be emphasized.

적 요