Terms of Reference: Evaluation of the Honey Value Chain Project (HVCP)

I.Background

The design of Honey value Chain Project (HVCP) was strategically linked to the Phase II REDD+ Participatory Forest Management (REPAFMA) in Southwestern Ethiopia Project. Both projects had an overlapping timeline from January 2018 to December 2021. By improving the livelihoods of target communities, the HVC project seeks to contribute to the long-term sustainability of forest resources in the intervention areas which are under the Participatory Forest Management (PFM) scheme since 2003. Both projects were implemented by The Development Fund’s (DF’s) long time Implementing Partner (IP), Ethio-Wetlands and Natural Resources Association (EWNRA).

Geographically, the honey value chain (HVC) project was implemented in the Masha and Anderach woredas, Sheka Zone, which used to be under the Southern Nations and Nationalities Peoples Region of Ethiopia until recently and became under the newly established Southwestern Ethiopia Region since September 2021. The Sheka forest in the project intervention area is characterized by diverse fauna and flora and is home for enormous Non-Timber Forest Products (NTFP). Honey is one of the NTFPs having significant potential as forest dependent communities’ livelihoods.

The project seeks to address challenges facing the traditional beekeeping management system, which is characterized by poor quality, limited honey production volume, poor infrastructure, and limited access to market for fully utilizing the benefits from the resource. Through introduction of improved technologies, building the capacity of the beekeepers, cooperatives, and union; support communities and creating market link with an international buyer, the project aimed to increase 200 target beekeepers’ production by 150 tons and export 100 tons of quality honey that can fulfill international buyer’s requirements.

EWNRA has prior experience in the sector and had been implementing similar activities through the earlier NTFP project and the Phase I REPAFMA in SW Ethiopia Project. The HVC project required the IP to work in collaboration with stakeholders at various levels to increase the number of beehives per beekeeper, improve the production and quality of the product and ultimately increase the share of livelihoods income from beekeeping. This would shift the sector from just what has been livelihoods complementary to the mainstay of the forest dependent communities in the project intervention area.

This document spells out the terms of reference for conducting final evaluation of the HVC project for the period January 2018 to end of December 2022.

2.Project Objectives/Outcomes

The HVC was designed with the objective to contribute to the HVC improvement in Ethiopia by increasing income and job opportunities for beekeepers while protecting the ecosystem.

Overall goal: Improved livelihoods of target communities.

Main results:

Increased production and quality of honey **

· Beekeepers’ knowledge on improved beekeeping raised

· Increased access to necessary equipment for improved honey production

· Increased access to processing and packaging materials

Increased export of honey

· Honey union and cooperatives capacity strengthened

· Honey Union and cooperatives linked to international market

· Honey sector forum established and strengthened

3.Purpose and Scope

The purpose of this evaluation is to draw lessons and obtain recommendations from an external review that can offer inputs to improve quality in designing and implementation of upcoming project and/or similar projects. The evaluation assesses to what extent the HVCP implementing partner (EWNRA) has addressed accountability procedures in terms of resource utilization and timely delivery of project’s outputs. Moreover, the evaluation will review, to what extent the HVCP was relevant, effective, efficient, coherent, and assess the sustainability and impact of the project.

The evaluation will be conducted in the selected targeted districts ensuring the representation of all target groups, partners, and stakeholders at all levels. As such, the consultant is required to deliver concrete lessons and actionable recommendations that will help to identify areas that need improvement and areas to capitalize on which ultimately be used by DF for constructive decision-making.

Summary of Evaluation objectives:

Objective 1: Evaluate relevance,coherence,effectiveness, and efficiency of the HVC project.

This evaluation will primarily assess the relevance, effectiveness, and efficiency of the project’s design in terms of addressing the needs and priorities of the target beneficiaries, attaining its intended objectives, and appropriate usage or the cost effectiveness of the project resources.

Objective 2: Evaluate impact and sustainability of prioritized areas

This evaluation will review impact and sustainability of the HVC project implementation in terms of its contribution in changing or improving the livelihoods of the target communities and the extent to which the achieved impacts and outcomes are sustainable.

In achieving both objectives above, the evaluation will serve for both learning and accountability purposes and will seek to:

· Identify key achievements during HVC project cycle management that are the strengths so far and,

· Identify any challenges faced by the project and formulate appropriate recommendations for future actions.

a.Key Evaluation Questions for Objective 1

i.Relevance

· To what extent and how have the Project operators (DF and the implementing partner) identified and addressed the livelihood security needs and interests of forest dependent communities?

· To what extent was the project strategy built on an adequate understanding of the honey sector including broader market analysis?

· Are the activities and outputs of the project consistent with the overall goal and the attainment of its objectives?

· To what extent and how does the project design respond to the development priorities of the country (including the Growth and Transformation Plan (GTP), the Climate Resilient Green Economy Strategy (CRGE) and sector strategies for forest, i.e, National Forest Sector Development and REDD+ Strategies, etc.)

· To what extent and how does the project design respond to the development priorities of Norway?

ii.Coherence

· To what extent has the Project created synergies and collaboration between partners within the Project (internal coherence)?

· To what extent has the Project created synergies and collaboration with similar interventions (value chain) and actions outside the project (external coherence)?

· Has the communication/collaboration between the partner and DF been adequate?

iii.Effectiveness

· To what extent have the results (outcomes and outputs) of the program been achieved compared to the planned targets?

· Are activities implemented effectively? What is the record of time overruns in the project (i.e. where activities implemented and outputs delivered according to schedule)?

· Have adequate M&E system arrangements been in place for effective M&E (e.g. a M&E/MEAL plan), including results-based reporting and a routine for measuring performance against the Results Framework? Have necessary adjustments been made based on reporting and achievement or non- achievement of results?

· What is the level of satisfaction among direct beneficiaries on the project interventions?

· Has the risk management strategy been adequate to cope with identified risks and/or cope with unexpected impacts or opportunities?

· How can future similar project improve their planning process to address internal and external challenges and constraints?

iv.Efficiency

· To what extent was the project implemented in the most efficient[1] way to deliver expected results in an economic and timely manner?

· Since the project aims at supporting the development of profitable businesses – has it used resources efficiently in a way to support motivation for business and profit?

· To what extent are the observed results commensurate with the planned inputs and budget allocations?

· What measures have been made to prevent delays? Strengths and weaknesses.

b.Key Evaluation Questions for Objective 2

i.Impact

· To what extent and how has the Project contributed to strengthened livelihoods for the target group? Does the target group report any changes in livelihood or increased income over time due to project interventions?

· To what extent has the project contributed towards the intended outcome(s) and impact?

· Are the HVC related activities demonstrating meaningful contribution to sustainable forest management? If yes How? If No, what needs to be improved?

· To what extent has the project contributed to any unintended impacts or effects, positive or negative (e.g. social, economic, environmental, or other)?

· What were the main challenges towards achieving the outcomes and how might these be addressed in future similar interventions?

· Did impact vary for different targeted areas, households, or individuals? If so, how and why?

ii.Sustainability

· To what extent are the Project’s benefits (positive actions) likely to continue after completion?

· Does a sensible exit strategy exist for transfer of responsibility and activities to other stakeholders (local government, and communities)?

· What are the major factors which influence the achievement or non-achievement of sustainability of the project?

· How have the methodology applied contributed to the sustainability of the project achievements?

c.Cross-cutting issues

· To what extent the gender sensitive approach impacted the differing needs and priorities of gender groups?

· To what extent and how has project design managed to take into account particular challenges or adjustments needed in order to include women on an equal level to men? Have particular attention been made on the needs of minorities?

· Have environmental factors been considered adequately in the project design? Were mitigation measures put in place?

· To what extent have good environmental practices been followed in implementing the project?

· To what extent and how are we delivering appropriate and effective programming for persons with disability?

· To what extent is the project implemented in an honest and ethical manner, complying with donor policies related to anti-corruption?

4.Methodology

The consultant will develop appropriate methodologies and data collection tools that will provide sufficient basis for cost effectiveness analysis and should exhaustibly address the above questions. The evaluation should include participatory methods that give voice to the target group, including direct quotes and testimonials or case stories. The evaluation should also consult direct beneficiaries, partners, relevant government institutions and other stakeholders. Relevant document review, focus group discussion (FGD) and Key Informant Interview (KII) will also be part of the methodology. Therefore, the consultant is expected to craft appropriate methodologies and include in the inception report for verification by DF.

Partners and Geographical locationand sampling

The geographic areas covered in HVC project are in the South-Western Ethiopia Region, Sheka zone, 2 districts/woredas, 20 Kebeles. The evaluation will sample representative communities or HHs from selected kebeles/villages of the two woredas. Therefore, to make it more representative, the consultant shall take enough sample villages for each of the cooperatives. The respondents from each kebele/village will be randomly selected to ensure that the evaluation objective will be attained effectively.

5.Ethical Considerations

The whole evaluation study process should follow standard good practice for engagement and enquiry with all participants. To this effect, the assessment will be carried out in line with the following research ethics:

• Enough information/ explanation will be given to all participants of the evaluation assessment about the purpose, objective and privacy criteria and other related issues.
• Participants will be asked for their verbal consent before participation.
• All participants will be informed that they have the right not to partake in or withdraw from the assessment at any stage.
• The privacy of the informants and the confidentiality of the information provided by respondents will be respected.
• The assessment will be carried out with respect to cultures, traditions, knowledge, and customs.
• Caution will be taken to avoid raising any unrealistic expectation of participants.

6.Expected Deliverables

Inception report: the inception report should provide details of the evaluation methodologies; data collection process including what will be assessed and why, showing how each assessment question will be answered by way of: proposed methods; proposed sources of data; and data collection procedures. The inception report should also include a proposed schedule of tasks, activities and deliverables, composition of team members in the field with the lead responsibility for each task.

Preliminary assessment report: After data collection and analysis is completed the consultant will submit preliminary assessment report and a validation workshop will be organized in which the consultant is required to present the main findings of the assessment to DF, consortium members, and implementing partners. Feedback on the assessment report will be obtained during the validation workshop. Relevant program staff both from partner organizations and DF will review the draft assessment report and give feedback in writing to the consultant to finalize the assessment report.

Final Evaluation Report: This will be prepared after receiving comments from DF, consortium members, and partners. The content and structure of the final analytical report with findings, recommendations and lessons learnt covering the scope of the assessment should meet the requirements as stated in the ToR, should not exceed 35pages plus annexes and should include the following:

• Cover page
• Table of contents
• Acronyms/Glossary
• List of Tables
• List of figures
• Executive summary
• Introduction/Background
• Description of the assessment methodology
• Main report of the assessment divided by major areas of assessment (this needs to be discussed and agreed with DF when starting to draft the report or during validation workshop)
• Summary of findings and lessons learnt.
• Annexes (filled data in SPSS or other app., qualitative raw data)

Additional Deliverables That should be submitted with Final Report:

• Cleaned complete dataset(s) of all quantitative data.
• Output of statistical analysis of quantitative results in MS Word/SPSS outputs
• Complete transcripts of all qualitative data collected in English.
• Content analysis or other analytical output of the qualitative data, including key themes identified and frequencies of those themes.
• Documentation of the qualitative topics analyzed.

All written documentations are to be submitted in English using Microsoft word in both soft and hard copy.

7.Timeline

The duration of the study is estimated to a maximum of Four weeks. The study should commence, no later than February 14, 2022. Activities, dates (to be agreed) and deadlines (to be proposed by consultant and agreed) for the consultancy work are:

Table 2: Timeline for evaluation

S/N / Tasks/ Time Required/ Responsibility

1: Contextual Analysis and Literature Review; Initial meeting with DF and implementation partners etc. / / Consultant

2: Prepare inception report, including initial preparations of data collection methodologies and tools, as well as detailed work plan. / / Consultant

3: Submit draft inception report / / Consultant

4: Meeting between DF and consultant regarding inception report. / / DF

5: Inception phase; Finalize inception report, completion of data collection methodologies and tools, training of survey team or similar, logistics of fieldwork including appointments. / / Consultant

6: Field work / / Consultant

7: Data entry, data analysis, development of draft, report and preparation for validation workshop; / / Consultant

8: Validation of draft report in a workshop to receive preliminary comment from DF & IPS / / Consultant with DF support to organize the workshop

9: Submission of draft report / / Consultant

10: Receive further comments from DF & IP / / DF

11: Submission of final report / 14 Marc 2022/ Consultant

8.Roles and Responsibilities

The consultant will prepare a comprehensive participatory methodology for undertaking the evaluation. The consultant will collaborate with partners to appraise existing data collection tools and design as needed and collect data. The consultant is expected to travel to project implementation areas, to adequately familiarize him/her with the project logical framework and any relevant information relating to policy and legal frameworks in the agriculture, environment and related sectors. The consultant will be responsible for data analysis and report writing which will be accompanied by presenting study results to DF, and partner organization for validation. Whereas field visits and contact sessions with communities will be facilitated by partners.

The main responsibility for ensuring that all responsibilities are carried out lies with the consultant.

The consultant will undertake the following activities:

· Conduct review of data sources, reports, and relevant literature.

· Develop appropriate study methodology, design, and tools.

· Select most appropriate sample project areas for the visit.

· Submit an inception report and finalize evaluation plan.

· Conduct field work.

· Synthesize findings.

· Organize validation workshops in which local partners, DF and other relevant stakeholders participate.

· Produce draft report.

· Produce final report, including incorporating comments from DF and its implementing partner.

· Present findings to DF staff and to partner staff as required by DF.

Other responsibilities of the consultant:

· Adhere to all terms/conditions stipulated in the contract.

· Pay income tax or other taxes as required.

· Obtain relevant insurance as needed.

· Adhere to DF’s code of conduct while undertaking the assignment.

· Adhere to the agreed timeframes with regard to all activities outlined in the timeline

· Consultant’s own laptops and phones to be used during the assignment.

· Pay for own accommodation and per dime etc.

DF will be responsible for the following:

· Provide consultant with all required documents.

· Link consultants to partner organization and other relevant stakeholders and set up meetings as required.

· Support in the process and provide technical input on methodology both from Oslo and Ethiopia office

· Plan (in collaboration with partner and the consultant/s) the field visits.

· Follow up the progress of the evaluation as per plan ensuring quality.

Implementing partner will be responsible for the following:

· Avail for the consultant during the field visits.

· Arrange interviews, focus groups discussions, meetings, project site visits as requested.

· Logistic arrangement in the field in discussion with DF.

9.Desired competencies and skills of the consultant

The evaluation team shall preferably be international consultant. The term consultant in this ToR refers to the evaluation team. The Consultant will liaise closely with DF, implementing partner staff, staff of relevant government and non-government institutions in the areas where the evaluation will take place.

The members of the consulting team should hold the following competencies and skills:

· Advanced university degree in the field of agriculture, development studies, economics, or related fields; working particularly on climate change, food security, and rural development.

· Extensive experience in designing and conducting independent evaluations on rural development, value chain development of agricultural products more specifically in honey value chain, and climate change programs/projects.

· Good understanding of gender, inclusion of minorities and community-based organizations.

· Good communication, analytical, drafting and document presentation skills.

· Demonstrated ability to meet deadlines.

· Practical experience on gender issues and gender integration analysis.

· Excellent writing and presentation skills.

[1] Note:“Economic efficiency” is the conversion of inputs (funds, expertise, natural resources, time, etc.) into outputs, outcomes, and impacts, in the most cost-effective way possible, as compared to feasible alternatives in the context. “Timely Efficiency” delivery is within the intended timeframe, or a timeframe reasonably adjusted to the demands of the evolving context. This may include assessing operational efficiency (how well the intervention was managed).

How to apply

Interested company/organization shall submit both technical and financial proposal latest by 4:00 PM on 31 January2022.

The technical proposal shall demonstrate at least the following:

o The eligibility of a company/organization,

o Proposed approach, methodology, and deliverables,

o Proposed team including CVs

The financial proposal shall disclose all pricing information related to the consultancy service as described in this Terms of Reference. Fee (nonrecurring and recurring costs), travel cost, and other out of pocket expenses should be given separately as a lump sum. Conditional cost is not acceptable. Total costs may not exceed 250,000 NOK.

The proposals shall be submitted separately, one containing technical proposal and the other financial proposal, and must be marked “Consultancy Service for HVCP Review” and delivered to the following address: post@utviklingsfondet.no

For any questions regarding the ToR, please contact Sisay Kassahun, Country Program Coordinator, sisay@utviklingsfondet.no.