BACKGROUND
In October 2011, USAID and the Government of Ethiopia launched the project, Empowering New Generations for Improved Nutrition and Economic Opportunities (ENGINE). ENGINE was a multi-sectoral program aimed at improving the nutritional status of pregnant and lactating women and children under five years of age. A follow-up program, Feed the Future Ethiopia Growth through Nutrition Activity, was launched in September 2016. Save the Children is the primary awardee, with Tufts University, Friedman School of Nutrition Science and Policy, leading the learning and research component of the program.
As part of the research strategy, Tufts University is preparing to conduct operational research assessing multi-sectoral coordination for nutrition policy effectiveness in Amhara, Oromia, SNNP and Tigray regions. The research builds on earlier research conducted in 2013/14, under the ENGINE project, which focused on how a range of policies and programs get translated from the design phase (on paper) to the implementation phase (in practice). The aim of this initial research was to examine the alignment of policy goals with government commitment and coordination of stakeholders around complex and multidimensional policy and implementation challenges. The aim of this follow up research will focus on understanding the facilitators, barriers, and constraints involved in the conversion of policy initiatives into actions at the woreda level. A key objective of this research will be to determine the outcome of ENGINE support to multi-sectoral coordination bodies during the period 2014-mid 2017. To this end, Tufts University is seeking a qualified local consulting firm to conduct data collection for the study.
These terms of reference are developed to outline the purpose/objective, scope, and methodology for undertaking the research, and the roles and responsibility of Tufts, the consulting data collection firm, and the Feed the Future Ethiopia Growth through Nutrition Activity in conducting the study.
OBJECTIVE OF THE RESEARCH PROJECT
General Objectives
The key objective of this study is to expand the concept of process evaluation, to go beyond a singular program to understand the "process" of implementation of policy and multi sector activities simultaneously. This research will build on what was conducted under ENGINE. The process research that was conducted in 2013/14 found that (1) collaboration/coordination, (2) lack of awareness of the multi sector nutrition plan (3) communications and (4) budget were the main constraints to effective implementation at the woreda level. The 2013/14 study was conducted when the NNP was in the early stages of implementation. The newly proposed study evaluates the multi sector nutrition plan with a more mature history of implementation. This study will focus on understanding the facilitators, barriers, and constraints involved in the conversion of policy initiatives into actions at the woreda level. A key objective of this research will be to determine the outcome of ENGINE support to multi-sectoral coordination bodies during the period 2014-mid 2017.
Specific Objectives
MATERIALS AND METHODS
Study Design
The study is a qualitative study with purposive sampling. The study covers 12 woredas in four regions (Oromia, Amhara, SNNPR and Tigray). Ten model woredas in four regions of Ethiopia were part of focused multi-sector coordination technical support activities targeted by the ENGINE project. Four of the ten woredas included in ENGINE will be selected from the group of "model" woredas, which received support in setting up coordination boards as well as routine follow up. Four additional "non-model" woredas will be selected from the ENGINE woredas that received support in establishing the coordination boards but did not receive the routine follow-up. These ENGINE woredas (Model and Non –Model) will be called the intervention woredas. In addition, four non-ENGINE woredas will also be included in the study as non-intervention woredas, purposefully selected from the same region for comparison to the intervention woredas, for a total of 12 woredas. Table 1 lists the selected woredas for the study. The qualitative study will be conducted using Key Informant Interviews (KII), Focused Group Discussions (FGDs), and Desk Review.
Table 1: Woredas by Region and Study Arm
ENGINE Supported Woredas Non-ENGINE Woredas
Region Model Non-Model Non-ENGINE
SNNP Malga Endegagne Enemore
Oromia L/Bibilo Dedesa Dendi
Amhara Takusa Tamera Bera Chilga
Tigray Tsegede Tahtay Adiabo Aseged Tsimbla
Study Areas, Data Collection, and Sample Size
The study will be conducted in one woreda in each of the four regions targeted by ENGINE, for a total of eight interventions woredas. In addition, four non-interventions woredas in the same four regions will also be selected, for a total of twelve woredas.
There will be approximately 72 interviews at the woreda level and 8 at the regional level; this is a purposive sample of individuals involved in policymaking and governance from the various woreda level NNP implementing ministries. Individuals from different sectors will be interviewed. The sectors will include, but not be limited to, health, agriculture, water and energy, education, finance and economic development. Selected staff from INGOs will be interviewed as well as selected regional nutrition focal persons. There will also be one focus groups per woreda with the members of the Woreda Health Office Technical Team. members (10 members each).
Table 2 Key Informant Interviews
Woreda Level | KII per Woreda | Total | |
1 | Woreda Administrator | 1 | 12 |
2 | Woreda Health Office Head | 1 | 12 |
3 | Woreda Water and Energy Office Head | 1 | 12 |
4 | Woreda Education Office Head | 1 | 12 |
5 | Woreda Agriculture office Head | 1 | 12 |
6 | Woreda Finance Person | 1 | 12 |
7 | NGO Representatives at region level (SCI, UNICEF, etc) | 1 per region | 4 |
8 | Regional Health Bureau Nutrition focal persons | 1 per region | 4 |
Total | 80 |
Table 3 Focus Groups
Woreda Level | Number of Participants per FGD | Number of FGD per woreda | Total FGDs | |
1 | WHB – Technical team supporting multi-sectoral coordination (10 individuals) | 10 | 1 | 12 |
Quality Control
Data collection will mostly occur at the woreda level and 8 KII at regional level. A standardized questionnaire translated into Amharic, Tigrigna and Oromifa will be used for woredas in Amhara/SNNPR, Tigray and Oromia respectively each woreda. Interviews will be conducted in Amharic (Amhara & SNNPR), in Tigrigna (Tigray) and Oromifa (Oromia), tape recorded and translated into English. The local co-investigators in collaboration with trained data collectors will conduct the interviews using semi-structured questionnaires and focus groups using the focus group discussion guide. Interviews are expected to take approximately 30 minutes, while the focus groups will take approximately one hour. Supervisors will oversee the data collection team with random checks of questionnaires and focus groups administered by data collectors. Prior to data collection, a training and pretesting of the study tools will be conducted in order to refine the tools and ensure standardization among data collectors and supervisors.
Desk review of key documents including meeting minutes, reports, activity plans will also be conducted at the woreda.
Data analysis, interpretation and writing reports
The primary analysis for the study will use the Framework Analysis Approach. Once interviews have been translated into English, local co-investigators will assist in the development of the framework, coding, and indexing, as well as contribute to the mapping and interpretation, in order to benefit from the local context and experience. Findings of the analysis will be written up and disseminated widely in applicable forums including the project website. Analyses of the data will be presented through meetings at the national and sub-national level (such as the NNP review meetings), as well as through peer-reviewed publications.
SCOPE of WORK for the Data Collection Firm
Once the study protocol is approved by relevant ethic committees, the consulting firm will translate the data collection tools to the three local languages and train data collectors and supervisors before the commencement of the actual data collection. The training process will include the pretesting and recommendations for tool refinement. The data collectors should have documented experience in conducting similar qualitative surveys and should speak both English and Amharic, as well as Tigrigna for Tigray and Oromifa for Oromia region. After finalizing all the preparatory activities, the consulting firm will carry out field level data collection based on the time-frame indicated in the document. The consultancy firm should make the maximum effort to schedule and ensure the prior availability of eligible respondents in advance for the KIIs and FGDs to ensure completion within the prescribed time frame. Additional responsibilities include monitoring field level data quality, facilitating transcription and translation, performing data cleaning and data quality checks and preparing the data for data analysis (coding). The firm is also fully responsible for desk review of key documents at woreda offices using provided check lists
Key deliverable: Pretest report, fieldwork/data collection plan, report on field level data collection completion, soft copy of final survey data with data dictionary/ and other relevant information, translation and preliminary data coding.
Required expertise and experiences of the key personnel of the consulting firm for data collection
Key Personnel: CO –PI
Key Qualification and Experience
Roles and Responsibilities
Key Personnel: Data Collection Supervisor
Key Qualification and Experience
Roles and Responsibilities
Key Personnel: Data Collectors
Key Qualification and Experience
Roles and Responsibilities
Specific Roles and Responsibilities of Consulting Firm
Roles and Responsibilities of Tufts University
Time Frame and level of effort
Please refer to the table below for detail schedule of activities. Please note that the following schedule needs to be strictly followed. Tufts suggests having two data collectors per region (a team of two persons, one interviewer/facilitator and one note taker) to accomplish data collection transcritption and translation on time as per the schedule.
Steps | Activities | Output/ Deliverables | Responsible Party |
1 | Develop & Submit Study Protocol for IRB approval | Study protocol | Tufts University |
2 | Translate data collection tools from English to Amharic, Oromifa and Tigrigna | Translated Tools | Consulting firm |
3 | Recruitment and Training of Data collectors | Trained Data Collectors | Consulting Firm |
4 | Pre-test (pilot) data collection tools and tool refinement | Pretest Report Finalized data collection tools | Consulting & Tufts |
5 | Conduct data collection using KII and FGD guides | Qualitative Data collected | Consulting Firm |
6 | Desk Review of key documents and filling check list | Desk Review Checklist | Consulting firm |
7 | Transcribe audio taped data on respective language | Transcribed 23 Oromifa, 23 Tigrigna and 46 Amharic scripts | Consulting firm |
8 | Translation of transcribed data from Amharic, Oromifa and Tigrigna to English | English scripts of 80 KIIs and 12 FGDs | Consulting firm |
9 | Initial coding of qualitative data responses | Coding Report | Consulting firm |
10 | Entering desk review and check list collected data to excel | Semi-quantitative data from desk review and check list using excel sheet | Consulting firm |
11 | Analysis, interpretation and drafting report | First draft of the research report | Tufts University |
12 | Solicit feedback on report from consulting firm and other stakeholders |
| Tufts University |
13 | Finalize report | Final report | Tufts |
14 | Conduct dissemination workshop | Workshop organized | Tufts University |
IMPLEMENTATION OF THE PROJECT BY PHASE:
List of activities by phase | Duration/Responsibility | ||
Phase | Activities | Time frame | Responsibility |
Phase 1 | Secure IRB approval and support letters | Before August 25, 2017 | Tufts University |
Receive applications from competing consulting firms | August 14-28, 2017 | Tufts University | |
Select consulting firm and complete contractual agreement process | August 29-Sept 01, 2017 | Tufts University | |
Phase 2 | Translating Questionnaire from English to Amharic, Oromifa and Tigrigna | Sept 02-07, 2017 | Consulting firm |
Pre-test of Questionnaire and revising the tools | Sept 02-11, 2017 | Consulting firm | |
Reviewing and approval of the translated questionnaire by Tufts | Sept 12-13, 2017 | Tufts University | |
Phase 3 | Data collectors training | Sept 15-17, 2017 | Consulting firm |
Data collection | Sept 18-29, 2017 | Consulting firm | |
Transcription and translation from Amharic, Oromifa and Tigrigna to English. Entering check list collected data in to excel | Sept 18- October 05, 2017 | Consulting firm | |
Phase 4 | Submission of audio taped, transcribed and translated data to Tufts for review | October 06, 2017 | Consulting firm |
Expert review of audio taped, transcribed and translated data | 2nd & 3rd week of October 2017 | Tufts University | |
Incorporate comments/suggestions from expert review | 4th week of October 2017 | Consulting firm | |
Initial coding of responses | 1st & 2nd week week of November 2017 | Consulting firm | |
Review of Initial coding by Tufts | 3rd week of November 2017 | Tufts University | |
Incorporating comments/Suggestions | 4th week of November 2017 | Consulting firm | |
Submission of revised codded data | 1st week of December | Consulting firm | |
Phase 5 | Analysis and Paper Writing
| December 2017-January 2018 | Tufts University |
Phase 6 | Presentation of preliminary findings
| February 2018 | Tufts University |
Phase 7 | Publication finalization
| March 2018 and onwards | Tufts University |
Dissemination workshop | March 2018 | Tufts University |