Terms of Reference (ToR) for the Evaluation of NCD and MH Components in Health Projects in NW Syria Using the RE-AIM Framework

Terms of Reference (ToR) for the Evaluation of NCD and MH Components in Health Projects in NW Syria Using the RE-AIM Framework

1. Background and Context

• 1.1 Project Overview:
  The health projects in NW Syria under the Earthquake Response of Stichting Vluchteling (SV) aim to provide Primary Health Care (PHC) services to affected populations, with a significant focus on Non-Communicable Diseases (NCD) and Mental Health (MH). These projects began after the earthquake in March 2023 in Turkey and NW Syria as part of SV’s emergency response. The response included major health projects in NW Syria through local partners already present in the area.

  The primary focus was on delivering essential and life-saving health services through PHCs operated by five different partners. Each partner supported 2 to 3 PHCs during two phases of funding. Each phase lasts 6 months. All partners have finished the first phase projects and are in their second phase, which are expected to be finalized by November or December 2024. These projects ran from September and October 2023, with different starting dates for each partner project.

  All these health projects included an integrated NCD/MH component with various levels of implementation and different modalities of services provided. The NCD/MH programming is a strategic focus of SV’s health programme, aimed at addressing critical health needs in a humanitarian context.

  The partners, codes and budgets are the following:

  1- HiHFAD :
  Phase 1: SYR-23-00213 ( 251.744 EUR)
  Phase 2: SYR-24-00315 ( 523.889 EUR)

  2- SRD:
  Phase 1: SYR-23-00207 ( 256.192 EUR)
  Phase 2: SYR-24-00317 ( 343.030 EUR)

  3- SEMA:
  Phase 1: SYR-23-00146 (494.925 EUR)
  Phase 2: SYR-24-00274 (366.146 EUR)

  4- Shafak:
  Phase 1: SYR-23-00214 (284.346 EUR)
  Phase 2: SYR-24-00327 (358.881 EUR)

  5- AHF:
  Phase 1: SYR-23-00227 (250.000 EUR)
  Phase 2: SYR-24-00316 (349.974 EUR)

• 1.2 Purpose of the Evaluation:

  The primary purpose of this evaluation is to assess the reach, effectiveness, adoption, implementation, and maintenance of the NCD and MH components within these health projects using the RE-AIM framework. The evaluation will provide a comparative analysis to identify best practices and areas for improvement, ultimately enhancing the impact of these interventions. Additionally, the evaluation aims to formulate recommendations, document lessons learned, and draw conclusions that will inform the design and implementation of future programming.

• 2. Objectives of the Evaluation
  • To assess the reach of the NCD and MH services in terms of the number and demographics of individuals served.
  • To evaluate the effectiveness of the NCD and MH services in achieving intended health outcomes and patient satisfaction.
  • To examine the different approaches used by organizations in adopting the NCD and MH components and the factors influencing their programming.
  • To analyze the fidelity of implementation, including adaptations made and barriers encountered in NCD and MH services.
  • To assess the sustainability and long-term maintenance of the NCD and MH interventions.
  • To compare the performance of the NCD and MH components across the five projects and provide recommendations for enhancing their impact.
  • To formulate recommendations, document lessons learned, and draw conclusions that will inform the design and implementation of future programming.

• 3. Scope of Work

  3.1 Evaluation Framework:
  The evaluation will be guided by the RE-AIM framework, focusing on the five dimensions: Reach, Effectiveness, Adoption, Implementation, and Maintenance.

  3.2 Key Evaluation Questions:

  Reach:

  1. How many individuals were served by the NCD and MH services in each project?
  2. What are the demographic characteristics (age, gender, socioeconomic status, etc.) of the participants receiving NCD and MH services?
  3. What proportion of the target population was reached by the NCD and MH services?
  4. Were there any subgroups within the target population that were underrepresented or not reached effectively?
  5. What methods were used to engage and reach the target population, and how effective were these methods?

  Effectiveness:

  1. What specific health outcomes were achieved for NCD and MH conditions (e.g., reduction in symptoms, improved management of conditions)?
  2. How satisfied were the patients with the NCD and MH services provided?
  3. What improvements in quality of life were observed for individuals with NCD and MH conditions?
  4. Were there any unintended or negative outcomes associated with the NCD and MH services?
  5. How do the health outcomes and patient satisfaction levels compare across the different projects and partners?

  Adoption:

  1. What different approaches were used by each organization to integrate the NCD and MH components into their services?
  2. What were the key factors that influenced the programming of NCD and MH services in each organization?
  3. How did the organizational structure and resources affect the adoption of NCD and MH components?
  4. What were the perceptions and attitudes of the staff towards the NCD and MH components, and how did these influence their implementation?
  5. How did external factors (e.g., community needs, local policies, funding availability) impact the adoption strategies for NCD and MH services?

  Implementation:

  1. How well were the NCD and MH interventions implemented according to the original protocol?
  2. What adaptations were made to the NCD and MH services during implementation, and why?
  3. What were the main barriers and facilitators to the implementation of NCD and MH services?
  4. How consistent was the implementation of the NCD and MH services across different PHCs and partners?
  5. What resources (e.g., training, funding, infrastructure) were required for the effective implementation of the NCD and MH interventions?

  Maintenance:

  1. What long-term outcomes were sustained for NCD and MH conditions beyond the initial project period?
  2. How well were the NCD and MH interventions integrated into routine practice within the PHCs?
  3. What strategies were employed to ensure the sustainability of the NCD and MH services?
  4. Were there any challenges in maintaining the NCD and MH interventions after the end of the initial project support?
  5. How did the continuation of support in the second phase impact the maintenance and sustainability of the NCD and MH services?

• 4. Methodology

  4.1 Data Collection:
  The consultant(s) is expected to propose an evaluation methodology employing a mixed-method data collection approach consisting of both qualitative and quantitative primary and secondary data collections. This approach should include, but is not limited to, a desk review, Key Informant Interviews (KIIs), Focus Group Discussions (FGDs), and a review of existing health records and routine data. The Final Evaluation should triangulate data from multiple sources and stakeholders to ensure reliable, accurate, and representative findings.

  4.1.1 Desk Review:
  • Conduct a thorough desk review synthesizing existing literature on the project and/or related or similar projects, including the project proposal, logframe, and other relevant documents.
  • Set meetings with key personnel to better understand the project, its goals, and activities.
  • Review documents as part of the desk review phase, including any available literature to draw on existing empirical evidence in the field of NCD and MH interventions in humanitarian settings.

  4.1.2 Primary Available Monitoring and Evaluation Data:
  • Utilize primary monitoring and evaluation data previously collected by project staff, including activity surveys, project quarterly reports, and other relevant documentation.

  4.1.3 Quantitative Data Collection:
  • Health Outcome Surveys:
  o Administer surveys to patients to measure changes in health status, symptom severity, and quality of life related to NCD and MH conditions.
  o PHC Registry:Review routine data collected by PHCs, including patient demographics, visit frequencies, and service types utilized.
  o Routine Data Collected: Assess regularly collected data on key health indicators, such as blood pressure readings, blood glucose levels, mental health assessments, and other relevant metrics for NCD and MH conditions.

  4.1.4 Qualitative Data Collection:
  • Interviews:
  o Conduct in-depth interviews with healthcare providers, administrators, and support staff.
  o Interview patients receiving NCD and MH services.
  o Engage key stakeholders, including local health authorities, partner organizations, and community leaders.
  • Focus Group Discussions (FGDs):
  o Facilitate group discussions with various stakeholders (e.g., patients, community members, healthcare providers).
  • Health Records and Case Studies:
  o Review detailed documentation of individual patient histories, treatment plans, and follow-up actions.

  4.2 Sampling Framework:
  • Provide a detailed sampling framework in the inception report with clear criteria for respondent inclusion.
  • SV technical health and MEAL teams will work with the consultant to ensure the adequate selection of Key Informants and FGD participants.

  4.3 Validation and Inclusivity:
  • The methodology and data collection tools need to be validated by SV’s technical health and MEAL teams.
  • Develop a detailed evaluation matrix with criteria that will capture the study’s objectives and evaluation questions to guide the evaluation process towards reliable conclusions and outcomes.
  • Ensure data collection, analysis, and results are inclusive, showcasing a wide range of viewpoints

• 5. Deliverables
  • Inception Report: Detailed evaluation plan, including methodology and data collection tools tailored for NCD and MH components.
  • Draft report: Preliminary findings and initial analysis of the NCD and MH services.
  • Final Report: Comprehensive report with detailed findings, comparative analysis, and recommendations for NCD and MH services.
  • Presentation: Summary of findings and recommendations for NCD and MH components to be presented to stakeholders.

• 6. Timeline
  The period available to submit the final report for this evaluation is 3 months from the starting date.

• 7. Qualifications and Experience
  The evaluator should have:
  • Proven experience in public health evaluation, particularly in humanitarian settings.
  • Expertise in using the RE-AIM framework or similar evaluation frameworks.
  • Strong analytical skills and experience with both quantitative and qualitative data analysis.
  • Evaluation methods and data collection skills
  • Familiarity with NCD and MH conditions and interventions.
  • Knowledge of the context and challenges of working in NW Syria.

• 8. Management and Coordination

  8.1 Roles and Responsibilities:
  • Evaluator: Conduct the evaluation, collect and analyze data, prepare reports.
  • Project Partners: Provide access to data, facilitate interviews and focus groups.
  • SV: Introduce and coordinate communication between evaluator and partners, Review findings, provide feedback on interim and final reports.

  8.2 Logistical Support
  • Consultant(s) will be responsible for organizing their own logistics for data collection (vehicles, fuel, and drivers), and this must be budgeted into the study. SV cannot provide support in arranging logistics.
  • In addition, SV and partners will share project materials with the external consultant(s), such as the project proposal, logframe, M&E plan, etc.