Terms of Reference (ToR) EVALUATION LOCAL MULTI-SECTORAL RESPONSES TURKIYE

CONTEXT

The Netherlands Refugee Foundation, also known as Stichting Vluchteling (SV), is an emergency relief organization that supports partner organizations (international and local) worldwide to assist displaced populations.

The humanitarian response to the EQ in Turkiye/Syria was led from the start by mobilizing and funding its two international partners with in-country presence in Syria: Intersos and People in Need, with the latter also delivering SV funded assistance in Türkiye.

From May 2023, SV provided funding to 8 projects which were implemented by 3 local partners in Turkiye, specifically the Gaziantep, Hatay, Adiyaman, Diyarbakır and Adana to deliver assistance, such as shelter, protection, winterization, psychosocial support, clean water and sanitation, during times of acute need. SV worked with this small group of partners (3) where we provided support with a smaller amount and shorter project duration and following a good collaboration we continued with increased funding and duration of the project while strengthening the partnership. All 3 partners received at least two rounds of funding from the first month after the earthquake till the end of 2024 with a total funding of €1.379.759,06 The first round of funding for local partners started from May 2023 till November/December, and the last funding round took place in May 2024. One of the aims of this evaluation is to assess SV’s role and support to its partners while evaluating its overall partnership approach. Ultimately providing strategic-level-based learning opportunities by combining project-based learning with an assessment of SV’s governance, processes, decision making and strategic approaches in the EQ response.

EVALUATION OBJECTIVES

The purpose of the evaluation is to:
(1) Learn from the 2023-2024 Turkiye Earthquake Response, assess the effectiveness, relevance and impact of SV’s support to local NGOs/partners in delivering multi-sectoral humanitarian assistance in Turkiye in 2023-2024. In addition to assessing the alignment of SV’s support with the priorities outlined in the Turkiye year plan (2024) and Turkiye strategy.
(2) To evaluate the effectiveness of SV’s partnership approach in supporting local partners to deliver timely, high quality, and well-coordinated assistance. It will also identify lessons learned by formulating recommendations to improve SV’s support to partners and enhance partners’ effectiveness and capacity to respond in emergencies.
(3) Finally, the evaluation will assess how effectively SV implements its localization strategy focusing on the following (see Annex II, Localization Strategy):

• Partnerships (Commitments 1, 2, 18, 19, 20, 22, 23, 24, 25)
• Funding (Commitments 8, 9, 10, 11, 12, 13, 14)
• Capacity Strengthening (Commitments 18, 19, 20, 22)
• Emergency Preparedness & Response (Commitments 23, 24, 25)
• Safety, Security, and Risk Sharing (Commitments 26, 27, 28)
• Visibility and Communication (Commitments 29, 30, 31, 32, 33)
• Internal Organization (focusing on rapid emergency response capabilities)

SV is proposing the following evaluation questions:

Effectiveness

Effectiveness, relevance to the local needs, and impact generated by the sectoral assistance on the lives of the beneficiaries and on their communities, including challenges and gaps:

1. How well did the response address the needs identified by the affected population? To what extent did the program activities effectively respond to immediate humanitarian needs in a timely manner? To what extent did the local partner support contribute to a more agile, flexible, relevant, and local response? What lessons can we gather from both internal and partner perspectives?
2. What positive/negative changes were observed in the lives of the affected population as a result of the response? What were the key factors that contributed to the success or challenges of the response in delivering assistance?
3. Did the program contribute to the overall well-being and protection of the affected population? And how did the different approaches between our partners have an impact on the affected population (e.g. in terms of appropriateness to the needs, adaptation to the context, cost-efficiency, etc. )?

Localization
Localization and the application of quality standards by local partners in the delivery of assistance, most notably parameters related to accountability (AAP), community participation, as well as promotion of beneficiaries’ dignity and safety, and do no harm

1. What role did the affected population have in shaping or commenting on the successes and challenges of the response? What lessons can be learned and how can improvements be identified for future programming?
2. How has SV’s expertise/mandate and partnerships (including SV’s presence) affected the quality of the partners projects?
3. What has been the impact of the partner’s presence and response on other local stakeholders? (e.g. on other local organizations, authorities, relationship between communities and authorities, etc.)
4. What was the local embeddedness/representativeness of local partners, and impact of power dynamics and relationships between different local stakeholders? And how has this affected the response?

Quality of partnerships and local governance
1. How did SV’s approach impact the decision-making power of its local partners concerning the design, implementation and modality of delivery? How did (local) partnerships promote the inclusion and meaningful participation in the decision-making of local actors and marginalized groups within the affected population?

EVALUATION METHODOLOGY

The evaluation team is expected to employ a methodology for unbiased results, designed to contribute to the general learning objective by semi- structured evaluation questions for all projects, in order to aggregate the findings.
The evaluation can be divided into two types and covers 3 partners (representing in total €1.379.759,06 EUR) who SV has supported for multiple grants. The first concerns a light-touch desk research for projects that have ended in 2023 and early 2024, for which only KIIs with key stakeholders can be considered:
• MaviKalem: TUR-23-00142, TUR-23-00205 (End in February), TUR-23-00245 (End in May)
• INSAN Charity: TUR-23-00234
• Support to Life: TUR-23-00240 (End in April)
The primary evaluation is a final evaluation for 2024 projects that will end in December 2024, which would also include a survey with a sample of beneficiaries.
• Support to Life: TUR-23-00119 (End in June)
• MaviKalem: TUR-24-00307 (End in October)
• INSAN Charity: TUR-24-00312 (End in October)

It is required that this evaluation approach revolves mostly around consultations and participation of beneficiaries and local stakeholders. Methods include KIIs and a survey, and if relevant, FGDs
Since there are projects with similar activities a degree of comparison should be included in the evaluation concerning the relevance, timeliness and effectiveness of the projects. During the data analysis process, it is important to pinpoint specific details that pertain to the local partner, project, and/or local partner, where possible. This ensures a clear differentiation in how each project and partner contributed to the overall outcomes. Additionally, if there are noted variations between partner and projects, it is essential to provide a thorough explanation for these particularities. Furthermore, the evaluation conclusions should focus on providing insight into the reasons ‘why’ a particular finding was observed rather than solely describing them.
For the inception report:
• Desk-based literature review: meta-analysis and summary of reports addressing humanitarian context, needs, missed opportunities, etc; analysis of project proposals and workplans, baseline and existing MEAL data, performance indicators and budgets and other broader regional and contextual background material
• Secondary data analysis: analysis of data already collected by the project staff
For the evaluation:
• Semi-structured interviews for all projects and partners: key informant interviews with staff, local stakeholders, direct beneficiary groups, and the affected population. Interviews can take place in person or remotely.
• A survey with beneficiaries from different partners for which projects will end in June-October 2024
• Direct observation of project interventions and relationships between different stakeholders

STAKEHOLDER INVOLVEMENT
The principal intended users of the evaluation are SV, the partners, local partners, and SHO as the project funding source. The partners are expected to share relevant findings with the affected population and local stakeholders.

The partners and SV staff will provide the evaluation team with all the necessary background documentation and data available from the projects to conduct desk-based assessments in preparation for the fieldwork phase. Partner staff and SV staff will accommodate the evaluation team’s needs and facilitate the fieldwork including access to the relevant communities and project sites for data collection and setting up meetings with beneficiaries, community leaders, local authorities, relevant partner staff and other key stakeholders. The evaluation team will coordinate with SV MEAL advisor and Turkiye/Syria Program Officer and should discuss any technical and methodological matters that arise.

The evaluation team is expected to hold a post-fieldwork meeting with SV and partner staff and other stakeholders to present and discuss preliminary findings. Draft reports are expected to be shared with partner staff, HQ staff and SV staff to incorporate feedback and ensure factual correctness before final versions are signed off prior to completion of the final evaluation report. The final version of the evaluation report will be provided to both SV and shared with the partners.

The evaluation team is expected to deliver the following key deliverables:
• Inception Report (of a maximum of 5 pages) produced by the lead consultant within two weeks of contract being signed detailing the evaluation team’s research design and intended methodologies (for data collection and analysis), activity workplan, team responsibilities and overall analysis of the desk-based review and secondary data from partners;
• Project Evaluation Report with detailed findings and recommendations specific to this project;
• Summary of Evaluation Report for wider distribution (including in-country sharing with local stakeholders, such as beneficiaries, local authorities, etc.)
• The Project evaluation report must be written in English and Turkish not exceed 25 pages (excluding annexes). The report must be written in Word and submitted in one stamped paper and one electronic copy. The draft report will need to be presented to SV for final input and fact checking before the final version is submitted.
• The written report should be based on (but not limited to) the following framework:
i. An executive summary (2 pages maximum)
ii. A methodology section outlining the inception report content and planning process, make- up of the team (e.g. sectoral expertise, local knowledge, gender balance) and its appropriateness and independency for the evaluation and any potential biases.
iii. A constraints, limitation (including reflexivity) and challenges of the evaluation section
iv. A general context analysis of challenges and good practices of Localization after the earthquakes in Turkiye, and how the projects are situated within it.
v. A findings section in line with the set evaluation questions discussed in detail
vi. A recommendations and lessons learned section following the analysis of the findings and the discussion with key stakeholders of the preliminary findings. It will be structured with specific subsections for each evaluation question, partner and project, ensuring that findings are tailored to each context/question providing general conclusion for each and any observed differences are explained.
vii. List of Annexes, including to the minimum: the present ToR, the inception report, sample of the data collection tools used, a list of sources of information used/documents reviewed, any relevant graphic material that could not be placed in the report.