Terms of Reference
Independent External Final Evaluation
Introduction & Rationale for Evaluation
The William & Lily Foundation (WLF) is an Indonesian-based philanthropic foundation established in 2009 as an outgrowth of the philanthropic legacy of William and Lily Soeryadjaya. As an active grant-making foundation, WLF works closely with implementation partners on four focus areas: education, health, economic empowerment and enabling environment. WLF is committed to impactful and responsible interventions in its efforts to empower underprivileged communities across Indonesia. Although the scope of WLF philanthropic endeavours is countrywide, its current geographical focus centres on eastern Indonesia, particularly on the island of Sumba and Bali.
In October 2019, WLF formed a partnership with Platform Usaha Sosial (Plus) in implementing a two-year project entitled Business Development Support to Salt Farmers in Amed, Bali. This project is called Mekarya, which aims to provide capacity building assistance to the salt farmers community (“MPIG”) in Amed by achieving the following outcomes: 1) improved MPIG capacity to run better business practices; and 2) MPIG has a stronger pipeline in establishing business partnerships.
The Mekarya project’s interventions include providing one-on-one business coaching assistance to the management team of Amed Salt Farming Cooperative (also called MPIG Garam Amed) to improve their business literacy and skillset, and to design tools for their operations (such as for marketing and business processes) that can be used by the management team moving forward. Project’s beneficiaries are mainly members of MPIG Garam Amed. The project also collaborates intensively with key stakeholders, such as village government, civil society organizations and government institutions at district and provincial level (i.e., BPOM, Tourism office, Industry and commerce Office, Cooperative offices, etc).
As this project is nearing completion, WLF plans to conduct a comprehensive final evaluation that will provide a 360 degree outlook not only on project performance measured against its intended outcomes, but also on the effectiveness of project management and implementation process, the appropriateness of chosen interventions, as well as to evaluate whether this project has applied the value for money principal. As per WLF’s policy, this project is subject to external independent evaluation. WLF is therefore seeking individual or a team of evaluators to conduct this evaluation
Purpose, Scope and Users of Evaluation
The first objective of this independent evaluation is to provide a comprehensive evaluation on the project management and implementation process performed by the implementing partner, in this case Platform Usaha Sosial (PLUS); particularly on how the project was able to meet the agreed upon project objectives, outcomes and outputs, as well as to evaluate whether the project has been carried out in a cost-efficient manner. The second objective is to analyse and document the challenges faced, lessons learned and, if any, best practices produced. Last but not least, this evaluation also aims to understand the efficiency and effectiveness of the grants and technical support WLF provided to the implementing partner during the lifetime of the project.
The evaluation will cover the project's various components (such as the expected outcomes, outputs, and activities) as reflected in the project document as well as subsequent modification and alterations made during its implementation. As a key objective of this project has been to build sustainability for MPIG’s business operations, this evaluation is also expected to cover the effectiveness of MPIG’s marketing strategies and partnerships with other parties as well as the strengthened institutional legal aspects. The main user of findings and recommendations from this evaluation will be the William and Lily Foundation. Other potential users may include but not limited to the Foundation’s implementing partners and relevant stakeholders as necessary.
Methodology
The evaluation will be based on a participatory approach, involving a wide range of selected key stakeholders. The evaluation will use both primary and secondary data sources for probing the progress and for generating evidence in support of the findings. Overall, the evaluation approach should refer to OECD-DAC criteria (considering factors such as relevance, coherence, effectiveness, efficiency, sustainability and impact). To the extent possible, quantitative and qualitative data will be collected, validated, triangulated and analysed. The evaluation process will include desk reviews, interviews and observation. The proposed methodology is not fixed, however, and can be adjusted by the Evaluator if considered necessary, in accordance with the scope and purpose of the evaluation and in consultation with the WLF team.
Evaluation Criteria and Suggested Questions
The Project will be evaluated against criteria such as its relevance to organization mission and objectives of organization, the validity of project design, project effectiveness, the efficiency of resource use, the impact orientation and sustainability. To summarize, the external independent evaluation should address the following key questions:
Relevance and value to organization strategy and mission statement
To what extent does the project design take into account the needs of beneficiaries and stakeholders?
To what extent does the project design align with WLF’s mission and objectives?
Validity and relevance of design
Are the planned project objective, outcomes, outputs and activities aligned?
Project achievements and effectiveness
To what extent has the project achieved their stated objectives?
What are the main factors – internal to the project and external – that have hindered the project’s ability to reach the desired results thus far?
Efficiency of resource use
To what extent have technical and administrative support been sufficient and adequate to meet project objectives?
Effectiveness of grant management; and
To what extent have the WLF grant management support been adequate to meet project objectives?
Impact orientation and sustainability in relevance to the target community
To what extent are planned results of the project likely to be sustained and/or scaled up and replicated by stakeholders?
What further concrete steps could be taken to increase the sustainability of the results?
Evaluator’s Responsibilities and Deliverables
Key Responsibilities
The design, planning and implementation of the evaluation and the write-up of the evaluation report, using an approach agreed by WLF and for delivering in accordance with the WLF’s specifications and timeline
Consulting and liaising, as required, with WLF, stakeholders and partners to ensure satisfactory delivery of all deliverables
Making herself/himself available, if required, to take part in briefings and discussions, online or other venue, on dates to be agreed
Key Deliverables
Inception Report with Methodology
The inception report should detail the evaluator’s understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods; proposed sources of data; and data collection procedures. The inception report should also include an evaluation framework matrix, a stakeholder list, proposed interview tools/guides, proposed schedule of tasks, activities, deliverables, and outline of final report.
Draft of Evaluation Report
The Evaluator will submit draft of evaluation report to WLF. The recommendations, lessons learned, and emerging good practices should be documented in the draft evaluation report.
Presentation of Draft Report
A presentation should be prepared for the WLF on the draft report, to be used during the debriefing.
Final Evaluation Report
To be submitted to WLF as per report outline and timeline. The quality of the report will be determined based on quality standards defined by WLF. The report should also, as appropriate, include specific and detailed recommendations by the Evaluator based on the analysis of information obtained. The report should also include a specific section on lessons learned and good practices that are relevant for better performance and delivery for future project or could be replicated or those that should be avoided.
Team Composition
The evaluation will be conducted by a team comprised by a team leader and a team member.
Proposed Time Line
The evaluation is foreseen to be undertaken in the time period of September 2021 to November 2021 with the aim to submit the final evaluation report to WLF no later than end of November 2021. The actual implementation of this evaluation will depend on the pandemic situation where movements are restricted
Proposal Submission Criteria
The proposed evaluation design must take into account Covid-19 health protocol and alternative methods of data collection during the period of restricted movement. The proposal should at least contain the following information:
Proposed methodology and approaches for the evaluation design
Proposed team composition (clearly stating roles and responsibilities of each team member)
Financial proposal to conduct the evaluation, complete with breakdown of man-hours or days required to conduct each stage of the preparation, data collection tools development, data collection and analysis, field visit/travel, report writing, and presentation
Most current Curriculum Vitae, highlighting related experiences and skills to the required scope of work
Sample of previous evaluation reports
Essential Requirements:
A bachelor’s degree in political sciences, international relations, development economics or any relevant social sciences (Master’s degree is preferred)
A minimum of 5 years of experience conducting monitoring and evaluation of development programs in impoverished regions in Indonesia, particularly in carrying rigorous evaluation methodology
Demonstrable research and strong analytical skills
Demonstrable strong knowledge on farming (salt farming specifically is preferred), business development and community empowerment
Experience in workshop facilitation
Strong interpersonal skills in a multicultural setting, experienced in engaging community leaders and key government officials at subnational levels
How to Apply
Interested organizations/consultants are invited to send a proposal (including cover letter, proposal, budget, most updated CVs of team members, and samples of previous evaluation reports) through the following link, https://tinyurl.com/mekarya-eval before 6 August 2021 at 23:58 Jakarta time. Only shortlisted candidates will be contacted.
