Terms of Reference
Independent External Evaluation
Project Title:
Strengthening the Concept of Basic Literacy Education
Curriculum Through Local Potential for Elementary Schools Sumba Barat Daya
Funder: William & Lily Foundation
Project Period: December 2017 – November 2020
Evaluation Period: October 2020 – January 2021
Introduction & Rationale for Evaluation
The William & Lily Foundation (WLF) is an Indonesian-based philanthropic foundation established in 2009 as an outgrowth of the philanthropic legacy of William and Lily Soeryadjaya. As an active grant-making foundation, WLF works closely with implementation partners on four focus areas: education, health care, economic empowerment and enabling environment. WLF is committed to impactful and responsible interventions in its efforts to empower underprivileged communities across Indonesia. Although the scope of WLF philanthropic endeavours is countrywide, its current geographical focus centres on eastern Indonesia, particularly on the island of Sumba.
In December 2017, WLF formed a partnership with Kuark International, in implementing a two-year program entitled Strengthening the Concept of Basic Literacy Education Curriculum through Local potential For Elementary Schools in Sumba Barat Daya. This project is called Kelas Lentera. The project aims to improve the quality of teacher’s teaching and students' basic literacy and numeracy skills by strengthening teacher’s capacity on literacy and numeracy learning strategy, namely play based learning (i.e. drama class), brain training (i.e., brain fit, tik-tok class, tik-tok book) as well as designing media and teaching materials and facilitating reflection and evaluation of those activities. The beneficiaries of this project are the school principals, teachers and students in three supported elementary schools in Sumba Barat Daya.
As per WLF’s policy, this project is subject to external independent evaluation. The main purpose of this activity is to evaluate the effectiveness of project implementation and the appropriateness of chosen interventions to target beneficiaries. WLF therefore is seeking individual or a team of evaluators to conduct this evaluation.
Purpose, Scope and Users of Evaluation
The purpose of this external independent evaluation is to confirm efficacy of the program approach, particularly with the addition specific interventions on the second half of the program such as Brain fit activity and the use of alternative communication media (i.e., WhatsApp, etc.) and to validate results and potential replication of this project. The evaluation will cover the project's various components (such as the expected outcomes, outputs and activities) as reflected in the project document as well as subsequent modification and alterations made during its implementation.
The main user of findings and recommendations from this evaluation will be the William and Lily Foundation. Other potential users may include but not limited to the Foundation’s implementing partners and relevant stakeholders as necessary.
Methodology
The evaluation will be based on a participatory approach, involving a wide range of selected key Stakeholders and beneficiaries. The evaluation will use both primary and secondary data sources for probing the progress and for generating evidence in support of the findings. Overall, the evaluation should refer the OECD-DAC criteria for evaluation (where possible) which is looking into the elements of relevance, coherence, effectiveness, efficiency, sustainability and impact. To the extent possible, quantitative and qualitative data will be collected, validated, analysed and triangulated. The evaluation process will include a desk review, interviews, and observation. The proposed methodology is suggestive and can be adjusted by the Evaluator if considered necessary, in accordance with the scope and purpose of the evaluation and in consultation with the WLF team.
Evaluation Criteria and Suggested Questions
The Project will be evaluated against criteria such as its relevance to organization mission and objectives of organization, the validity of project design, project effectiveness, the efficiency of resource use and potential of replication or scalability. The evaluation should address the following key questions:
1. Relevance and value to organization strategy and mission statement
· To what extent does the project design take into account the needs of beneficiaries and stakeholders?
2. Validity and relevance of design
· To what extent is the monitoring and evaluation framework appropriate in assessing the project’s progress and useful in refining the project design?
3. Project achievements and effectiveness
· To what extent is the program implementation address the needs and challenges of beneficiaries and stakeholders?
· What are the main factors –internal to the project and external- that have hindered the project’s ability to reach the desired results thus far?
4. Efficiency of resource use
· To what extent have technical and administrative support been sufficient and adequate to meet project objectives?
5. Potential of replication or scalability in relevance to the target community
· To what extent are planned results of the project likely to be replicated and scaled up by stakeholders?
Evaluator’s Responsibilities and Deliverables
1. Key Responsibilities
· The design, planning and implementation of the evaluation and the write-up of the evaluation report, using an approach agreed with WLF and for delivering in accordance with the WLF’s specifications and timeline
· Consulting and liaising, as required, with WLF, stakeholders and partners to ensure satisfactory delivery of all deliverables
· Making herself/himself available, if required, to take part in briefings and discussions, online or other venue, on dates to be agreed
2. Key Deliverables
· Inception Report with Methodology
The inception report should detail the evaluator’s understanding of what is being evaluated and why, showing how each evaluation question will be answered by way of: proposed methods; proposed sources of data; and data collection procedures. The inception report should also include an evaluation framework matrix, a stakeholder list, proposed interview tools/guides, proposed schedule of tasks, activities and deliverables
· Draft of Evaluation Report
Independent external consultant will submit draft evaluation report to WLF. The recommendations, lessons learned, and emerging good practices should be documented in the draft evaluation report. WLF will provide feedback on the draft evaluation report
· Presentation of Draft Report
A presentation should be prepared for the WLF on the draft report, to be used during the debriefing
· Final Evaluation Report
To be submitted to WLF as per report outline and timeline. The quality of the report will be determined based on quality standards defined by WLF. The report should also, as appropriate, include specific and detailed recommendations by the Evaluator based on the analysis of information obtained. The report should also include a specific section on lessons learned and good practices that are relevant for better performance and delivery for future project or could be replicated or those that should be avoided
Proposed Time Line
The evaluation is foreseen to be undertaken in the time period, October 2020 to January 2021 with the aim to submit the final evaluation report to WLF no later than mid-January 2021.
Proposal Submission Criteria
The following will be considered minimum contents of the proposal. Please submit in the order listed:
Proposed methodology and tools for the evaluation design
Proposed team composition (clearly stating roles and responsibilities of each team member).
Financial proposal to conduct the evaluation, complete with breakdown of man-hours or days required to conduct each stage of the preparation, data collection tools development, data collection and analysis, field visit/travel, report writing and presentation
Most current Curriculum Vitae, highlighting the related experiences and skills to the required scope of work.
Sample of previous evaluation reports
Essential Requirements:
A bachelor’s degree in education sciences or any relevant social sciences, Master’s degree is preferred
A minimum of 5 years of experience conducting monitoring and evaluation of development programs in impoverished regions in Indonesia, particularly in carrying basic evaluation methodology
Demonstrable research and strong analytical skills
Demonstrable strong knowledge on basic literacy education in elementary schools, having strong pedagogic background on literacy and numeracy particularly for primary school and knowledge of inquiry-based methods
Strong interpersonal skills in a multicultural setting, experienced in engaging principals, teachers, Students in elementary schools and key government officials at subnational levels
Proposals must be sent to support@wlf.or.id by 27 September 2020 at 05:00 PM Jakarta time at the latest.
