Evaluator for Final Evaluation (Consultant)

  • Contract

Company Description

CARE International Indonesia (CII), as an international NGO, has carried out large scale operations in Indonesia, encompassing emergency operations, transitional activities centered on agriculture and nutrition and a range of development initiatives in such areas as water and sanitation, health and enterprise & economic development.

 

SCOPE PROSPER PROJECT

Over 27 months of implementation, PROSPER has directly targeted 6,000 elementary students, 300 teachers, 1,200 parents and 100 canteen (cafeteria) workers in 20 schools in Banten and South Sulawesi provinces. The program contributes to CARE’s 2020 Program Strategy and UN Sustainable Development Goal (SDGs) by targeting 25,600 community members (indirect beneficiaries) to make healthier food choices and improve their hygiene habits. CARE will realize this by reducing their vulnerability to diarrhea and other infectious diseases.

 

The following are the four objectives under PROSPER:

Objective 1: Government and NGOs have greater capacity to support schools with the construction and rehabilitation of water and sanitation facilities, and promotion of clean and healthy behaviour and good nutrition practices. To achieve objective 1, PROSPER implements the following key activities:

  • Joint CARE, NGOs and district government working group established in each district.
  • 40 schools pre-selected and assessed.
  • 20 schools selected for program participation.
  • Construction/rehabilitation, clean and healthy behaviors and nutrition guidance manuals developed, approved and distributed.
  • 45 master trainers selected and trained on construction, nutrition, hygiene (two days each) and facilitation skills (one day).

 

Objective 2: Elementary students and teachers have increased access to clean water and sanitation facilities. To achieve objective 2, PROSPER implements the following key activities:

  • 20 school construction/ rehabilitation plans developed and approved by Public Works Department.
  • 20 water and sanitation committees established and trained.
  • 20 schools have WASH facilities constructed/ rehabilitated and made official at grand opening events.

 

Objective 3: Students and communities have improved hygiene practices. To achieve objective 3, PROSPER implements the following key activities:

  • 52 little doctor trainers and mentors trained.
  • 300 little doctors selected and trained.
  • Little doctor promotion events, competition and school exchange activities implemented.

 

Objective 4: Students and communities make healthier food choices. To achieve objective 4, PROSPER implements the following key activities:

  • 40 teachers and 12 community health center staff trained as nutrition trainers and coaches.
  • 300 students, 1,200 parents, 180 community members and 100 canteen workers trained.
  • ·Activities that promote nutrition and the availability of healthy food in schools implemented.

 

Table 1. Geographic area, population coverage, key participants, direct, and indirect beneficiaries.

  • Direct Beneficiaries : 2 Water Supply and Environmental Sanitation District Working Groups
  • Male : 20
  • Female : 25
  • Total : 45
  • Location : Makassar (South Sulawesi) and Serang (Banten)
  • Direct Beneficiaries : 20 Schools
  • Male : -
  • Female : -
  • Total : -
  • Location : Makassar (South Sulawesi) and Serang (Banten)
  • Direct Beneficiaries : 20 WASH and Nutrition Committees
  • Male : 67
  • Female : 133
  • Total : 200
  • Location : Makassar (South Sulawesi) and Serang (Banten)
  • Direct Beneficiaries : 6,000 Elementary Students
  • Male : 3,000
  • Female : 3,000
  • Total : 6,000
  • Location : Makassar (South Sulawesi) and Serang (Banten)
  • Direct Beneficiaries : 300 Teachers
  • Male : 150
  • Female : 150
  • Total : 300
  • Location : Makassar (South Sulawesi) and Serang (Banten)
  • Direct Beneficiaries : 1,200 Parents
  • Male : 400
  • Female : 800
  • Total : 1,200
  • Location : Makassar (South Sulawesi) and Serang (Banten)
  • Direct Beneficiaries : 100 Canteen Workers
  • Male : 67
  • Female : 33
  • Total : 100
  • Location : Makassar (South Sulawesi) and Serang (Banten)
  • Direct Beneficiaries : 12 Community Health Center Staff
  • Male : 4
  • Female : 8
  • Total : 12
  • Location : Makassar (South Sulawesi) and Serang (Banten)

 

  • Indirect Beneficiaries : 25,600 Community Members[1]

    [1] 25,600 = 4 x (6,000 students+300 teachers+100 canteen workers); four per household.

  • Male : 12,800
  • Female : 12,800
  • Total : 25,600
  • Location : Makassar (South Sulawesi) and Serang (Banten)

 

Table 2. Project partners, roles, and responsibilities

  • Project Partners : Partner NGOs
  • History/ Experience and Role in the Project Location : CARE established partnerships with NGOs that operate in the district, to support the implementation of the program. NGOs will work directly with the schools on assessments, construction/rehabilitation and training on clean and healthy behaviours and nutrition.
  • Project Partners : Members of the Water Supply and Environmental Sanitation District Working Groups (AMPL)
  • History/ Experience and Role in the Project Location : 
  • a. Planning and Development Agency: The Planning and Development Agency coordinated AMPL activities, mostly consisting of Public Works, the Water Supply Agency, the Health Department and the Education Department. CARE will work with the Planning and Development Agency for program management and coordination.
  • b. Public Works : The collaboration with the Department of Public Works linked to the construction/rehabilitation of WASH facilities and through their participation on the AMPL. CARE sought input from the Department of Public Works on the construction/rehabilitation curriculum to ensure the proposed facilities met district standards. CARE facilitated school public works communications related to construction plans, following standardized procedures that should have led to permits and potential leverage of funding (if required).
  • c. Water Supply Agency : CARE coordinated with the district Water Supply Agency to facilitate water supply to the schools. 
  • d. District Government Health Service and Education Service : CARE worked with the district Health and Education Services and contribute to the national UKS and PHBS programs and nutrition objectives. CARE will seek input on clean and healthy behaviours and nutrition training manuals, involve staff in training-of-trainers, and will stimulate the departments to leverage funding and replicate activities.

Job Description

PURPOSE OF ASSIGNMENT

The final evaluation consultant is assigned to design, implement and present the project endline evaluation to collect both quantitative and qualitative information on the PROSPER Indonesia project and be able to measure achievement of the project activities over the last 27 months. The PROSPER Indonesia project activities include: building of WASH construction facilities, promotion of healthy-clean behaviour (i.e. handwashing), and nutrition trainings and related events during the project implementation. Students, teachers, canteen workers, government stakeholders, parents, and street vendors in the project sites are notable target groups of the project and will be eligible for inclusion in the final evaluation. The consultant, in collaboration with the PROSPER Indonesia project team, will agree on the approach, methodology, and indicators for the endline.

 

RATIONALE

The final evaluation is needed to get the whole picture of the project results. The data from the evaluation not only can be used to see the achievement of the project, but also to collect the lessons learned and the intended and unintended impacts of the project. Furthermore, the result of the final evaluation can also be used as the source of knowledge if CARE/ the donor will implement a similar project(s) in a different scale or areas.

 

CONTRACT DESCRIPTION

The evaluator will conduct endline data collection and analyses working in coordination with the PROSPER team in CARE Indonesia. The endline evaluation will take a mixed-methods approach, employing both qualitative and quantitative techniques. Primary data will be collected using stratified random sampling and the sample will be beneficiary based to ensure comparison against the baseline. Primary data collection instruments and data collection methodology, including Knowledge, Attitude and Practice (KAP) tools, will be identical to those of the initial baseline survey and shared with the consultant. Qualitative key informant interviews and focus group discussions will also be employed to help enhance our understanding of the project’s intervention participation and effectiveness. Qualitative data will allow triangulation with quantitative findings.

The evaluator should focus on the level of achievements of the intended projects results. These will include: project document review and analysis; observational data collection; focus group discussions; questionnaire-based survey (KAP survey to be shared by CARE team); sample and sampling procedure. The evaluator will be responsible for defining and carrying out the overall evaluation approach. This will include specification of the techniques for data collection and analysis, structured field visits and interactions with beneficiaries and the evaluation team. All evaluation activities will be coordinated with the PROSPER Indonesia project team. The Program Manager of the PROSPER Indonesia project will be in regular communication with the hired consultant to coordinate the implementation of evaluation activities; additionally, CARE Indonesia will also participate in /support this process.

 

SCOPE OF SERVICES

  1. Quantitatively determine whether there has been an improvement in the knowledge, attitude and practice of the health and clean behaviour of the schools’ environment, capturing change in the indicators reported in the PROSPER Indonesia baseline report.
  2. To capture qualitative data on the awareness, attitude and practice of health and clean behaviour as well as observational data on WASH practices and infrastructure.
  3. To provide an objective assessment of the achievements and results, weaknesses and strengths of the project, as well as the sustainability of the project.
  4. To document evidence, lessons learned and good practices per regional and nationally to inform future WASH at schools programming.
  5. Assure data quality during field data collection in coordination with PROSPER project team.
  6. Provide electronic version of any field notes or observations during data collection to PROSPER Project team.
  7. Export complete data and provide endline data set to PROSPER Project in SPSS format (.sav) – (see data required here)
  8. Conduct basic frequency and cross-tab analysis as needed; any bar charts as needed
  9. Write and submit complete endline report using provided analysis plan to PROSPER Project.

 

EXPECTED OUTCOMES & DELIVERABLES

  • Qualitative tools including consent form
  • Research protocol describing sampling methodology, etc.
  • A complete final evaluation report which includes:

o   Clear presentation of quantitative comparisons of indicators

o   Clear presentation of “themes” that came out of FGDs and KIIs

o   Clear conclusions and lessons learned

o   Clear recommendations for PROSPER – in terms of changes if the project were to repeat work again in another place

o   Photos that show impact and tell a story (photos of focus groups answering questions or workshops are insufficient to make the human connection);

  • Quotes and Human-Interest Stories
  • A power point presentation which summarizes the brief
  • A report brief that is no more than 5 pages in length
 

FINAL REPORT REQUIREMENTS

The external evaluator is accountable to maintain the requirements for the content, format, or length of the final report, overall quality and approved timelines. They will produce a comprehensive report that assesses the achievements, relevance, coherence, coverage, effectiveness, efficiency, outputs and early outcomes of title of project/program/initiative so far and provide prioritized recommendations to maximize results.

 The report must include:

 • A Title: A title that conveys the name of the project, location, implementation period, as well as the main impact or key finding of the report.

• An executive summary that focuses both on process as well as impact that is no more than 2 pages in length and is formatted so that it can be printed as a stand-alone 2-pager about the project.

A display of impact early in the report, including 3-5 key impacts/findings: What changed because of the program? What happened in the world, and why did it matter? This are the most significant accomplishments, supported by solid evidence. Each impact should be written as one or two sentences. Talk about impact early on the report so that the audience does not have to read the entire report before seeing evidence of change.

• 3-5 key lessons learned: These should be short, actionable, and the most important aspects of what the program/analysis found. They need to be relevant and new for people outside of the direct program.

3-5 bullets describing how the project got to impact/3-5 recommendations: It is important to have non-jargon descriptions of what a project did to get to impact. These are highlights of the most effective, relevant, and scalable approaches and tools. If this is an analysis and not an evaluation, then this section should be 3-5 key recommendations for what the project/program/initiative should do based on your findings.

• Shareable Evidence: Clearly separate evidence collected by the external evaluation from the conclusions and recommendations must be submitted along with the final report. Sources of all evidence must be identified (e.g. primary data sets) and conclusions must be based only on evidence presented in the report, and recommendations must directly correspond to the conclusions.

·       Sustainability assessment: Include 1 page on the sustainability of the project, including uptake of and investment in PROSPER Indonesia project practices by other actors

 

DISTRIBUTION STRATEGY

The evaluation findings and processes will be used and shared by relevant stakeholders (donors, CARE Indonesia HQ, CARE US, government and schools). PROSPER Project team will keep them informed about the evaluation progress (CARE Indonesia HQ and CARE US) and present completed findings to the donor, government and schools.

 

DURATION

The contract will be conducted over a period of <30 days >.  

 

CONDUCT OF THE WORK

During data collection and analysis, the primary roles of CARE program staff and any implementing partner with direct stake in the project, are as informants and reviewers. They may review and provide comments on data collection tools, instruments, and all other deliverable before they are finalized. They must not collect primary data, or participate in translation, analysis, or interpretation of the data. During the contract the consultant will report directly to Lola Amelia, Project Manager of PROSPER.

Qualifications

CONSULTANT SPECIFICATIONS

Overall, the consultant (team) should have knowledge, experience and competencies in:

  1. At least 5 years’ experience on project evaluation, assessment, and impact study with using quantitative and qualitative design and methods.Experienced in conducting similar evaluation which is proven by stating at least three evaluation titles in the cover letter;
  2. Having enough capacity to implement this evaluation which is proven in the CVs of the evaluators showing that they have managed a similar evaluation on WASH project.

Additional Information

DATA DISCLOSURE

The external evaluator should deliver, at minimum, all files including: quantitative data sets (raw and refined products), transcripts of qualitative data and debriefing notes from every qualitative interview or FGD in an easy to read format and maintain naming conventions and labelling for the use of the project/program/initiative and key stakeholders.

All documents should be compliant with the following conditions (see data format requirements):

  • CARE requires that the datasets that are compiled or used in the process of external evaluation are submitted to CARE when the evaluation is completed.
  • Data must be disaggregated by gender, age and other relevant diversity, etc.
  • Datasets must be anonymized with all identifying information removed. Each individual or household should be assigned a unique identifier. Datasets which have been anonymized will be accompanied by a password protected identifier key document to ensure that we are able to return to households or individuals for follow up. Stakeholders with access to this document will be limited and defined in collaboration with CARE during evaluation inception.
  • In the case of textual variables, textual datasets or transcripts please ensure that the data is suitable for dissemination with no de-anonymizing information UNLESS these are case studies designed for external communication and suitable permission has been granted from the person who provided the data.  In these circumstances, please submit, with the case study, a record of the permission granted, for example a release form.
  • Where there are multiple datasets (for example both tabular and textual datasets) identifiers must be consistent to ensure that cases can be traced across data lines and forms.
  • CARE must be provided with a final template of any surveys, interview guides, or other materials used during data collection. Questions within surveys should be assigned numbers and these should be consistent with variable labelling within final datasets.
  • Formats for transcripts (for example: summary; notes and quotes; or full transcript) should be defined in collaboration between CARE and the external evaluator at the evaluation inception
  • In the case of tabular datasets variable names and variable labels should be clear and indicative of the data that sits under them. Additionally, the labelling convention must be internally consistent, and a full codebook/data dictionary must be provided.
  • All temporary or dummy variables created for the purposes of analysis must be removed from the dataset before submission. All output files including calculations, and formulae used in analysis will be provided along with any Syntax developed for the purposes of cleaning.
  • We require that datasets are submitted in one of our acceptable format types. 
  • CARE must be informed of and approve the intended format to be delivered at evaluation inception phase. Should this need to be altered during the project CARE will be notified and approval will be needed for the new format.
  • The external evaluator will be responsible for obtaining all necessary permissions, approvals, insurance, and other required permits needed for data collection. These include required permits related to data collection from human subjects, including necessary ethical review board approvals (ERB) and health and accident insurance for evaluation team members.

 

CARE’S EVALUATION PRINCIPLES AND STANDARDS

The evaluation should always respect the security and dignity of the stakeholders with whom CARE works, incorporating gender and power elements (see CARE’s gender analysis framework) during the evaluation. To gain a better understanding of potential differences in gender and power elements, evidence should be able to be disaggregated by sex, age and other relevant diversity, etc.

 

TERMS OF OFFER

Interested applicant must submit the following documentation:

  1. Comprehensive curriculum vitae of individual/team members involve in the activities.
  2.  A detailed implementation plan/technical proposal in accordance with the scope of work, expected outputs and deliverables provided for herein.
  3. Financial Proposal that indicates the all-inclusive fixed total contract price, supported by a breakdown of costs i.e. consultancy fee, ect.

 

Please submit your applications latest on Friday, January 18, 2019.

"Only qualified applicants will be shortlisted"