TOR
PULIH BERSAMA FINAL EVALUATION
DECEMBER 2022 - FEBRUARY 2023
A. Background of Project Information
The Australian Humanitarian Partnership (AHP) CAN DO Indonesia COVID-19 Surge Response: Pulih Bersama ("Pulih Bersama Project") is a 15-month project supported by the Australian Government and implemented by the CAN DO consortium. The consortium is comprised of four Australian organizations who have been operated the Pulih Bersama project through Indonesian-based partners and local civil society implementing NGOs in nine provinces: ADRA Australia partners ADRA Indonesia have implemented the activities in Aceh, East Java, West Java, South Sulawesi, Nusa Tenggara Timur (NTT), Papua and West Papua; Caritas Australia partners with Catholic Relief Services (CRS) have implemented the activities in Central Java, West Java, East Java and Aceh in collaboration with local implementing partner Human Initiative (HI); Act for Peace partners Church World Service (CWS) have implemented the activities in South Sulawesi, and NTT with local implementing partner Yayasan INANTA and PMPB; and Uniting World partners Maha Bhoga Marga (MBM) have implemented the activities in Bali. The Pulih Bersama's overarching goal is to mitigate the impacts of the COVID-19 pandemic by supporting subnational authorities in vaccination efforts, providing psychosocial support in targeted priority communities, and improving families' ability to absorb the shock to their livelihoods. Targeted populations will benefit from multiple outcomes through an integrated response covering the following outcomes:
1. People in target communities, especially the elderly, persons with comorbidities, pregnant women, people with disabilities, and other marginalized groups have improved knowledge, attitudes, and practices on issues and impacts of the COVID-19 pandemic.
2. Sub-national health authorities manage the spread and impact of COVID-19 through increased uptake of vaccines targeting priority groups (including elderly, persons with comorbidities, pregnant women, PWDs, and other marginalized groups).
3. Local communities, including marginalized people, are provided with psychosocial support and referrals.
4. People with livelihoods affected by COVID-19 have improved access to livelihood support to meet basic needs.
Pulih Bersama has entered last stage of program implementation and needs to see overall project effectiveness (including final achievement), project efficiency (correlation between budget spent with actual achievement), project relevancy (to determine whether it's connection with the needs community), project impact, and sustainability. These five key components will be measured in a final evaluation and proposing the management team a strategic recommendations that will be useful for exit strategy possibility and perhaps an extension of timeframe of the project. The final evaluation result will be utilized for various purposes of Pulih Bersama members consortium in terms of finding dissemination and advocacy purposes.
B. The Objective of Final Evaluation
· To identify and analyze project effectiveness and efficiency through measurement of effectiveness, efficiency, timeliness, relevancy, sustainability, and impact within project timeframe
· To provide accountability to consortium member, donor, and other related stakeholders
· To analyze project significant change/impact in connectivity with beneficiaries lense
· To provide strategic recommendation to Pulih Bersama members consortium and donor in terms of project planning, scaling up, and replication
C. Key question of Final Evaluation
Evaluation design refer to Pulih Bersama logical framework. To assist the evaluation team, key questions will be built around the following aspects but not limited to :
1. Effectiveness
· What is the last status of the project? Is the project achieving all objectives from the above points? What are the main factors/challenges faced in the implementation of the project (only if the project didn't achieve target/objectives)?
· Did the project achieve its planned outputs (as per the detailed implementation plan) on the planned timeline?
· How does the effectiveness of project partner or stakeholder contribute to the project achievement?
2. Relevance:
· Do the objectives and goals of the project match with the problems or needs that are being addressed?
· Was the implementation strategy of the project is able to adapt and respond to changes? Why/why not? How the project to adapt and respond to these changes?
3. Efficiency
· Were the project program/activities delivered in a timely manner?
· Does the budget allocate for this project is comparable with the result achieved?
4. Impact
· To what extend the project could achieve and sustain? and to what extend will the impacts match the needs of the intended beneficiaries?
· How extent the project to improve knowledge, attitude, and behavior (KAP) on Covid 19 risk management issues?
· How the project having impact to knowledge, Attitude, Practice (KAP) of beneficiaries related to vaccination?
· What has been the quality of implementation in different sites?
· To what extent are differences in impact explained by variations in implementation
· How do the activities impact to the government agenda/action plan on vaccination at district level
5. Inclusiveness
· To what extent are response activities and processes protecting the safety, dignity, and rights of affected people?
· How does this project identify the needs of women, girls, those with a disability, and other disadvantaged people?
· How equitably are women, those with a disability, and other disadvantaged people benefitting from the project?
· To what extent has disability inclusion, gender and child protection been incorporated in livelihood support intervention?
6. Localisation
· To what extent are systems and processes of this project enable local women and men and communities to participate in and influence humanitarian responses (local and national) in place, and able to be sustained?
· To what extent are systems and processes of this project support national actors to define and lead on humanitarian action in place, and able to be sustained?
7. Sustainability
· To what extent will the changes achieved remain sustainable for the target groups without further project support?
· Are there any possibilities of project method or lesson learn to be continued by government in macro and micro level r to develop programs that are specific for Covid 19 risk management issue?
· Are there any possibilities of project method or lesson learn to be replicated or be scaled up by other institution?
D. Methodology
The consultant is requested to propose an adequate study methodology as specified by this TOR. It will inform the methodology details such as sampling technique, sampling frame, data technique collection and analysis potentially applied in this study. The study will be conducted online and/or offline in Pulih Bersama Project area where number of activities have been implemented. The method used should be child friendly, participatory, and inclusive wherever possible and do no harm policy should be strictly followed throughout the study considering the Covid-19 situation.
During the study, the consultant is tasked to undertake:
· Desk-based review on existing relevant documents
· Briefing session with Pulih Bersama evaluation team
· Administer questionnaires for respondents
· Focus Group Discussions technical guideline
Key Informant Interviews particularly in-depth interview with relevant stakeholders selected to participate in the study as well as the partner of Pulih Bersama program
The respondents of of final evaluation would be the targeted communities and key stakeholder who participated in Pulih Bersama project areas, such as but not limited to, community influencer, vaccine event participants, consortium member, government institution, religious leaders, custom leaders, education and health institution. Potentially it will be also cover findings dissemination to strengthen Pulih Bersama accountability.
E. Scope of Work
The consultant is expected to form a team for the evaluation and manage coordination and communication with Pulih Bersama Evaluation team. Tasks and responsibilities of the consultant are as follows:
· Proposal that detailing the methodology and budget required to conduct the final evaluation
· Recruit, train and manage final evaluation research teams including enumerators
· Propose tools to be used in the final evaluation research, including sampling frame and quality control scheme and discuss the suggestion with Pulih Bersama evaluation team
· Develop a data entry system and management
· Supervise field research implementation
· Responsible for field operations, including logistics, permission to conduct the research, informed consent from individuals and families taking part in the final evaluation research
· Coordinate with Pulih Bersama consortium member during field research implementation
· Collect, compile, and analyse all data (gender and age disaggregated) gathered and develop a final report on the final evaluation results (including all findings and statistics).
· To keep all information provided by CANDO Consortium as well as the findings of the Final Evaluation confidential.
· Facilitate the online workshop on evaluation findings for Pulih Bersama Consortium Member.
F. Outputs and Deliverables
The consultant is expected to produce and submit the following deliverables:
1) Study protocol specifying a detailed study work plan and proposed study tools that will be discussed and agreed upon prior to field activity.
2) Presentation of initial findings to Pulih Bersama Consortium, after field activities have been concluded.
3) Draft reports in Bahasa Indonesia and English in two weeks' time after field activities have been concluded
4) Presentation of draft and reporting data flow diagram to Pulih Bersama evaluation team for discussion and feedback.
5) Submission of a comprehensive final report in Bahasa and English one week after receiving feedback from Pulih Bersama Member team.
6) Executive summary (max. three to maximum five pages) in English describing the study results.
7) Hardcopies of filled-in questionnaires (if available), interview transcripts and attendance lists, photos with informative subtitles and other valuable study materials (For Pulih Bersama Documentation)
G. The Responsibility of Pulih Bersama Project /Pulih Bersama Consortium
· Provide a working contract
· Provide data and related documents as per consultant's request
· Arrange meetings, discussions and field visits for the study team
· Provide timely feedback and response on consultant's study report
· Settle payment for the consultant service
H. Timeline
The final evaluation expected to be started in mid of December and expected to be completed on 16 February 2023, then the report should be finished the latest on second week of February 2023.
| No | Activity | Date |
| 1 | Interview the candidate consultants | 3 – 4 January 2023 |
| 2 | Inception meeting | 6 January 2023 |
| 3 | Submission the evaluation tools etc | 9 January 2023 |
| 4 | Data collection | 10-20 January 2023 |
| 5 | Data analysis and reporting | 21-28 January 2023 |
| 6 | Submit the draft and review process | 28-31 January 2022 |
| 7 | Submit draft final and review | 3-7 February 2023 |
| 8 | Submit final report | 13 February 2023 |
I Report Content
Final report should be produced in Bahasa and English and should contain and be developed according to the agreed structure (attached)
J. Ethical Consideration
The evaluation team must ensure adherence to ethical guidelines as outlined in the American Evaluation Association's (AEA) Guiding Principles for Evaluators. A summary of these guidelines is provided below:
- Informed Consent: All participants are expected to provide informed consent following standard and pre-agreed consent protocols.
Informed consent form will be developed and translated in local language before the data collection training. The data collectors will be trained on these ethical guidelines to ensure its adherence. The respondents will first receive an explanation of the purpose of the study and that only respondents who have formally consented and agreed to participation, will be enrolled and interviewed.
2. Systematic Inquiry: Evaluators conduct systematic, data-based inquiries.
- Competence: Evaluators provide competent performance to stakeholders.
- Integrity/Honesty: Evaluators display honesty and integrity in their own behavior and attempt to ensure the honesty and integrity of the entire evaluation process.
- Respect for People: Evaluators respect the security, dignity and self-worth of respondents, program participants, clients, and other evaluation stakeholders. It is expected that the evaluator will obtain the informed consent of participants to ensure that they can decide in a conscious, deliberate way whether they want to participate.
6. Responsibilities for General and Public Welfare: Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation.
Source: AEA's Guiding Principles for Evaluators
K. Criteria for Consultant Selection based on competency and experience
Consultant interested in submitting a proposal should have the following criteria:
1. Possess equal composition of qualified academic background, knowledge, experience, and capacity to manage evaluation.
2. Have an extensive experience in managing large-size studies in the context of basic automechanic skill.
3. Demonstrate understanding and sensitivity on cross-cultural, tradition and language of project target.
4. Proven experience as an evaluator, preferably on Disaster Risk Management issues
5. Understanding of project management structures and change management concepts
6. Clear, effective writer in English.
7. Experience of working with participatory methodologies.
L.Budget Allocation
The available budget will be including professional fee, accommodation, transport and survey cost, and tax).
M. Procedure for Expression of Interest
First Phase: Interested consultant should submit an Expression of Interest to at: rfp.cwsindonesia@gmail.com the latest on 30 December 2022 by enclosing the following required documents:
1. Organizational Profile (for institutional consultant) or a CV (for individual consultant).
2. A proposal containing two parts:
· Technical Proposal comprising the description of proposed activities, methodology (sampling design, data collection technique), and implementation plan which includes study schedule and timeline for data analysis. The technical proposal should demonstrate understanding of study ethic, qualification of study team members and their CVs, as well as their roles in the study. The consultant should also describe her/his experience in conducting similar survey, and financial and technical capacity.
· Budget Proposal comprising details of budget plan such as consultant and team's fees, transportation, stationeries and other survey necessities.
Second Phase: Based on received proposals, Pulih Bersama Evaluation Team will shortlist candidates and invite them to present their proposals. Pulih Bersama will afterwards proceed with selecting and appointing a consultant and produce a working contract with the selected consultant.
Only shortlisted consultant will be contacted for recruitment.
