TERMS OF REFERENCE - MID-TERM EVALUATION I.
Background
WUSC is one of Canada's leading non-profit international development organizations committed to building a more equitable and sustainable world. The organization works with a unique and powerful network of post-secondary institutions, private sector partners and volunteers to provide educational, employment and empowerment opportunities that improve the lives of millions of disadvantaged youth around the world. CECI is a Canadian international development organization whose mission is to fight poverty and exclusion. CECI's Canadian and field teams have 15 years of experience in governance and inclusive development in the extractive regions of West Africa. WUSC and CECI are jointly implementing the West Africa Governance and Economic Sustainability in Extractive Areas (WAGES) project in key regions of Burkina Faso, Ghana and Guinea that are impacted by the presence of extractive industry. This implementation timeline for this 6-year, $22.18 million project is April 2016 to March 2022. The project aims to break the vicious circle in which local communities, especially women and youth, are excluded from the benefits of mining investments. Additionally, the project works to ensure the participation of local communities, and specifically, the integration of women and youth in the governance of natural resources, entrepreneurship and the sustainable development of these areas. The project staff work closely with key partners that include local and national governments, mining companies as well as private sector and civil society organizations. The project has three main areas of intervention. The first component, focusing on local governance, aims at strengthening regional and local governments and local institutions, and allowing them to leverage opportunities arising from natural resource development. The second component, focusing on sustainable and inclusive local economic development, aims to increase gender and youth inclusive employment opportunities for community members. The third component, focusing on knowledge sharing, aims to establish an enhanced informed dialogue on inclusive and sustainable strategies for economic growth and local governance in the targeted regions of Burkina Faso, Ghana and Guinea. It is important to mention that the project was conceived in the context where the legislative texts of the three countries of intervention provide that the local governments of the intervention zones will have access to financial resources derived from mining revenues to invest in local development. Although limited and unpredictable, its resources are accessible to the Districts of Ghana, the first funds have just reached the Communes of Guinea but these are still not a reality in Burkina Faso. There is, however, direct investment by mining companies to support local economic development in the 3 areas of intervention. The project's strategy is therefore to maximize the benefits of its local public and private investments to improve the well-being of populations living in extractive zones. The project’s Logic Model (LM) was revised in September 2018 to take into account lessons learned from the first years of implementation, as well as Canada’s newly released Feminist International Assistance Policy (FIAP). The 2018 revisions aimed to strengthen the project's focus on gender equality and youth inclusion. WAGES also has a Performance Measurement Framework (PMF) which guides process and results monitoring. During the 2018 LM revision
process, the project’s PMF was also revised which means that some of the project original indicators are not the same as those used in the project’s baseline study. After more than three full years of implementation, the WAGES project wishes to carry out an externally led mid-term evaluation as part of its overall monitoring, evaluation, research and learning strategy (MERL) for the project. For this reason, WUSC is looking for proposals from qualified bilingual (English and French) firms to lead and carry out this planned evaluation in the three implementing countries of the project. II.
Purpose of the evaluation
The purpose of this formative evaluation is to highlight what is working well and what needs to be improved in order for WAGES to make further progress in achieving expected outcomes. More specifically the objectives of the evaluation are threefold: ●
●
●
III.
To provide evidence on progress towards achievement of the projects’ expected outcomes, as well as evidence of any unexpected outcomes (including those that contribute to reducing inequalities between women and men as well as better inclusion and engagement of young people);; To generate knowledge about the project's achievements and challenges, barriers and enablers (while incorporating a gender perspective), and help disseminate this across key partners and stakeholders; and To provide practical recommendations (including those that address issues of equality between women and men and the issues of inclusion and engagement of young women and men) for implementation in the remaining period of the project. Scope of the evaluation
The time to be covered by this evaluation is from its baseline study in September 2016, to November 2019. The geographical focus will be targeted areas within the three countries in which the project works, more specifically: ●
● ●
IV.
Burkina Faso: the regions of Hauts-Bassins (province of Tuy) and Boucle du Mouhoun (province of Balé) in seven communes located in areas impacted by Houndé Gold Operations (HGO) and Roxgold: Houndé, Boni, Koumbia, Bereba, Bekuy, Bagassi and Pompoï. Ghana: Daboase, Ateiku, Bogoso, Prestea, Aboso and Huni -Valley. Guinea: in ten (10) communes of the prefecture of Boké: Bintimodia, Dabiss, Kamsar, Kanfarandé, Kolaboui, Malapouya, Sangarédi, Sansalé, Tanènè. Intended users and use
The primary intended user for this evaluation will be the WAGES's project implementation team (WUSC and CECI staff), plus key partner and stakeholder representatives more specifically the National Consultative Committees and the Steering Committee. These include Global Affairs Canada (the donor), plus local governments, CSOs, mining companies and local communities that have been targeted by the project.
To assure that the evaluation is used and has value-added for its intended users, the evaluation design must include opportunities for direct involvement of key stakeholder’s representatives in this mid-term review process, including validation of findings, and approval of recommendations.
V.
Evaluation questions
The following 3 research questions and 6 sub-questions will be addressed by the evaluation: 1. Have the sub-national governments and local institutions, supported by WAGES, leveraged opportunities that enable communities located in extractive areas, and particularly women and youth, to participate in and benefit from local development? ● Do key representatives of the communities that WAGES has supported, and especially women and youth, have increased ability to engage and hold accountable local government in the management of sustainable and inclusive development and the respect of rights? ● Do communities supported by WAGES have the capacity to ensure inclusion and engagement of youth in their development actions? ● Do local government institutions that have been targeted beneficiaries of WAGES support, have increased capacity and accountability to plan and manage inclusive and sustainable local socio-economic development? 2. Have communities in targeted extractive areas, and specifically women and youth, experienced economic empowerment related to support received through WAGES? ● Do businesses supported by WAGES; have improved ability to provide sustainable, local employment and revenue generation opportunities, especially for women and youth? ● Do the women and youth supported by WAGES have improved confidence and skills to leverage self-employment, employment and entrepreneurship opportunities? 3. Is the support from WAGES contributing to increased informed dialogue on inclusive and sustainable local economic development and governance strategies that benefit local communities, especially women and youth? ● Has there been increased opportunities for local, national and regional stakeholders, to exchange information and build knowledge on gender and youth inclusive local governance and sustainable economic development? ● Has participation by CSOs that represent women and youth, and by local government representatives, increased in national debates on extractive issues, and are their voices becoming more influential in the dialogue on inclusive and sustainable local economic development and governance strategies that benefit local communities, especially women and youth ? VI.
Methodology
This will be a formative, mid-term, mixed-methods, non-experimental evaluation with the emphasis on qualitative data collection. For quantitative data collection and analysis, descriptive rather than inferential statistics will be the focus. All data should be analyzed and disaggregated by sex and age (below 35 and over 35 for each sex). In cases where an organization (company, association, NGO, etc.) is referred to, it is also important to specify whether this type of
organization is managed by one or more women, one or more men, one or young women and one or more young men. The research questions, and key performance indicators that structure the evaluation must be aligned with the expected outcomes and indicators depicted in the project’s revised LM and Performance Measurement Framework (PMF). The data collection and analysis methods in this evaluation should be informed by, compliment, and support the project’s approved PMF, i.e., methods used should provide evidence for the indicators identified in the PMF. It is expected that the evaluator will design a methodology based primarily on the following methods: document review, key informant interviews (KKIs), focus group discussions (FGDs) including small group meetings with targeted direct project beneficiaries including women and youth. Sampling for household interviews will be purposive, and limited to collecting a range of views from direct project beneficiaries. The methodology should allow equal participation of women, men and young people and should be sensitive to the special needs of certain people with special needs such as people with reduced mobility or illiterate people. This methodology will allow to take into account and bring out the point of view of women and young people.The main activities that the evaluator will carry out will primarily include the following: ●
●
●
● ●
●
Participating in an assignment scoping meeting with the country team, the regional MEL Advisor and the gender specialists or advisors, during which, the expectations related to the mandate will be addressed and the further information for drafting the inception report will be shared. Reviewing all relevant documents, studies and other sources of data related to the WAGES project in the country concerned, including data and analysis of the baseline study and indicators and other relevant elements identified in the project’s PMF as well as the Gender Equality and Youth Inclusion (GEYI) strategy of the project.. In consultation with the project team, drafting the inception report including a detailed work plan for the evaluation, to be approved by the Evaluation Review Committee. The report should include the following: 1. Study methodology 2. Participative, inclusive and gender sensitive data collection strategy including sampling size of sample and questionnaires for focus groups (FGDs) and interviews (KIIs) 3. Detailed work plan, including all the tasks the evaluator and other team members, as well as the schedule of activities of the evaluation; 4. Detailed budget for the evaluation, including fees, expected reimbursable expenses, etc.; 5. Detailed investigator training plan 6. A copy of the tools for the collection of qualitative data, as well as a plan for necessary translations. Developing guidelines for investigators and protocols for data collection and ensure that the investigative team has at least 50% women. .. Coordinating and leading the training of investigators (including the participatory and inclusive data collection method), following the schedule included in the inception report and approved by the WAGES team. Coordinating, leading and supervising data collection, in accordance with selected methodology.
● ●
●
●
Ensuring the quality of data entry by verifying gender sensitive data entered in the country and providing appropriate comments and observations. Analyzing qualitative data. It is expected that the evaluator will conduct a critical analysis of the data, through appropriate processing, and triangulation with other sources including comparison with data presented in project documents and in relevant external literature. Drafting and delivering the final version of the study report, which should include the summary and the consolidated report. The final report must be sent in Word and PDF format, including all appendices and raw data. Instructions for the table of contents will be provided by the project. Copies of all original and cleaned data, including all field notes, must be sent to WUSC along with the final report. Participating in the restitution workshop of the evaluation to key stakeholders.
The evaluator must follow appropriate ethical procedures, including: ● ● ● ●
Ensuring that the dignity of all respondents and participants research participants' is respected Obtaining consent of the respondents Ensuring no one is subjected to harm in any way during data collection Assuring confidentiality and anonymity of the respondents
The evaluators must work in close collaboration with the WAGES team using inclusive and participatory processes that directly involve the WAGES M&E teams. Methods should include at least one event in each country to validate findings and discuss practical recommendations, followed by an event designed to aid validation and meta-analysis across the three countries and across the full project. Note that the total budget for this work is CDN $60,000. The methodology proposed by the Evaluator must take into account this amount, plus the timelines proposed (see below). This amount must be inclusive of all expenses directly related to the evaluation including any direct costs that would be incurred by project offices in the 3 countries.
VII.
Roles and responsibilities of the evaluator
Roles and responsibilities of the Evaluator: The Evaluator will be mobilized to lead and carry out the midterm evaluation, and will recommend a single competent team to carry out the study and coordinate the evaluation in the project intervention areas in the 3 target countries. The Evaluator may choose to lead three separate teams, one in each country. WUSC-CECI would need to be involved in the teams’ selection. The responsibility to collate data, findings, conclusions and recommendations across the 3 countries, and for the whole project, rests with the evaluator. The Evaluator will assume responsibility for all aspects of the evaluation, including its design, methods, finalization of data collection tools, data collection including any training required, data entry, data processing and analysis, validation of findings and recommendations, and report writing. The Evaluator will report directly to the Evaluation Manager: the Project’s Regional
Monitoring, Evaluation, and Learning (MEL) Advisor based in Burkina Faso. The Evaluator will work in close collaboration with the gender resources of the project. Roles and responsibilities of the Evaluation Manager: The WUSC Regional MEL Advisor in Burkina Faso has overall responsibility for managing the evaluation. She will be the conduit for providing all necessary support and guidance to the evaluator throughout the implementation of the study. She will be supported by the Lead Project Coordinator. She will be responsible for providing the evaluator with the country report template and the consolidated report. Further technical support to the will be provided by the WUSC MERL Advisor based at WUSC in Ottawa, Canada and the Gender advisors, based at CECI, Montreal. Roles and responsibilities of Evaluation Review Committee: An Evaluation Review Committee will provide input at three critical stages of the evaluation: approval of the inception report, approval of the draft evaluation report, and approval of the final report, including recommendations. The evaluation committee will include: -
The Regional MEL advisor The MERL advisor (HQ) The lead coordinator of the project The national project coordinators of each country The senior advisor, strategic partnerships The Montreal-based Gender CECI advisor
VIII.
Timeline and milestones
The contract period will be between November 11th, 2019 and February 15th, 2020. Estimated contributions expected in working days will be determined in consultation with the selected evaluator. The evaluator will have to put in place all the necessary actions to launch the evaluation, in accordance with the following schedule. Task
Indicative deadline
Signing of contract with WUSC
November 11th , 2019
Framing meeting[1]
November 11th (approximately 2 working days after the beginning of the contract)
Submission of the detailed work plan and draft start-up report, including data collection tools
November 19 (approximately 7 working days after the start of the contract) Please note: WUSC will send comments within three business days of receiving the draft report
Evaluation review committee’s comments
November 21, 2019
Submission of the final version of the inception report (incorporating comments from the evaluation review committee)
November, 28 (7 working days after the submission of the evaluation review committee comments)
Literature review
November 21 to 22 (2 days)
Recruitment, training of investigators and tools pretest
November 25 to 27 (3 days)
Data collection
November 29 to December 12, 2019 (14 days)
Validation, analysis and data processing
December 13 to 20, 2019
Drafting and submission of the draft mid-term report (in accordance with the report template provided)
January 17, 2020
Amendments from WUSC and transmission
January 20 - 24, 2020
Incorporating WUSC’s amendments in the interim report
January, 27 - 30, 2020
Restitution workshop for the mid-term evaluation report in the project regions in the country concerned
February, 03 - 07, 2020
Integration of the amendments resulting from the restitution workshop
February, 10-12, 2020
Submission of the final version of the mid-term evaluation report
February 15, 2020
[1]
Before the scoping meeting, all relevant documents (including the PMF) will be provided to the selected evaluator.
IX.
Qualifications of the evaluator’s team
Expertise in socio-economy ●
●
●
● ● ●
Minimum of 10 years of experience in technical and operational management of data collection and production of quality baseline, mid-term and final reports, preferably for non-profit international organizations or multilateral agencies, including multi-country evaluation management experience. Track record for conducting formative, largely qualitative formative evaluations for other large, complex development projects that are geographically spread over multiple countries. Demonstrated experience in designing baseline and final studies, including sampling experience, mixed methods (quantitative and qualitative), tool development, investigator training, etc. Excellent facilitation skills, as well as the ability to recruit and manage facilitators, for the qualitative components of the study. Demonstrated experience in analyzing quantitative and qualitative data integrating a gender perspective. Knowledge and experience in the field of local governance, extractive sector and local economic development in West Africa and preferably in the country concerned
● ● ● ● ●
Knowledge and experience in gender equality issues related to the above-mentioned sectors are preferred. Ability to integrate the gender approach into all stages of the project evaluation process. Participatory and inclusive skills in project evaluation. A good command of English or French according to the country is required, and local languages are preferable Ability to produce high quality work within tight deadlines.
At least one member of the team should have expertise in statistics: ● ● ● ● ● ● ● ● ●
X.
Hold a degree (Master’s) in statistics Have at least five (5) years experience in household surveys. Have at least one (1) successful experience in consulting for a public body or NGO. Demonstrated experience (5 years) in the production of quantitative and qualitative data collection and processing, sampling and collection methodology Knowledge in household and community groups surveys Wide experience in training and supervision of investigators Have experience in coordinating the collection, processing and analysis of both quantitative and qualitative disaggregated data by sex and by age. Have work experience in integrating GE with project appraisals. Have a good command of modern data collection and processing tools.
Application process
Interested and qualified evaluator shall send the following documents: 1. 2.
Letter of interest Outline of a technical proposal (maximum 10 pages) which demonstrates a comprehensive understanding of these ToRs and which includes the following components: i. Outline of a recommended methodology that will enable the evaluation team to answer the research questions posed in this ToR ii. Proposal for a data analysis plan and management of performance indicators iii. iv. v. vi. vii.
1
Proposed timeline for activities and suggested work plan (including a Gantt chart)1 Team composition and level of effort for each member of the proposed team. Previous demonstrated experience in mixed research methods and other elements identified in these ToRs Curriculum vitae of the evaluator and CV of members of the entire proposed team indicating relevant experiences Names and contact information of three references who can be contacted about relevant experiences
Before the initial report is drafted, the selected candidate will receive the proposed dates for data collection to be included in the initial report and work plan.
viii.
A copy of similar reports written previously for baseline, mid-term or final studies
ix.
Profile of the consulting firm (if applicable).
x.
Demonstrate previous experience in using the various methods and/or any other
3. Financial proposal with a detailed breakdown of the expenses related to the study: i. ii. iii. iv. v.
Detailed consultation fees Detailed budget for field data collection Detailed administrative expenditure Period of validity of quotations Deadlines and methods for payment
4. A fact sheet describing the areas of practice and the experience of the consulting firm (if applicable). Full applications should be emailed to: WUSC, Sandrine Tapsoba, Monitoring and Evaluation and Learning regional advisor, to the following email adress :
[email protected] Only electronic proposals will be accepted. The deadline for the submission of applications is November 4, 2019 at the close of business. Up to the 4th of November, interested parties are encouraged to send to the address above, questions for further clarification of these terms of reference. All questions and written responses from WUSC will be posted to all bidders who request to be included.
Annex 1 - Revised WAGES Logic Model (Final Version 01/18/19)
ULTIMATE OUTCOME
1000 Enhanced socio-economic benefits from extractive industries for communities, especially women and youth, living in West Africa
🡹
🡹 1200 Increased economic empowerment of women, youth, and communities in targeted extractive areas
🡹 1300 Increased informed dialogue on inclusive and sustainable local economic development and governance strategies that benefit local communities, especially women and youth
🡹 1210 Improved ability of businesses to increase sustainable local employment and revenue generation opportunities, especially for women and youth
🡹 1220 Improved confidence and skills of women and youth to leverage selfemployment, employment and entrepreneurship opportunities
🡹 1310 Increased opportunities for local, national and regional stakeholders to exchange information and build knowledge on gender and youth inclusive local governance and sustainable economic development
↑ 1211 Participatory multi-stakeholder economic subsectors/value chains and market opportunity assessment conducted and shared to support diversification and sustainability in mining areas
↑ 1221 Technical assistance provided to local training providers for delivering curricula responsive to market opportunities that build technical skills along with the confidence, advocacy and leadership skills of women and youth
↑ 1311 A knowledge sharing, advocacy and communication strategy is developed and implemented
🡹 1100 Strengthened sub-national governments and local institutions leverage opportunities that enable INTERMEDIATE communities, particularly women OUTCOMES and youth, to participate in and benefit from local development in extractive areas 🡹 🡹 🡹 1110 Increased 1120 Increased ability of key capacity and representatives of accountability the community, of local especially women government and youth, to institutions to engage and hold plan and IMMEDIATE accountable local manage OUTCOMES government in the inclusive and management of sustainable local sustainable and socio-economic inclusive development development and the respect of rights 🡹
OUTPUTS
↑ 1111 A crosscutting gender equality and youth inclusion strategy is developed and implemented 1112 Key community representatives, especially women and youth, supported to identify their priorities in development, local governance, and environmental sustainability 1113 Key community representatives, especially women & youth, informed
↑ 1121 Participatory multistakeholder dialogue platforms on sustainable and inclusive socioeconomic development facilitated 1122 Support provided to the development of local development plans responsive to the needs and interests of women and youth
1212 Capacity building and coaching provided to existing local/national businesses, associations and cooperatives to leverage market opportunities and build a conducive environment for
1222 Women and youth trained on market relevant technical skills as well as advocacy and leadership skills
🡹
1320 Raised participation and voices of CSOs that represent women and youth and of local governments in national debates on extractive issues
↑ 1321 Local & national multistakeholder platforms on the implementation of Mining Development Funds 1312 Sharing facilitated forums, 1322 Technical workshops, study assistance tours organized provided to at the national CSOs level on gender representing and youth women and inclusive local youth, and local governance and governments to sustainable network and economic participate in development national debates on extractive issues 1313 Regional events organized
and trained on policies, standards and best practices related to inclusive governance and accountability, including on revenue and expenditure management, human rights and environmental protection 1114 Women and youth groups supported to promote participation in local governance, GEYI, respect of rights, and environmental sustainability
1123 Technical assistance provided for the design of inclusive local economic development strategies and plans adapted to extractive areas 1124 Initiatives supported on transparent and participatory management of local development resources, including mining development funds, NGO and private sector development funds
1125 Local governments supported on the design and implementation of participatory capacity development plans, integrating GEYI
women and youth employment
1213 Linkages facilitated or strengthened between private sector buyers, including mining companies and local/national businesses, associations and cooperatives 1214 Local business development organizations supported 1215 Master crafts people trained on relevant market skills and the promotion of nontraditional employment for women and youth
1223 Selected Women and youth trained in entrepreneurship and business development including gender equality, and sustainable development 1224 Women and youth supported to start their businesses (associations, cooperatives and entrepreneurs), formalize them, access finance and network
for stakeholders to share experiences, best practices and lessons learned on inclusive governance and sustainable development 1314 Key information, best practices, and lessons learned documented and disseminated on gender and youth inclusive local governance, sustainable development, and GEYI 1315 Baseline study, midterm and final evaluations are conducted and disseminated
