This mid-term evaluation report (2014) shows the progress made within the Bridging the Gaps programme until 2014. Based on document analysis, case studies, focus groups and interviews with management, staff and beneficiaries, this evaluation identifies priorities of action for the remainder of the programme.
The Institute of Development Studies (United Kingdom) conducted an evaluation of the Bridging the Gaps programme (‘the Programme’) since the start (September 2011) until 2014. This mid-term evaluation report shows the progress made within the Programme, currently the largest programme in the world focusing on key populations. The total four-and-half-year budget is 46.7 million euros, 35 million euros coming from the Dutch Ministry of Foreign Affairs and 25% being ‘co-funding’ and ‘own contribution’.
Three key populations – Four projects – Sixteen countries
The Programme links three key populations through combining four projects into one comprehensive approach:
- Female, male, and transgender sex work project
- People using drugs project (with specific focus on injecting drug users)
- Lesbian, gay, bisexual and transgender people, including men who have sex with men project
- Global advocacy
The Programme, running from September 2011 till December 2015, operates in 16 countries worldwide across the main key populations at risk for HIV. It is implemented by an Alliance, currently consisting of Aidsfonds (the Lead), four Alliance Partners (AFEW, COC, Mainline, and GNP+), and four Global Partners (MSMGF, INPWUD, ITPC, and NSWP). Each of these partners has its own local (national) counterparts for service delivery and local advocacy. A number of working groups and mechanisms for coordination, learning and sharing have been set up.
This participatory mid-term evaluation (December 2013–April 2014) was conducted to inform the Programme Team and the Board on possible improvements for the remainder of the Programme period, and beyond. A mix of participatory and results oriented methods was used. Five key questions, as outlined below, guided the evaluation.
Question I: To what extent has the Bridging the Gaps programme been effective?
The Programme is well on its way to make significant contributions to its five key objectives.
Objective 1: Improve the quality of and the access to HIV prevention, treatment, care and support
Extremely relevant services are being delivered to key populations at a large number of locations in difficult settings. Exclusive attribution to the Programme funding – assign the credit of these services to this particular programme ‐ is difficult for several reasons. Many services are co-funded, which means that the Programme funds a small part within a broader programme. Secondly, Programme partners were partially selected based on existing experience and partnerships, which means that current results build upon earlier efforts. And thirdly, no baseline data are available. However, the Programme significantly contributes to the success of the services it supports.
- Maintain current levels of support for service delivery.
- Develop packages of minimum services and benefits for individual support group members with a ’graduation’ strategy and development perspectives. Packages need to be developed by the groups. Implementing Partners and national counterparts can facilitate upon request.
Objective 2: Improve the human rights of key populations
Human rights work is being done at various levels – from local to global and from health policy development to decriminalisation under the law – with varying degrees of success. A long-term view and legal expertise are needed.
- Link with other human rights organisations and experts on specific issues for collective engagement at different levels (global, regional and national). Be aware of legal requirements of evidence and risks for staff.
- At national levels continue engagement with lawyer(s), magistrates, police specialised in the relevant areas of national laws through which international human rights principles are implemented. If possible, link with groups that also engage in these areas to create broader movements/links.
- Address effectiveness of global advocacy through strategic planning and adjustment of the logical framework.
Objective 3: Integrate specific services for key populations within the general health system
Integration into the general health system is an ambiguous objective. Some successes are being noted by local partners. There are vast differences within the Programme about the perceived efficacy of integration into the health system. Decentralisation is pertinent.
- Conduct strategic discussions to define key terms which form the framework in this objective and formulate a context-specific course of action.
Objective 4: Strengthen the capacity of civil society organisations that work on HIV and key populations
In terms of organisational strengthening there is huge variety in activities and quality. With a view to sustainability, more focus on learning organisations, institutional memory, continuous learning, and so on, is required. The Global and Alliance Partners are appreciated for their support. Capacity building strategies need more structure and graduation strategies.
- Establish and implement common adult learning principles throughout the Programme, that recognise and value learning based on experience. Experiential learning theory is one option that could be taken into consideration as a framework.
- Conduct capacity and needs assessments (with budget allocation as needed) when developing partnerships.
- Broaden focus to organisational and systemic strengthening (vs individual capacity building) and use internal lessons learnt.
Objective 5: Develop and strengthen a comprehensive and concerted approach on HIV and key populations by the Alliance Partners
- Develop common principles, yet decentralised implementation and an operationalisation of advocacy with exit and benchmarks.
Recommendations with regard to accountability and the M&E Framework:
- Speedily develop a plan for final evaluation (also consider for Phase II).
- Use the current logical framework to monitor progress at output level.
Question II. To what extent have its activities been relevant and politically, socially and financially sustainable?
The Programme was and still is relevant at a global level. Work with key populations tends to be underfunded and warrants special attention. Key populations have some similar interests, but they are different groups with different needs. These differences affect programming.
The funds allocated to the partners give value for money. Given the funding gaps for key populations, the political marginalisation of key populations and the general under-spending in health in many countries, the allocation of sufficient funds by national governments to take over the programmes is unrealistic. Upscaling by large donors and some support of national governments is feasible. The global political climate on key populations is volatile, but there are many ways to engage with the law and legislators. National partners are taking leadership. Linkage with human rights organisations might be explored.
- Retain emphasis on key populations and flexible programming, review the ’Dutch Approach’.
- For the next phase, revise the key objectives to align them with the work being done in a way that they build on the strengths of the Programme.
- Do not increase the number of countries and partners.
- Endeavour to extend the Programme by five years building on current strengths.
Question III. What lessons can be learned about the management of the Programme?
The Lead and the Alliance Partners are appreciated as partners especially for their flexible and reasonable way of working. The complexity and size, along with the great variety among the 78 partner organisations, prescribe a decentralised management approach. Generally, the Aidsfonds Lead strikes the right balance between leadership, Alliance and Global Partners’ partnership and local partners’ ownership. The Programme’s structure is satisfactory, yet some challenges are noted.
- Operationalise the role of Global Partners in the Programme and build consensus, first among the Global Partners, and then with the Alliance Partners.
- Review current Programme working groups to ensure clear TORs and membership.
- Develop a simplified M&E system with a few simple indicators and additional qualitative and participatory methods for the next phase.
- Develop and establish sex-disaggregated data collection throughout the Programme.
- Clarify representation and upward and downward responsibility and accountability between global, national and local partners.
- Ensure that a general overview on expenditures (overhead, Global Partners, Alliance Partners, local partners) is available, when needed.
- Improve internal communications in general, including the conduct of meetings, and recognise importance of direct interpersonal human contact.
Question IV. How has the Programme contributed to intended and unintended learning and linking within key populations and across geographical areas, key populations and approaches?
Linking between global and national Programme partners is taking place, yet can be strengthened. Centrally collected indicators through the logical framework do not contribute meaningfully to learning. The M&E systems of some local partners are of high quality and documented through learning exercises. Local partners are eager to learn and connect with each other, but more facilitation of structured opportunities would be welcome.
- Facilitate more structured opportunities for linking and learning.
- Identify and promote the establishment of learning organisations/centres of excellence and institutional memories, and document learning.
Many different organisations collaborate within Bridging the Gaps in different roles:
- Alliance Partners: Aidsfonds (the Lead agency), AFEW, COC, GNP+, Mainline. As a requirement of the Ministry applicants for grant funding that made the Programme possible needed to be based in The Netherlands.
- Global Partners: GNP+, MSM Global Forum, NSWP, INPWUD and ITPC, five constituency-led global networks which – within the Programme – primarily focus on advocacy at the global level.
- Local or national partners: The organisations working in any of the sixteen countries where the Programme is active.
To avoid confusion in the future (due to the fact that GNP+ is an Alliance Partner with a global advocacy mandate and not responsible for implementation of the Programme at the local level), a new term was suggested by the Board during a discussion on this report:
- Implementing Partners: Agencies who are responsible for the implementation of the work in the Programme countries, principally through funded partnerships with national and local organizations: Aidsfonds, AFEW, COC, Mainline, MSMGF, i.e. the Implementing Partners and the Global Partners are mutually exclusive.
In all official documents and throughout the evaluation the term ‘Alliance Partner’ had been used by all involved. The evaluation team therefore recommends that the advantages and disadvantages of a new terminology be discussed with all involved before changing it. This can be part of the recommended meeting with the Global Partners to operationalise and plan the Programme’s advocacy strategies.
Question V. If there were Phase II of the Programme, which major changes would be recommended?
There are no evident burning issues related to the approach, the scope, structure, and implementation of the Programme. Possible changes would distract from strengthening the current work. Hence, no major changes are suggested for the remaining programme period.
- Develop the possible Programme Proposal for Phase II in collaboration with all prospective partners.
- Explore the option to NOT conduct a Final Evaluation of Phase I:
- Data collection and processing set in motion in the context of this evaluation (for example the two ‘backgrounders’) is valuable and should be built on.
- The Mid-Term Evaluation is quite late in the Programme. A Final Evaluation would not impact on the decision on possible extension.
- In case there is a two-year Phase II, it would possibly be more expedient to now lay the groundwork for a high-quality Final Evaluation of the ‘total programme’ (Phase I and II together) towards the end of 2017.
Key words: evaluation, Bridging the Gaps programme