GLOBAL FUND OBSERVER (GFO), an independent newsletter about the Global Fund provided by Aidspan to over 8,000 subscribers in 170 countries.
Issue 106: 19 August 2009. (For formatted web, Word and PDF versions of this and other issues, see www.aidspan.org/gfo)
+ + + + + + + + + + + + + + + + + + +
CONTENTS
+ + + + + + + + + + + + + + + + + + +
Editor’s Note: This issue of GFO is entirely devoted to reporting on the results of the Global Fund’s Five-Year Evaluation.
1. NEWS: Final Reports Released on Global Fund Five-Year Evaluation
The Global Fund recently released the results of the Health Impact Evaluation, the third and final area of study in the Five-Year Evaluation; and the Fund also released reports synthesising the overall evaluation findings.
2. NEWS: Health Impact Evaluation – Major Findings
The rapid increase in funding from all partners for the three diseases has resulted in a major expansion in access to services in these countries and has improved coverage of interventions, which will likely impact disease burden. However, gaps in the availability of data prevented the evaluation from actually measuring the impact of the Global Fund and its partners on the three diseases. These are the major findings from the Health Impact Evaluation.
3. NEWS: Highlights from the Synthesis Reports of the Five-Year Evaluation
The Global Fund has laid the foundation for continued, successful scale-up. However, not all of the original expectations of the Global Fund have been realised. Concerted effort will be required to continue the revision and refinement of the Global Fund’s principles, systems and practices in order to increase funding for scaling up, especially in light of the current financial environment.
These are the main conclusions of the full Five-Year Evaluation.
+ + + + + + + + + + + + + + + + + + +
1. NEWS: Final Reports Released on Global Fund Five-Year Evaluation
+ + + + + + + + + + + + + + + + + + +
After more than two years of work, the Global Fund Five-Year Evaluation has been completed. The Global Fund recently released several documents (a) providing the results of the third and final area of study in the evaluation; and (b) synthesising the overall evaluation findings.
Launched in November 2006, when the Global Fund was approaching its fifth anniversary, the evaluation was organised around three study areas:
- Study Area 1 – Organisational efficiency and effectiveness of the Global Fund
- Study Area 2 – Effectiveness of the Global Fund partner environment
- Study Area 3 – Impact on HIV, tuberculosis and malaria
The evaluation was planned and implemented by two international teams of evaluators led by Macro International Inc., a research and management consultant firm. The evaluation was overseen by the Technical Evaluation Research Group (TERG), an advisory body providing independent assessment and advice to the Global Fund Board on issues related to monitoring and evaluation.
The two reports on Study Area 3 recently released by the Global Fund were:
"Final Report – Global Fund Five-Year Evaluation: Study Area 3 – The Impact of Collective Efforts on the Reduction of the Disease Burden of AIDS, Tuberculosis and Malaria," May 2009
"Technical Evaluation Reference Group Summary Paper: Study Area 3 – Health Impact of Scaling Up Against HIV, Tuberculosis and Malaria: Evaluation of the Current Situation and Trends in 18 Countries," April 2009
This first document is the report from the evaluators, Macro International Inc. The second document is a report from the TERG. It provides a summary of the Macro International report, plus the TERG’s assessment of that report.
The two reports synthesising the findings of all three study areas, recently released by the Global Fund, were:
"The Five-Year Evaluation of the Global Fund to Fight AIDS, Tuberculosis and Malaria – Syntheses of Study Areas 1, 2 and 3," March 2009
"Technical Evaluation Reference Group Summary Paper: Synthesis Report of the Five-Year Evaluation of the Global Fund," April 2009
The first document is the report from the evaluators, Macro International Inc. The second document is a report from the TERG that provides a summary of the Macro International report, discusses the limitations of the evaluation that was conducted, identifies areas that the TERG thinks need further investigation, and presents a list of "key issues and priorities" that the TERG has identified from the evaluation findings.
All four documents are available on the Global Fund website at www.theglobalfund.org/en/terg/evaluations/5year.
In the two articles that follow, we report on the findings from Study Area 3, and on the highlights of the synthesis reports.
Editor’s Note: GFO recently carried an article on the results of the Study Area 2 evaluation. See Issue 101, available at www.aidspan.org/gfo.
+ + + + + + + + + + + + + + + + + + +
2. NEWS: Health Impact Evaluation – Major Findings
+ + + + + + + + + + + + + + + + + + +
Editor’s Note: This article summarises the major findings of Study Area 3 of the Global Fund Five-Year Evaluation – Impact on HIV, tuberculosis and malaria, also referred to as the "Health Impact Evaluation." It is based on the report entitled "Technical Evaluation Reference Group Summary Paper: Study Area 3 – Health Impact of Scaling Up Against HIV, Tuberculosis and Malaria: Evaluation of the Current Situation and Trends in 18 Countries" (hereinafter Summary Paper: Study Area 3), which summarises the Study Area 3 report of the international evaluators (see previous article), and provides an assessment of that report by the TERG.
The rapid increase in funding from all partners for the three diseases has resulted in a major expansion in access to services in these countries and has improved coverage of interventions, which will likely impact disease burden. However, gaps in the availability of data prevented the evaluation from actually measuring the impact of the Global Fund and its partners on the three diseases.
These are the main conclusions from the Health Impact Evaluation, the third and last study area in the Global Fund Five-Year Evaluation.
The objective of the Health Impact Evaluation was to comprehensively assess, in selected countries, the collective impact that the Global Fund and other international and national partners have achieved on reducing the disease burden of AIDS, TB and malaria.
The study was conducted in 18 countries – 10 where the evaluation was based primarily on existing information (Benin, Burundi, Democratic Republic of Congo, Ghana, Kyrgyz Republic, Lesotho, Moldova, Mozambique, Rwanda and Viet Nam) and eight where additional data was collected specifically for the evaluation (Burkina Faso, Cambodia, Ethiopia, Haiti, Malawi, Peru, Tanzania and Zambia).
The study methodology was designed to document the trends in the three diseases, including mortality and morbidity. As part of the methodology, country-level Impact Evaluation Task Forces were established in each participating country. These Task Forces had broad representation from local institutions, including Ministries of Health, civil society, CCMs and donors. The Task Forces reviewed the country evaluation work plans and budgets, as well as draft and final country reports.
In total, $11.7 million was spent on the Health Impact Evaluation, 40 percent of which was for data collection.
Key findings
The following is a summary of the key findings of the study, as reported by the evaluators. These findings relate to the 18 countries included in the study; they do not necessarily reflect the situation in all countries that receive Global Fund grants.
Impact on HIV/AIDS
- HIV funding has increased rapidly. Eighteen percent of the additional money came from the Global Fund.
- There has been a major expansion in access to services in all countries. However, there are gaps in basic requirements – such as trained personnel, guidelines, medicines, and equipment – that negatively impact the quality of services.
- There have been dramatic increases in estimated coverage of antiretroviral (ARV) treatment.
- There have been reductions in HIV high-risk behaviours among men in the general population in most of the countries.
- There is little evidence of large-scale changes in behaviours among the most at-risk populations.
Impact on TB
- Positive trends in treatment success rates have continued in most countries, but there is little evidence of accelerated progress since 2003 (when scaling up began in earnest).
- There is widespread access to TB services, although there are no major increases since scaling up.
- There is considerable scope for improving the quality of diagnostic and treatment services.
- TB notification rates are stable or declining in several countries, but the required supporting data on diagnostic intensity is often lacking.
Impact on malaria
- There have been major increases in funding, led by the Global Fund.
- Malaria diagnostic capacity remains suboptimal.
- The availability of artemisinin-based combination therapy (ACT) in most countries is limited.
- Major progress has been made in coverage with insecticide-treated bed nets and intermittent preventive treatment of malaria during pregnancy.
Impact on health systems strengthening (HSS)
- Health worker density is low in all districts, especially in rural areas.
- In many facilities, there are serious deficiencies in terms of basic amenities, especially improved water supply and essential equipment. The situation is somewhat better in facilities that offer HIV services.
- There is inadequate availability of many essential medicines, especially for chronic adult diseases but also for childhood illnesses.
- There are major gaps in the availability of diagnostics.
The evaluators found that although there was evidence of increased data collection for the three diseases, major data gaps and weak health information systems remain. These gaps and weaknesses seriously limit the ability to evaluate progress.
The consultant team commented that because investment in evaluation by the Global Fund and its partners over the years has been limited, it was only possible to partially document trends in the three diseases – and, therefore, only possible to partially assess the impact of the Global Fund and its partners.
Comments by the TERG
In it assessment of the study carried out by the evaluators, the TERG said:
- The Health Impact Evaluation was exceptionally challenging from both a methodological and practical point of view.
- The design of the study, as submitted to the Global Fund Board in 2006, recognised that certain questions likely could not be fully answered.
- The study identified major gaps in availability of data and greater health information system weaknesses than originally predicted.
- There were inherent tensions between the desire for inclusive participation and country capacity building, on the one hand, and the requirement for high-quality evaluation products that had to be delivered within a short time frame, on the other.
- A recurrent issue in the evaluation is the absence of solid and consistent baseline data upon which to base conclusions regarding the effects of scaling up.
The TERG said that to address the information gaps, additional studies are needed in the following areas:
- civil society and community-based interventions
- non-health-based HIV prevention interventions
- interventions targeting high-risk groups
- differences in performance between countries
- effectiveness of specific interventions
- HIV/TB co-infection interventions
In conclusion, the TERG cited the following comment from one of the persons who reviewed a draft of the evaluation report:
"In many respects, this evaluation process shares many of the characteristics of the Global Fund itself. It was conceived with the right principles and approach in mind, along with engaging the best technical people and giving them at least reasonable financial resources to initiate an innovative process. The technical team developed a thoughtful and, in most respects, state-of-the-art approach towards tackling the problem. However, this evaluation faced significant challenges once it entered the real world of extremely weak country institutions, multiple stakeholders with poor in-country coordination, and very poor routine information systems."
Nevertheless, the TERG said, the Health Impact Evaluation was worthwhile because it produced useful data in 18 countries, and it has helped to strengthen the foundation for future impact assessments.
+ + + + + + + + + + + + + + + + + + +
3. NEWS: Highlights from the Synthesis Reports of the Five-Year Evaluation
+ + + + + + + + + + + + + + + + + + +
Editor’s Note: This article summarises the main points made in the synthesis summary report on the Global Fund Five-Year Evaluation prepared by the Technical Evaluation Research Group (TERG). The full title of the report is "Technical Evaluation Reference Group Summary Paper: Syntheses Report of the Five-Year Evaluation of the Global Fund." The report summarises, and comments on, the synthesis report prepared by the international evaluators (see first article above).
The Global Fund has laid the foundation for continued, successful scale-up. However, not all of the original expectations of the Global Fund have been realised. Concerted effort will be required to continue the revision and refinement of the Global Fund’s principles, systems and practices in order to increase funding for scaling up, especially in light of the current financial environment.
These are the main conclusions of the TERG with respect to the Global Fund Five-Year Evaluation.
A large portion of the evaluation focussed on 24 countries: Benin, Burkina Faso, Burundi, Cambodia, Democratic Republic of Congo, Ethiopia, Ghana, Haiti, Honduras, Kenya, Kyrgyz Republic, Lesotho, Malawi, Moldova, Mozambique, Nepal, Nigeria, Peru, Rwanda, Tanzania, Uganda, Viet Nam, Yemen and Zambia.
Key findings
The following is a list of the overarching findings of the Global Fund Five-Year Evaluation, as reported by the evaluators. The findings are relevant to the 24 participating countries; they do not necessarily apply to all countries that receive Global Fund grants.
Finding 1: The Global Fund, together with major partners, has mobilized impressive resources to support the fight against AIDS, TB and malaria.
Finding 2: Collective efforts have resulted in increases in service availability and better coverage, which will ultimately reduce disease burden.
Finding 3: Health systems in most developing countries will need to be greatly strengthened if current levels of services are to be significantly expanded.
Finding 4: The Global Fund has modelled equity in its guiding principles and organisational structure. However, much more needs to be done to reflect those efforts in grant performance.
Finding 5: The performance-based funding system has contributed to a focus on results. However, it continues to face considerable limitations at the country and Secretariat levels.
Finding 6: The Global Fund partnership model has opened spaces for the participation of a broad range of stakeholders. This progress notwithstanding, existing partnerships are largely based on goodwill and shared impact-level objectives rather than negotiated commitments or clearly articulated roles and responsibilities, and do not yet comprise a well-functioning system for the delivery of global public goods.
Finding 7: As the core partnership mechanism at the country level, CCMs have been successful in mobilising partners for the submission of proposals. However, in the countries studied, their grant oversight, monitoring, and technical assistance mobilisation roles remain unclear and substantially unexecuted. The CCMs’ future role in these areas and in promoting country ownership is in need of review.
Finding 8: The lack of a robust risk management strategy during its first five years of operation has lessened the Global Fund’s organisational efficiencies and weakened certain conditions for the effectiveness of its investment model. The recent work to develop a comprehensive, corporate-wide risk management strategy is a necessary step for the Global Fund’s future.
Finding 9: The governance processes of the Global Fund have developed slowly and less strategically than required to guide its intended partnership model.
With respect to the partnership model (findings 6 and 9), the evaluators said that at the level of the governance of the Global Fund, there has been unprecedented and largely successful participation of civil society, the private sector and other international development organisations in the Global Fund model. However, it added, despite some notable exceptions, little of this has yet translated into clearly defined, durable and formalised operational partnerships.
The evaluators said that the Global Fund’s approach during its first five years more accurately reflects a "friendship model" than a genuine "partnership model." Finally, the evaluators said that, in operational terms, the Global Fund has become a largely stand-alone entity whose staff growth trajectory appears to be a consequence of the unwillingness of partners – or the unwillingness of the Global Fund – to seriously pursue the stated partnership objectives.
In discussing other findings, the evaluators made the following additional observations:
- The current reliance of countries on external support raises significant concerns with respect to the long-term sustainability of programs; the risk of external funding replacing domestic investments; and the effect of the large-scale infusion of international resources on the cost-effectiveness and maintenance of programs.
- There is inadequate monitoring of gender, sexual minorities, urban-rural, wealth, education, and other types of equity as part of grant performance and impact assessment.
- Performance-based financing, a key tenet of the Fund’s guiding principles, has evolved into a complex and burdensome system that has thus far focused more on project inputs and outputs than on development outcomes. There remain inadequate information system and monitoring and evaluation capacities in countries, critically limiting the feasibility of the performance-based funding approach.
- CCMs are still largely perceived as Global Fund entities rather than as mechanisms for promoting country ownership.
- The governance process has yet to reconcile competing principles established in the Framework Document, establish the corporate strategy that the Global Fund so badly needs, systematically monitor its own performance or decision-making with respect to the initial guiding principles, or interact sufficiently with other governing boards to shape the global partnership environment.
- The Global Fund has increasingly become a stand-alone entity with a growing and increasingly complex portfolio of grants requiring ever-increasing numbers of staff at the global level to maintain effective financial oversight in countries.
In conclusion, the evaluators said that while the Global Fund has achieved an incredibly rapid start-up in its first five years, the Board’s focus has drifted from long-term and strategic issues toward ad hoc and incremental decision-making and operational details. The team said that in the absence of a considered longer-term strategy, the Global Fund Board and Secretariat have not sufficiently differentiated between areas of responsibility that can and cannot be delegated to the Secretariat or partnered with collaborating institutions. As a consequence, the team added, the Global Fund’s rapid organisational development, though impressive, has progressed at times in an unintended direction.
Comments by the TERG
The TERG noted that the Global Fund has several initiatives underway that address the findings of the evaluators. These initiatives include the following:
- an increasing involvement of the Global Fund in health systems strengthening (Finding 3);
- the development of an implementation plan for the gender strategy (Finding 4);
- the development of a revised grant rating system (Finding 5);
- the development of a partnership strategy (Finding 6); and
- the development of a risk-management framework (Finding 9).
The TERG said that the evaluation found that Global Fund principles continue to be fundamentally sound and relevant; that the governance structure and processes have achieved both broad participation and genuine power-sharing between key constituencies, including donors, developing and developed countries, the private sector, civil society organisations and people living with the diseases; that core partnership mechanisms at the global and country levels have been established; and that there is continued support for the performance-based funding system and no fundamental question as to the Global Fund’s role as primarily a financing mechanism.
On the other hand, the TERG observed that the evaluation identified important weaknesses, limitations and potential conflicts in the implementation of the Global Fund principles in practice. The TERG made the following additional observations:
- The performance-based system is complex and burdensome.
- The private sector is marginally involved.
- National health information systems do not provide sufficient regular and reliable information.
- Effective mechanisms for grant oversight and technical assistance are not in place at country level.
- The experience with CCMs is mixed.
- The fully demand- and country-driven model for applications does not guarantee that the most cost-effective interventions for those most in need will be selected.
Recommendations
With respect to the role of the Global Fund vis-à-vis its partners, the TERG recommended that the Global Fund maintain a small and efficient Secretariat, and that it seek to achieve maximum impact through partnerships. The TERG said that although this is not necessarily the most efficient approach, in the long term this is the most sustainable approach. The TERG said that in order to maintain a lean Secretariat despite the growing size of its portfolio, grant oversight by the Secretariat
should move away from direct program management through input/output indicators and more toward higher-level indicators and quality management principles, supplemented by periodic audits.
With respect to health systems, the TERG recommended that the Global Fund and its partners collectively address issues regarding health systems strengthening, and that they focus on key factors limiting scale-up: human resources, monitoring and evaluation systems and availability of essential services.
With respect to the performance-based funding system, the TERG recommended that the Global Fund assess the entire system – from proposal submission processes to grant rating methodologies,
including target negotiation, data quality assessment and disbursement.
In addition, the TERG recommended that the Global Fund:
- take steps to strengthen country ownership in proposal development;
- take a more proactive approach to grant-making, focusing on the most cost-effective interventions with the highest potential for impact;
- develop a systematic and focused strategy for improving country-level monitoring and evaluation systems in order to collect more reliable data in support of performance-based funding;
- encourage countries to increase investments in medium- to long-term capacity building;
- encourage countries to develop better quality assurance mechanisms that can help to improve grant oversight;
- encourage governments in countries (in particular those with high disease burden) to develop long-term financial mobilisation plans in line with the national disease control plans for guiding domestic investment and use of Global Fund grants;
- improve its communication with global and country partners in order to avoid misunderstandings and inconsistencies;
- adopt a more differentiated approach to grant management, based on the size of the grant, previous results, the state of the health system, contextual information, and other country-specific data;
- set clear incentives for countries to ensure availability of key data at the grant application stage and throughout the grant implementation process.
With respect to what happens next, at its meeting in May 2009 the Global Fund Board asked the Secretariat to develop a process for responding to and implementing the recommendations, and to report back to the Board at its meeting scheduled for November 2009. The Board also decided that its Chair would set up a small ad-hoc committee to assist the Board to (a) follow-up on, and formulate the Board’s responses to, the recommendations; and (b) further define the role of the TERG in relation to independent evaluations, the resources required and Board oversight of the process.
"Reproduced from the Global Fund Observer Newsletter (www.aidspan.org/gfo), a service of Aidspan."
Forwarded by:
---------------------------
Yours in Global Concern,
A.SANKAR
Executive Director- EMPOWER
107J / 133E, Millerpuram
TUTICORIN-628 008, TN, INDIA
Telefax: 91 461 2310151; Mobile: 91 94431 48599
· You are invited to join an e FORUM AIDS-TN. To join this free e Forum kindly send an e mail to AIDS-TN-subscribe@yahoogroups.com
· You are invited to join an e FORUM CIN - Confederation of Indian NGO’s. To join this free e Forum kindly send an e mail to ConfederationofIndianNGOs-CIN-subscribe@yahoogroups.co.in
· This e Forums is moderated by EMPOWER, a Non-profit, Non-Political, Voluntary and Professional Civil Society Organisation.
