The 2001 Declaration of Commitment on HIV/AIDS and the 2006 Political Declaration on HIV/AIDS adopted by the United Nations General Assembly are the guiding forces of the global response to AIDS. The process of preparing country progress reports should involve all partners in AIDS response and should provide an opportunity for reflection on the national response, its achievements as well as its shortcomings, in order to reach universal goals. Monitoring the response to the AIDS epidemic is essential to ensure that investments in AIDS achieve the expected results in health and society [¹].

HIV-related Non Governmental Organizations (AIDS-NGOs) work to promote prevention and to provide care and help to people affected by HIV in most countries of the world. The role of these AIDS-NGOs has been critical in the fight against HIV infection since the onset of the epidemic [²]. The activities of the AIDS-NGOs are important both around the world and in Catalonia (one of 17 Autonomous Communities in the North-East of Spain with 7,364,078 inhabitants in 2008) [³]. These activities are complementary and are often leaders of public and private health systems in various countries. AIDS-NGOs have lead the initiative against HIV and act as a bridge and "communications space" between the most vulnerable sectors of the population (because of socioeconomic inequalities) and health services [⁴]. AIDS-NGOs are the largest providers of preventive activities against HIV/AIDS, particularly amongst high-risk behavior groups: commercial sex workers, injecting drug users, men who have sex with men, youths in high-risk situations, prisoners and immigrants and persons living with HIV/AIDS [⁵,⁶]. Furthermore, they have wide experience of working at the community level, and their autonomous nature allows them to respond more quickly and employ innovative methods [⁷].

Since the inception of the AIDS National Program in Catalonia, the promotion and coordination of activities carried out by AIDS-NGOs has been emphasized. To support the AIDS-NGOs, since 1992 the Department of Health has provided annual subsidies that have progressively increased to reach 1.4 million Euros in 2007 [^8-10]. There has been an increase in the prevention, health promotion and care activities of AIDS-NGOs [⁶,¹¹].

The number of AIDS-NGOs and their activities has grown to meet the demand and the needs of individuals in preventive materials, determination of serostatus and HIV care. For this reason there is great diversity in the type of activities, as well as in the social levels and population groups that have been targeted.

In Spain, Peiró et al. systematically classified and evaluated the activities of national AIDS programs, including AIDS-NGOs activities using the Haddon matrix. This study concluded that there is much heterogeneity in the structure and indicators of the various programs. Concepts such as goals, objectives and priorities are poorly defined. Moreover, in all countries the implementation of preventive programs shows a predominance of individual interventions over social ones [¹²].

A more recent report, the Health Plan of the Community of Valencia 2005-2009 [¹³] includes a description of objectives and lines of action for HIV prevention based on the Haddon matrix axes [^14-16]. Other autonomous communities in Spain such as the Canary Islands and Asturias have also provided a basis for developing evaluation indicators for all activities [¹⁷,¹⁸]. In Spain, the "AIDS National Programme" of the Ministry of Health addresses the development of preventive programs for HIV and other sexually transmitted infections in individuals that work as commercial sex workers. The plan suggests both qualitative and quantitative evaluation indicators for these activities [⁹].

Moreover, the methodology that has been proposed to compare the different ways to prioritize issues and to forecast a policy's application in terms of results and outcomes is the Haddon Matrix. This matrix is a way to synthesizes the enormous amount of information available in strategy documents, shorten the time needed to produce an assessment, and improve a policy's value by comparing it with external models. In this paper we use this theoretical framework developed with the explicit intention of covering the need of the AIDS National Program of Catalonia to know what preventive and control interventions are done by AIDS-NGOs with the objective of improving the praxis of these activities and establishing new strategies for the AIDS National program [¹²]. Furthermore, a proposal of measurable indicators to monitor and assess each activity done by these organizations will be developed.

It is important to provide key constituents who are actively involved in a country's response to AIDS with essential information on core indicators that measure the effectiveness of the national response. The indicators will help to ensure the consistency and transparency of the process used by national governments. Increasingly, countries are strongly encouraged to integrate the core indicators into their ongoing monitoring and evaluation activities [¹⁹].

The key motivation for undertaking this work is to propose quality indicators which include users' satisfaction, psychosocial aspects that can have an impact on the fight against inequalities and stigmatization, and the perspectives of professional service providers. Until now, in many evaluation proposals, these aspects were not really addressed in the HIV literature, while their consideration would permit an important step toward the incorporation of strategies for evaluation of NGO prevention and more comprehensive control activities.

The purposes of this study are: to describe and analyze the objectives of and the HIV-AIDS preventive activities carried out by the AIDS-NGOs that work with AIDS in Catalonia and that receive subsidies from the Department of Health; and to develop a more comprehensive set of measurable and agreed upon core quality evaluation indicators to monitor and assess those objectives and activities that can have an impact on health and the fight against inequalities and stigmatization, and incorporate the perspectives of service providers and users.

A mixed method strategy was used in order to achieve the objectives of the study. The way to combine quantitative and qualitative research was by completeness, in order to achieve more complete and comprehensive answers to research questions. The strategy was divided into four phases and there was full data integration of each phase (each new phase includes the findings of previous phases) [^20-23] (Figure 1).

The present study followed a mixed method strategy that contributes to improving knowledge of activities undertaken by AIDS-NGOs and serves as a basis from which to develop a final proposal of indicators to assess these prevention-control based activities. As shown in the results, this methodological strategy implies that the different phases complement one another and progressively richer information is obtained [²⁰,³⁷].

The results allowed us to know all the preventive-control based activities done by AIDS-NGOs in Catalonia and how they could be assessed. The first two phases showed us a relationship between objectives and activities, and what other activities were not registered in the AIDS-NGOs annual reports. The qualitative phase showed how the preventive-control activities are done and what the main barriers and limitations to evaluate it are. Finally, the indicators consensus meetings enabled us to talk with AIDS-NGOs professionals and establish a consensus on the indicators proposed to evaluate the activities in the daily work flow.

The preventive activities of the AIDS-NGO are based on a participatory health education model adjusted to the people's needs, based on empowerment and focusing on the ideas of knowledge and skill. This education strategy follows the principles put forward by the WHO [³⁸].

Results emphasise the importance of offering ancillary services [³⁹] to people in need of HIV-AIDS prevention or treatment. Another study showed that the holistic person-based approach is essential to achieve a change in behaviour [⁴⁰]. On the other hand, the WHO Report 2008 [³⁸] underlines the impact of person-based care in health improvement, quality of life, user trust and treatment adherence.

In this relationship model, the professional takes into account the values and perspective of the user, and therefore incorporates them in the decision-making process [⁴¹]. The applicability of specific programmes requires a community approach to adjust them to match the needs of the target population [³⁹,⁴²].

The results confirm that the AIDS-NGOs perform most of their activities in relation to target groups at risk of social exclusion or because they are socially vulnerable. Moreover, it reduces social inequalities due to socioeconomic status, gender and social orientation. It also confirms the role of NGO-AIDS as a bridge acting as a "communications space" between health services and the population) and also with other services (legal, social, employment, etc.).

Evaluation is a continuous process that facilitates the identification of areas for improvement. It should also contribute to the recognition of tasks that have been satisfactorily carried out by professionals. The professionals belonging to the participating AIDS-NGOs of this study consider evaluation to be an activity which is relevant to them and share the evaluation needs. Therefore, there is concordance between the necessities detected by the HIV-AIDS program (Public Health Department) and those identified through a revision of international publications [¹,^43-45].

In order to evaluate the objectives and activities of AIDS-NGOs, it is essential to use good indicators bringing together characteristics such as: acceptability, feasibility, reliability, sensitivity to change, validity and meaningful and possible communicability [¹⁹,⁴⁶].

The latest update of the "Compendium of Evidence-Based HIV Prevention Interventions", elaborated by the CDC, as well as the "Community guide" website, both show a broad range of evidence-based preventive activities. However, these activities are included in programs aimed at very specific groups (female condom skills training; many men, many voices; personalized cognitive risk-reduction counseling), while our study presents a broader evaluation and suggests multiple indicators to evaluate concrete activities, which can be applicable to diverse intervention programs and to different groups. Thus, this proposal of indicators can help provide new evidence about the effectiveness of specific activities according to the level of prevention and intervention [⁴⁷].

Through the development of the first phase of this study, two more target groups of population were identified: inmates and immigrants. These two target populations had specific characteristics deserving of special consideration and their own preventive programs on the part of the AIDS-NGOs, and they were added as target populations for purposes of this study. Specific objectives and activities addressed to these groups were identified, and specific evaluation indicators proposed through the seven final Haddon matrices, available via the following link (http://www.gencat.cat/salut/depsalut/html/ca/dir2068/informefinal_actongsida2009.pdf) [³⁶].

Our proposal agrees with some of the quality measures of HIV care proposed by the CDC and UNAIDS [¹,⁴⁵,⁴⁷]. What is new and constitutes a strong point in our study is that we have added other quality indicators that measure user satisfaction (an essential aspect of the evaluation of service quality) and more psychosocial factors (legal advice) that can have an impact on the fight against inequalities and stigmatization [⁶,⁴⁸].

Another important aspect to note is that, by using the Haddon matrix, many preventive interventions and evaluation indicators have been identified and defined on the individual, external and social levels, while less have been directed towards the individual's immediate environment. These findings are consistent with those observed by Peiró et al [¹²]. A possible interpretation of this phenomenon is the concern of AIDS-NGOs professionals about data confidentiality; in order to act in the immediate environment of the affected person, he or she must agree to share the problem with individuals in his or her intimate environment.

The consensus phase among AIDS-NGOs professionals and the research team to develop the indicators has meant the cooperation and participation of the professionals in this study and constitutes a key element in facilitating the acceptance and implementation of the improvements proposed in the evaluation. It is also worth mentioning that professionals belonging to the AIDS-NGOs that participated in the consensus phase make up a multidisciplinary team much like the AIDS-NGOs. This gives the proposed indicators wide variability and richness, encompassing all of the activities undertaken by the AIDS-NGOs [³⁴,³⁵,⁴⁵].

The main limitation of this proposal of indicators is that it has not been applied. It would be important to share and disseminate it among professionals of the various programs and organizations that work with HIV in order to test its applicability, usefulness, validity and practical relevance.

A possible selection bias could also be argued, given that these indicators have been designed with the participation of professionals from AIDS-NGOs that work in Catalonia and are financed by this autonomous community's Department of Health. Nonetheless, as mentioned above, the enormous variability in the number of activities and proposed evaluation indicators minimizes this bias and raises their potential usefulness for AIDS-NGOs located in other geographical areas. Furthermore, one limitation of the study was not to have studied the views of other health providers that work with HIV-AIDS prevention or control-programmes (Primary Care, Public Health, Reproductive and Sexual Health, Faith-Based Organizations and Community Based Organizations). Although their contributions would have been of great interest, for reasons of practicality and insufficient resources this aspect could not be included.

We also want to mention that indicators for global results such as HIV mortality, rate of hospitalizations and approach to other health problems (diabetes, hypertension...) have not been proposed. However, the improvements in these epidemiological indicators can be attributed to multiple factors (improvements in security and effectiveness of drug treatments, healthcare, etc.) and not only to the AIDS-NGOs' activities. Nonetheless, studies that directly involve classic systems of epidemiological surveillance for HIV-AIDS would be of interest.

For the selection of indicators in future evaluations of preventive activities in AIDS-NGOs, the elements that make up the third dimension of the Haddon matrix should be taken into consideration: effectiveness, cost, freedom, equity, stigmatization, preferences of the affected community or individual and viability. These factors can be considered from different points of view and can have greater or lesser weight when selecting indicators, based on the general objectives of future health policies and on HIV-AIDS [¹²,¹⁵,¹⁶].

It is also important that the strategic plan of future HIV-AIDS policies emphasize activities that address the population that makes up the affected or at-risk person's immediate environment [¹²]. Although activities directed towards the social environment can have an indirect impact and their execution can be more difficult in some cases, it is necessary to emphasize the key role of the AIDS-NGOs in the development of these activities and the important social impact they can have.

This study showed the importance of methodological combination for a better understanding of results and the mechanisms of changes in the evaluation of AIDS-NGO activities. Although the study had some limitations, the combination of quantitative and qualitative data gave the opportunity to construct core evaluation activities and the corresponding indicators. The results could help to homogeneously assess the preventive-control activities carried out by several AIDS-NGOs. Moreover, the indicators could help the professionals to understand and therefore to improve the evaluation of the preventive-control AIDS-NGO activities. The Haddon matrix enables us to detect the gaps in intervention levels and, as a consequence, to keep in mind strategies that could enhance AIDS programs in the future. A further important message of the research was that qualitative approaches are essential in order to identify underlying processes and to detect activity procedures in order to better define the indicators.

The authors declare that they have no competing interests.

AB, EPR, CV, AR, RM, AG and JA have been involved in writing the manuscript, and all authors critically revised and approved the final manuscript. AB, EPR, CV, AR and JA designed the methodology and contributed to the description of the background. AB, EPR, CV, AR, CA and JA have done the fieldwork and the analysis and interpretation of data. All authors have participated sufficiently in the work to take public responsibility for appropriate portions of the content.

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6963/11/176/prepub

This study has been funded by the AIDS Programme, Public Health Department of the Ministry of Health, Government of Catalonia. We would like to thank all the AIDS-NGO professionals, and in particularly Kati Zaragoza, Anna Rafael, Ricardo Fuertes, Alba Mestre and Anna Lara, who took part in the consensus meetings. We specially acknowledge all users who have taken part in the qualitative study. We thank the IDIAP Jordi Gol for the funding of the study and translation of the manuscript into English.