Assessment and improvement of the Italian healthcare system: first evidence from a pilot national performance evaluation system.
Medical care (Italy)
Health care reform (Laws, regulations and rules)
Health care reform (Analysis)
|Publication:||Name: Journal of Healthcare Management Publisher: American College of Healthcare Executives Audience: Trade Format: Magazine/Journal Subject: Business; Health care industry Copyright: COPYRIGHT 2012 American College of Healthcare Executives ISSN: 1096-9012|
|Issue:||Date: May-June, 2012 Source Volume: 57 Source Issue: 3|
|Topic:||Event Code: 970 Government domestic functions; 930 Government regulation; 940 Government regulation (cont); 980 Legal issues & crime Advertising Code: 94 Legal/Government Regulation Canadian Subject Form: Medical care (Private); Medical care (Private) Computer Subject: Government regulation|
|Product:||Product Code: 8000001 Medical & Health Services; 9105210 Health Care Services NAICS Code: 62 Health Care and Social Assistance; 92312 Administration of Public Health Programs|
|Geographic:||Geographic Scope: Italy Geographic Name: Italy Geographic Code: 4EUIT Italy|
The Italian National Health System (NHS), established in 1978, follows a model similar to the Beveridge model developed by the British NHS (Beveridge 1942; Musgrove 2000). Like the British NHS, healthcare coverage for the Italian population is provided and financed by the government through taxes. Universal coverage provides uniform healthcare access to citizens and is the characteristic usually considered the added value of a welfare system financed by tax revenues.
Nonetheless, in Italy the strong policy of decentralization, which has been taking place since the early 1990s, has gradually shifted powers from the state to the 21 Italian regions. Consequently, the state now retains limited supervisory control and continues to have overall responsibility for the NHS in order to ensure uniform and essential levels of health services across the country. In this context, it has become essential, both for the ministry and for regions, to adopt a common performance evaluation system (PES).
This article reports the definition, implementation, and first evidences of a pilot PES at a national level. It shows how this PES can be viewed as a strategic tool supporting the Ministry of Health (MOH) in ensuring uniform levels of care for the population and assisting regional managers to evaluate performance in benchmarking. Finally, lessons for other health systems, based on the Italian experience, are provided.
A number of factors, including rising costs, technological advancements, population aging, and medical errors, have contributed to the adoption of performance measurement systems by many industrialized countries (Smith 2002; Arah et al. 2006; Kelley and Hurst 2006). The introduction of New Public Management (NPM) principles in the 1980s promoted a number of reforms in order to drive a more efficient, effective, and accountable public sector (Hood 1995a; Lapsley 1999; Saltman and Busse 2002; Saltman and Vrangbaeck 2007). OECD countries have applied these principles in different ways with different emphasis (Hood 1995b). Among the NPM principles, the one asking the public sector to adopt more explicit and measurable standards of performance measurement has motivated countries to create different performance measurement systems.
However, in the 1980s the use of performance measurement systems primarily focused on financial measurements and was therefore unable to help organizations achieve multiple strategic objectives or drive change (Pollitt 1985; Ghobadian and Ashworth 1994; Guthrie and English 1997; Lorden, Coustasse, and Singh 2008). Consequently, the adoption of a sophisticated and comprehensive multidimensional performance evaluation system, such as the balanced scorecard, which looks beyond traditional financial measures, has been suggested (Linard et al. 2000; Eccles 1991; Jackson 1993; Kloot and Martin 2000; Fottler, Erickson, and Rivers 2006; Yang and Tung 2006). In particular, within the healthcare sector, a growing number of studies describe the adoption of a multidimensional performance evaluation system by a broad range of healthcare organizations, including single providers and entire national health systems (e.g., Linard et al. 2000; Aidemark 2001; Zelman, Pink, and Matthias 2003; WHO 2003; Asbroek et al. 2004; Chang 2007; Ba-Abaad, 2009, McLoughlin et al. 2001).
Within the Italian healthcare system, the need for performance measurement has grown in urgency since the early 1990s when the government approved the first reform of the National Health Service (Legislative Decrees 502/1992 and 517/1993) (Lo Scalzo et al. 2009). This was a period when national reforms started transferring several important administrative and organizational responsibilities from the central government to the 21 regions, with the aim of making regions more sensitive to the need to control expenditure and promote efficiency, quality, and citizen satisfaction. This devolution process was recently enforced by the act regulating fiscal federalism (law 42/2009) delivered by the Italian parliament in 2009, which provided regions with significant autonomy in organizing healthcare services, allocating financial resources to their local health authorities (LHAs), and in monitoring and in assessing performance (Formez 2007; Censis 2008; Nuti 2008; Antonini and Pin 2009). The central government retains overall responsibility for ensuring that services, care, and assistance are equitably distributed to citizens across the country. However, this strong policy of decentralization, along with a series of rationalization measures, has contributed to accentuating the already existing interregional disparities in healthcare, especially the north-south divide, and have undermined the egalitarian principles of the National Health Service.
Italian regions differ for historical and geographical reasons. The sharpest division is between north and south. The north has one of the most advanced industrial societies in the world, whereas the south, which encompasses the area of seven regions (Campania, Molise, Basilicata, Calabria, Puglia, and the two islands, Sicily and Sardinia) is, by contrast, one of the most economically depressed areas in Europe.
In this context, at regional level, only a few Italian regions have adopted systems able to measure performance (Nuti 2008; Vittadini 2010; Provincia Autonoma di Trento 2011; Barone et al. 2006-2008). At a national level, the Italian NHS has had to face the challenge of creating and developing efficient systems capable of preserving both the principles of egalitarianism and high-quality services. In recent years a pilot multidimensional national PES has been developed. This program aims to reduce discrepancies and focus on the intrinsic goals of the NHS by monitoring the capacity of each region to guarantee its citizens, regardless of their social status, equal access to essential health services while maintaining quality, efficiency, and appropriateness.
Given this background, the purpose of this article is to describe and illustrate the development and design of the first Italian health performance measurement system, which is intended to assess the dimensions of performance at both the national and regional levels.
We next present the construction and development of the pilot conceptual framework, followed by a description of the PES reporting system. Finally, we report policy implications and conclusions.
THE ITALIAN PERFORMANCE EVALUATION SYSTEM
As a result of the ongoing process of devolution of power, the Italian NHS is currently organized on the basis of two levels: the central government, which has planning and funding responsibilities, while ensuring that all citizens have uniform access to healthcare, and the 21 regional governments, which organize and supervise the provision of healthcare services within their jurisdiction and allocate overall financial resources to the productive units--approximately, in 2009, 146 local health authorities and 100 independent hospitals across the country. (1) Each LHA, under the supervision of the corresponding regional government, is directly responsible for the provision of comprehensive care to its entire resident population, regardless of income or occupational status (Ferrario and Zanardi 2010).
In 2009, the Ministry of Health decided to take up the challenge in assessing the performance of the health services provided by the regions. The pilot national performance evaluation system was designed and implemented by the research team of the Laboratorio Management e Sanita (MeS Lab) in accordance with the National Agency for Regional Health Care Services (Agenas), which is in charge of its further development. (2)
This choice was based on the fact that the MeS Lab has had extensive experience in evaluating the performance of healthcare services. Since
2005, it has designed and implemented, in Tuscany, a multidimensional performance evaluation system to monitor the performance of all Tuscan Health Authorities (Nuti 2008; Nuti 2010; Nuti et al. 2010; Nuti et al. 2009; Nuti et al. 2011). Moreover, since 2008 other Italian regions (Liguria, Piedmont, Umbria, Aosta Valley, Marche, Basilicata, and Trento and Bolzano autonomous provinces) decided to adopt the same system, so that today almost half of Italian regions use the same framework in order to assess the performance of their healthcare services.
In the first phase of the development of a pilot national PES, the research team started from the Tuscan experience and carefully tailored the regional PES to meet the national health system's needs and strategies. The set of measures to be chosen should, in fact, reflect a robust picture of the healthcare performance that can be reliably reported across regions using comparable data. It should be appropriate to support the national health planning process by informing decision makers of how resources and health services are provided to a population and by highlighting inequalities within regional health systems.
After having discussed the strategic goals of the national health system with national policymakers and Agenas, the selected indicators and dimensions were derived and sent to all regions. On the basis of their feedback comments, the indicators and dimensions were further modified.
Selection of Domains of Indicators
As already stated, the main NHS goal is to ensure that the delivery of healthcare should be equally provided to the population across the 21 regions and across all the levels of care that constitute the health system. As a consequence, the national performance indicator framework should therefore include performance indicators for benchmarking within the following three domains (settings) of care: hospitals, primary care including pharmaceutical care, and public and preventative health.
Indicators should then be chosen and developed to provide information about the performance of the regional systems across the three domains in terms of the following:
* Quality of the services delivered to citizens, according to the specific goal of each level of care, in order to ensure that patients receive safe, prompt, and correctly delivered services.
* Equity, which deals with potential performance differences across and within regions. Differences across providers for the same indicator should result from epidemiological issues and not management or professional capabilities.
* Appropriateness: Each patient should receive nothing more but also nothing less than what is required.
* Efficiency, which means achieving desired results with the most cost-effective use of resources (Donabedian 2003).
Selection of Performance Indicators
In the second phase of the framework development, the team selected indicators reflecting the importance of the dimensions outlined above. Starting from the Tuscan PES, the initial comprehensive list was reduced to a more manageable group of performance indicators. Many performance indicators, although of relevance regarding the analysis, remain not calculable because of the reliance on only a few data sources, while some other indicators were not considered in order to avoid information overload. This limited set of measures should be able to give the Ministry of Health (MOH) a rapid and comprehensive overview of the NHS and its differences across regional healthcare systems. This is important given the Moll's role of supervisory control and its overall responsibility for the NHS in guaranteeing uniform and essential levels of health services across the country.
Each indicator was tested to determine its suitability for measuring NHS performance by means of iterative discussions, first with the MoH and then with regional representatives. A transparent, consensus-based process is critical, because there are frequent decisions involving trade-offs between accuracy and reliability, data availability and comparability across the different regional healthcare systems. In particular, the effort to identify the core set of performance indicators consisted of the following steps:
1. Starting from international evidence and the already developed and tested measures of the Tuscan PES, the research team identified and then calculated 38 performance measures at regional and provider levels.
2. The research team shared the list and results of the performance measures with the MoH.
3. The MoH officially showed to the 21 regional health councillors the above-mentioned list and results.
4. The research team was then asked by the MoH to collect and analyze all the comments coming from the regions.
Finally, after an intensive period of interaction between the research team and the regional representatives, a final set of 34 indicators out of the initially presented 38 was chosen (Exhibit 1). In particular, the following four indicators were excluded from the initial list:
1. Hospitalization rate for gastroenteritis (2-17 years old)
2. Hospitalization rate for pneumonia
3. Per capita cost for a defined daily dose (DDD) of a drug
4. Health services per inhabitants by homogeneous groups of clinical diagnosis that are provided within the confidence intervals of the national average
These indicators were not included either because they couldn't be applied uniformly to the different regional healthcare systems (the first two indicators) or because they did not receive the consensus by all the regional representatives.
Most of the 34 indicators were derived either from the framework already developed by Tuscany region or from international experience (OECD 2003; WHO 2003, AHRQ 2006; CIHI 2001; Department of Health 2008) and were selected on the basis of the following criteria (Kelley and Hurst 2006):
* The importance of what is being measured in terms of policy relevance
* The scientific soundness of the measure in terms of its validity, reliability, and the explicitness of the evidence base
* The feasibility and cost of obtaining nationally comparable data for the measure
The Italian hospital discharges database for the years 2007 and 2008 was used for all the measures belonging to the hospital and primary care dimensions, the 2008 OsMed report (Gruppo di lavoro OsMed 2009) for the indicators regarding pharmaceutical care, the 2008 national screening report, and the MoH database for the measures related to public health and prevention. Avoidable hospitalization rates for chronic conditions from inpatient data were used as a proxy of primary
care performance because of the lack of national comparable sources on territorial services (Ricketts et al. 2001). Moreover, indicators from hospital inpatient data, where possible, have been standardized according to sex and age using the Italian residents of the year 2001 as a standard population. Finally, all the indicators based on hospitalization data were calculated at both regional and interregional levels (all the providers (3) within each region), while pharmaceutical and public health and prevention care data were only available at aggregated (regional) level.
Starting from the final indicator list, 23 indicators out of the total 34 were chosen to be evaluation measures and were assigned performance assessment ratings in order to allow for regional benchmarking. This means that the 21 regions were divided into quintile groups on the basis of the distribution of each evaluation measure in order to derive five different levels for defining the indicator performance of each region categorically from worst to best. This approach allows the MoH to benchmark the Italian regions in terms of their performance results across the selected dimensions of care. The other 11 indicators were considered useful to provide further insights into each regional performance level and were therefore defined as observational indicators.
THE PES REPORTING SYSTEM
The data collected from the PES were translated into a color-coded reporting tool that indicates to users at a glance underachievement and exceptions of each region on the basis of only the 23 evaluation measures.
The target chart, which was developed within the Tuscan healthcare system in 2005 (Nuti 2008; Nuti et al. 2009), was then chosen as a reporting tool due to its user-friendly interface features. This specific performance graph was designed to provide a visual representation of performance results across all the indicators, thus enabling managers to quickly ascertain whether the regional health system is performing up to standard.
This chart is divided into five bands on the basis of the five performance levels, each with its own color, from dark green--corresponding to excellent performance--to red--poor performance. Within each regional target, the closer the evaluation indicator is positioned to the center of the target, the higher its performance level.
Exhibit 2 shows, as examples, the targets of three Italian regions, one from the north of Italy (Veneto region), one from the center (Umbria region), and the last one from the south (Campania region). (4) White circles on the target depict performance measures for each region on a particular indicator, with those on the dark and light bands indicating respectively worse and better performance. Target analysis confirms, at a glance, the existence of a clear division between the north and south of Italy: Best performances are all concentrated in northern-center regions while critical situations are in the south. The figures make clear that indicators in the targets of both Umbria and Veneto are all close to the center, indicating a good global performance, while, on the contrary, most of the southern regions' indicators are concentrated on the target boundaries, where the worst evaluation assessments are positioned.
The MoH received the target diagrams and tables and graphs reporting the values of all 34 indicators measured where it was possible, at the regional and provider level. As an example, Exhibit 3 shows the 2008 case-mix adjusted length of stay (LOS) indicator for all 21 Italian regions. This indicator is given as the sum of all differences between the average regional LOS of each diagnosis-related group (DRG) and the average national LOS for the same DRG (Servizio Sanitario dell'Emilia Romagna 2009). This indicator is a measure of efficiency that enables evaluation of the potential reduction in hospital days if each region had an average LOS equal to the benchmark value (here the whole nation). The indicator was restricted only to surgical DRGs because the LOS of medical DRGs (and thus efficiency) can be affected by inappropriate hospitalization rate. (This phenomenon is noticeable in the southern regions, which have a very high hospitalization rate for medical DRGs.)
Results of the indicator show that in 2008, the region with the worst performance is Lazio, with LOS longer than the national average by 1.1 days, while Emilia-Romagna has the shortest LOS of all regions (indicator equal to -0.7), followed by Tuscany, where LOS is shorter than the national value by about 0.6 days.
As already mentioned, the distribution of inpatient indicators is also studied at a provider level, thus also allowing the MoH to monitor discrepancies in the provision of the set of essential healthcare services within each region. As an example, Exhibit 4 shows the 2008 case-mix adjusted length of stay indicator for all the Tuscan healthcare providers.
Descriptive results of LOS at aggregate and provider levels highlight the presence of intra- and inter-regional differences in the distribution of the indicator values. This suggests that regions with values of the case-mix LOS index above the national average and with high internal variability have room to improve their efficiency in hospital care and thereby release funds to be invested in other healthcare services. For instance, Tuscany has calculated that if, in 2009, all its own health units with a LOS (per DRG) higher than the regional mean (per DRG) had reduced its value to the regional mean, the region would have saved around 59 million euros, which potentially could have been reallocated to other services (Nuti, Vainieri, and Bonini 2010).
The PES, together with its reporting system, represents a powerful tool for both regional and national levels. The PES is used at a national level to monitor performance in order to guarantee essential healthcare services to the whole population and at a regional level to benchmark health authorities' performance and thus to learn from the best practices (Johnston 2004; France 2008).
Moreover, the target chart as a reporting tool can support managers as it enables decision makers to
1. build up a more comprehensive picture of the strengths and weaknesses of each regional healthcare system through the ability to integrate a wide range of relevant information,
2. identify problematic areas and thus allow the targeting of policy and practical interventions more effectively,
3. rapidly benchmark performance across the various regional healthcare systems, and
4. make performance reports more user friendly for nonexpert users.
Another important feature of the national PES is that it may be considered a valid tool of communication between national and regional levels. In fact, other European experiences with the introduction of multidimensional performance systems, such as the balanced scorecard, show that multidimensional systems may represent valid tools of communication among actors (Aidemark 2001).
Given the importance of informing health system stakeholders and the public about the performance of regional health systems and their local organizations, the MoH decided to make regional targets and tables publicly accessible through the ministry website (www.salute.gov.it). The MoH has used this public disclosure to shake the regional systems and encourage them to undertake improvements. It is the first time, in Italy, that the MoH has displayed the performance evaluation of regional health services on its website using the most updated performance data. The public exposure of regional performance (and their local healthcare organizations) and the user-friendly reading of the performance evaluation through the target charts should powerfully enhance the accountability process between regions and citizens. Indeed, the PES may help citizens evaluate their local policymakers.
In addition, the PES can contribute to the current debate on how to define "the standard cost" introduced by the recent Italian law 42/2009, which enforced the fiscal federalism process. The standard cost has a central role in fiscal decentralization, as each region, in order to satisfy the residents' estimated health needs, must count on its own internal revenues and eventually on a national solidarity fund. Each region, however, may draw on this fund only if it delivers health services in line with national standards.
The PES can provide regions with national standards in terms of quality, volumes, and appropriateness on the basis of the average of regional best performers for selected indicators. Thus the standard costs may be calculated on the basis of the performance of the best practice regions (Nuti et al. 2011).
The performance evaluation system has become a public policy tool that helps the national government evaluate its strategic policy and promote a "managed" competition among regions. The information dealt with and uniformly represented can enable an efficient and constructive benchmarking process among Italian regions and health authorities. The proposed PES seems to reach a fair equilibrium between the regional governments' need to control the local health institutions and the local institutions' need to control their own performance (Greener 2003). Regions must be encouraged to measure their performance at a local level, creating an appropriate culture of evaluation and learning, focusing attention not only on cost control, but also on quality and appropriateness. The national government can support this process by coordinating a benchmarking system, both at regional and upper-regional levels. This will allow local administrations to learn from other experiences, overcome self-referencing, and improve their performance enforced by reputational pressure.
Moreover, the data accountability and transparency the PES affords help public bodies garner loyalty from citizens.
Ultimately, the PES represents a real contribution to measuring standard costs on the basis of the outputs that each region is able to provide to citizens, not on how much the region spends per habitant. This seems to be a fair approach to the deployment of fiscal federalism in Italy.
However, the national PES still has some drawbacks. They can be grouped into two kinds: data availability and assessment method. The readily available and quality national data sources allow the analysis to be focused only on selected dimensions of performance. For instance, neither individual experience nor direct primary care services can be used in the evaluation. In addition, the issue of privacy did not allow recording of linkages among different types of service data that could be useful for analyzing patient pathways.
Regarding assessment method, the quintile technique obliges classification of regions into the five assessment bands. This is particularly true for indicators in which performance is quite similar. This limitation can be overcome when central and regional governments share quantitative standards.
However, what we have presented is a pilot study and can be considered as a starting point for an ongoing process that aims to refine and improve the selected indicators and domains.
Finally, it's important to consider that the possibility of working within a Beveridge system should reduce the emphasis that policymakers place on the dynamics of the health market and reimbursement procedures and could enable them to pay more attention to the quality of treatment, well-being of citizens, and equity, which are the true keystones for keeping expenditure under control. However, the variability in the results presented here and obtained by the federalist-based Italian healthcare system is enough to question the efficacy of the Beveridge system in reducing inequalities. This geographic variability also exists in the United States (Wennberg and Gittlesohn 1973; Wennberg 2004). The conclusion is therefore that the choice of a Beveridge-style healthcare system is important but not in itself sufficient to guarantee equity of access for citizens. In reality, what really counts in both Italy and the United States is the capacity to manage variability by means of performance monitoring and assessment, which help political decision makers and health professionals to accurately focus their actions in order to improve equity and reduce unnecessary variability. In both countries it is still essential and desirable to have a system of performance measurement using federal benchmarking criteria. Whichever health system model is adopted, by means of public disclosure of results (Mannion and Goddard 2003; Fung et al. 2008; Hibbard, Stockard, and Martin 2003) and empowerment of the patient's role performance measurement ensures that variability is reduced and service quality is improved.
This study was supported by the Italian Ministry of Health.
The authors wish to thank the Ministry of Health, Ferruccio Fazio, and top managers of the Department of Health and Agenas, in particular Filippo Palumbo and Lucia Lispi, for their insightful suggestions.
The authors also wish to thank the researchers from Laboratorio Management e Sanita for their help in the development of the pilot performance evaluation system, in particularly Domenico Cerasuolo, who designed the software, Anna Bonini for her comments, and Maria Sole Bramanti for her help during data elaboration.
Agency for Healthcare Research and Quality (AHRQ). 2006. Guide to Inpatient Quality Indicators, version 3.0. Published February 20. www.qualityindicators.ahrq.gov/ Downloads/Software/SAS/V31/iqi_guide _v30.pdf.
Aidemark, L. G. 2001. "The Meaning of Balanced Scorecards in the Healthcare Organization." Financial Accountability & Management 17 (1): 23-40.
Antonini, L. and A. Pin. 2009. "The Italian Road to Fiscal Federalism." Italian Journal of Public Law 1 (16).
Arab, O. A., G. P. Westert, I. Hurst, and N. S. Klazinga. 2006. "A Conceptual Framework for the OECD Health Care Quality Indicators Project." International Journal for Quality in Health Care 18 (suppl. 1): 5-13.
Asbroek, A. H., O. A. Arah, J. Geelhoed, T. Custers, D. M. Delnoij, and N. S. Klazinga. 2004. "Developing a National Performance Indicator Framework for the Dutch Health System." International Journal for Quality in Health Care 16 (Suppl. 1): 65-71.
Ba-Abaad, K. M. 2009. "Review of the Literature of Balanced Scorecard and Performance Measurement: The Case of Healthcare Organizations." Business e-Bulletin 1 (1): 33-47.
Barone, A. P., G. Di Cesare, M. D'Ovidio, A. Lallo, D. Fusco, C. Sorge, M. Stafoggia, and C. A. Perucci. 2006-2008. Regional Program for Assessing the Outcomes of Health-care Interventions (Programma Regionale di Valutazione degli Esiti degli Interventi Sanitari). http://184.108.40.206/ vislazio/vis_index.php.
Beveridge, W. 1942. Social Insurance and Allied Services. London, HMSO.
Canadian Institute for Health Information (CIHI). 2001. Hospital Report 2001: Acute Care. www.hospitalreport.ca/downloads/2001/acute_2001.html.
Censis. 2008. I Modelli Decisionali Nella Sanita Locale. Roma: Censis.
Chang, L. 2007. "The NHS Performance Assessment Framework as a Balanced Scorecard Approach: Limitations and Implications." International Journal of Public Sector Management 20 (2): 101-17.
Department of Health. 2008. The NHS in England: The Operating Framework for 2009/10. London: Doll, 2008: 13. http://www.dh .gov.uk/en/Publicationsandstatistics/ Publications/PublicationsPolicyAnd Guidance/DH_091445.
Donabedian, A. 2003. An Introduction to Quality Assurance in Health Care. Oxford: Oxford University Press.
Eccles, R. G. 1991. "The Performance Measurement Manifesto." Harvard Business Review January-February, 131-37.
Ferrario, C., and A. Zanardi. 2010. "Fiscal Decentralization in the Italian NHS: What Happens to Interregional Redistribution?" Health Policy 100 (1): 71-80.
Formez. 2007. I Sistemi Di Governance Dei Servizi Sanitari Regionali. Rome: Formez. http://portale.formez.it/content/i-sistemi-di -governance-dei-servizi-sanitari-regionali.
Fottler, M. D., E. Erickson, and P. A. Rivers. 2006. "Bringing Human Resources to the Table: Utilization of an HR Balanced Scorecard at Mayo Clinic." Health Care Management Review 31 (1): 64-72.
France, G. 2008. "The Form and Context of Federalism: Meanings for Health Care Financing." Journal of Health Politics, Policy and Law 33 (4): 649-705.
Fung, C. H., Y. W. Lim, S. Mattke, C. Damberg, and R G. Shekelle. 2008. "Systematic Review: The Evidence That Publishing Patient Care Performance Data Improves Quality of Care." Ann Intern Med 148 (2): 111-23.
Ghobadian, A., and I. Ashworth. 1994. "Performance Measurement in Local Government--Concept and Practice." International Journal of Operations and Production Management 14: 35-51.
Greener, J. 2003. "Performance in the National Health Service: The Insistence of Measurement and Confusion of Content." Public Performance and Management Review 26: 237-50.
Gruppo di lavoro OsMed. 2009. L'uso dei farmaci in Italia. Rapporto nazionale anno 2008. Roma: Il pensiero scientifico Editore.
Guthrie, J., and L. English. 1997. "Performance Information and Programme Evaluation in the Australian Public Sector." International Journal of Public Sector Management 10 (3): 154-64
Hibbard, J. H., J. Stockard, and T. Martin. 2003. "Does Publicizing Hospital Performance Stimulate Quality Improvement Efforts?" Health Affairs 22 (2): 84-94.
Hood, C. 1995a. "Contemporary Public Management: A New Global Paradigm?" Public Policy and Administration 10 (2): 104-17.
Hood, C. 1995b. "The New Public Management in the 1980s: Variations on a Theme." Accounting Organizations and Society 20 (2/3): 93-109.
Jackson, P. M. 1993. "Public Service Performance Evaluation: A Strategic Perspective." Public Money and Management (13): 9-14.
Johnston, D. ]. 2004. "Increasing Value for Money in Health Systems." European Journal of Health Economics 5: 91-94.
Kelley, E., and J. Hurst. 2006. "Health Care Quality Indicators Project: Conceptual Framework Paper." OECD Health Working Papers 23.
Kloot, L., and J. Martin. 2000. "Strategic Performance Management: A Balanced Approach to Performance Management Issues in Local Government." Management Accounting Research (11): 231-51.
Lapsley, I. 1999. "Accounting and the New Public Management: Instruments of Substantive Efficiency or a Rationalising Modernity?" Financial Accountability and Management 15 (3/4): 201-7.
Linard, K., M. Basset, L. Dvorsky, and J. Yoon. 2000. "A Dynamic Balanced Scorecard Template for Public Sector Agencies." 19th International Conference of the System Dynamics Society.
Lorden, A., A. Coustasse, and K. P. Singh. 2008. "The Balanced Scorecard Framework--A Case Study of Patient and Employee Satisfaction: What Happens When It Does Not Work as Planned?" Health Care Manage Rev 33 (2): 145-55.
Lo Scalzo, A., A. Donatini, L. Orzella, A. Cicchetti, S. Profili, and A. Maresso. 2009. "Italy: Health System Review." Health Systems in Transition 11 (6): 1-216.
Mannion, R., and M. Goddard. 2003. "Public Disclosure of Comparative Clinical Performance Data: Lessons from the Scottish Experience." J Eval Clin Pract 9: 277-86.
McLoughlin, V., S. Leatherman, M. Fletcher, and J. Wyn Owen. 2001. "Improving Performance Using Indicators. Recent Experiences in the United States, the United Kingdom, and Australia." International Journal for Quality in Health Care 13 (6): 455-62.
Musgrove, P. 2000. "Health Insurance: The Influence of the Beveridge Report." Bulletin of the World Health Organization 78 (6): 845-6.
Nuti, S. 2010. Report 2010-Il sistema di valutazione della performance della sanita Toscana. Pisa: Edizioni ETS.
Nuti, S. 2008. La valutazione della performance in sanita. Bologna: Il Mulino.
Nuti, S., A. Bonini, A. M. Murante, and M. Vainieri. 2009. "Performance Assessment in the Maternity Pathway in Tuscany Region." Health Services Management Research 22: 115-21.
Nuti, S., C. Daraio, C. Speroni, and M. Vainieri. 2011. "Relationships Between Technical Efficiency and the Quality and Costs of Health Care in Italy." International Journal for Quality in Health Care Advance Access, March 31.
Nuti, S., and M. Vainieri. 2011. Federalismo fiscale e riqualificazione del servizio sanitario nazionale. Bologna: I1 Mulino.
Nuti, S., M. Vainieri, and A. Bonini. 2010. "Disinvestment for Reallocation: A Process to Identify Priority in Healthcare." Health Policy 95: 137-43.
Nuti, S., M. Vainieri, C. Seghieri, and A. Bonini. 2010. "Linking Measurement to Performance Management in Public Health Care Systems." JMG workshop 10-11 September, Pisa.
OECD. 2003. Health at a Glance 2003: OECD Indicators, OECD Publishing. doi: 10.1787/health_glance-2003-en.
Pollitt, C. 1985. "Measuring Performance: A New System for the National Health Service." Policy and Politics 13 (1): 1-15.
Provincia Autonoma di Trento. 2011. I1 sistema di valutazione della performance della sanita nella provincia autonoma di Trento, Infosalute 16, Provincia Autonoma di Trento. www.trentinosalute.net.
Ricketts, T. C., R. Randolph, H. A. Howard, D. Pathman, and T. Carey. 2001. "Hospitalization Rates as Indicators of Access to Primary Care." Health Place 7 (1): 27-38.
Saltman, R. B., and R. Busse. 2002. "Balancing Regulation and Entrepreneurialism in Europe's Health Sector: Theory and Practice." In Regulating Entrepreneurial Behaviour in European Health Care Systems, edited by R. B. Saltman, R. Busse, and E. Mossialos, 3-52. Buckingham: Mc Graw Hill Open University Press.
Saltman, R. B. and K. Vrangbaeck. 2007. "Drawing Lessons for Policy-Making." In Decentralization in Health Care, edited by Il. B. Saltman, V. Bankauskaite, and K. Vrangbaek, 77-84. Buckingham: Mc Graw Hill Open University Press.
Servizio Sanitario dell'Emilia Romagna, Sistema informativo sanita e politiche sociali. 2009. www.regione.emilia-romagna.it/sas/ nosologiche/gloss.htm.
Smith, P. C. 2002. Measuring Up: Improving Health Systems Performance in OECD Countries. Ottawa: OECD.
Vittadini, G. 2010. La valutazione della qualita nel sistema sanitario, analisi dell'efficacia ospedaliera in Lombardia. Edizioni Guerini e Associati. Milan: CRISP.
Wennberg, I., and A. Gittelsohn. 1973. "Small Area Variations in Health Care Delivery." Science 182 (4117): 1102-8.
Wennberg, J. E. 2004. "Perspective: Practice Variations and Health Care Reform: Connecting the Dots." Health Affairs.
WHO. 2003. Health Systems Performance Assessment: Debates, Methods and Empiricism. Geneva: World Health Organization.
Yang, M., and Y. Tung. 2006. "Using Path Analysis to Examine Causal Relationships Among Balanced Scorecard Performance Indicators for General Hospitals: The Case of a Public Hospital System in Taiwan." Health Care Management Review 31 (4): 280-2.
Zelman, W. N., G. H. Pink, and C. B. Matthias. 2003. "Use of the Balanced Scorecard in Health Care." Journal of Health Care Finance 29 (4): 1-16.
(1.) These values derive from the national health providers' register, 2009.
(2.) The National Agency for Regional Healthcare (Agenzia nazionale per i servizi sanitari regionali, www.agenas .it) carries out its activities in close collaboration with the Ministry of Health and with the regions and participates in research programs funded by the MoH.
(3.) The study considers both public providers and those private providers which are accredited by the regional healthcare system (about 99% of all the Italian healthcare providers). Providers that are not accredited (1%) are not included in the analysis.
(4.) All the regional performance results and targets are available (in Italian) on the following website: www.salute.gov .it/dettaglio/phPrimoPianoNew.jsp?id =273&area=ministero&colore=2.
Robert G. Kiely, LFACHE, president/CEO, Anadolu Medical Center in affiliation with Johns Hopkins Medicine, Istanbul, Turkey
As a retired CEO of a well-regarded New England community hospital who is now the CEO of a tertiary care hospital in Istanbul, I found the authors' assertion that the pilot Italian performance evaluation system (PES) will assist government regulators and health professionals to focus attention on improving quality to be valid.
It appears that to date the focus in Italy has been on utilization and financial regulation, as was the case in the United States during the 1990s and is still the case in Turkey. In Turkey, the Ministry of Health uses a highly proscriptive approach to regulation that lacks meaningful measures of clinical quality and patient safety performance. In fact, there is no publicly available health-sector-wide measurement and evaluation system in Turkey.
The Italian pilot PES makes, in my opinion, a credible start at benchmarking while avoiding the benchmarking overload that is extant in the United States. lt is my opinion that highly motivated and well-managed LIS hospitals strive to serve too many masters in terms of measurement and evaluation. Benchmark overload often occurs in US hospitals as attempts are made to comply with CMS, The Joint Commission, AHRQ, Leapfrog, Magnet, and Top 100 databases. As reimbursement tightens and margins shrink, it will be interesting to see if hospitals and health systems can continue to support the resources necessary to evaluate processes and outcomes for the purpose of benchmarking and continuous improvement.
It appears that the Italian government made the wise choice of working closely with the regional health authorities in selecting indicators that reflect a solid picture of healthcare performance, including accredited private and public hospitals, and that can be reliably reported using comparable data. The authors paint a picture of a
Ministry of Health that has wisely decided to walk before it runs in terms of the measurement of performance from the health sector. As the authors point out, the PES should enable policymakers to pay more attention to quality as a tool for keeping expenditures under control. The United States appears to finally be moving in this direction, and I think that the Italian experience could be instructive for policymakers in Turkey as well, particularly if applied to both public and private providers of healthcare services.
Sabina Nuti, associate professor, Laboratorio Management e Sanita, Istituto di Management, Scuola Superiore Sant'Anna, Pisa; Chiara Seghieri, PhD, assistant professor, Laboratorio Management e Sanita, Istituto di Management, Scuola Superiore Sant'Anna, Pisa; Milena Vainieri, PhD, assistant professor, Laboratorio Management e Sanita, Istituto di Management, Scuola Superiore Sant'Anna, Pisa; and Silvia Zett, PhD student, Laboratorio Management e Sanita, Istituto di Management, Scuola Superiore Sant'Anna, Pisa
For more information on the concepts in this article, please contact Dr. Seghieri at firstname.lastname@example.org.
EXHIBIT 1 The First Set of Indicators of the National PES Indicator Indicator Label Evaluation Code Indicators HOSPITAL CARE (H) H1.1 Ordinary hospitalization rate--acute X admissions H1 Global hospitalization rate--acute admissions H1.2 Day-hospital hospitalization rate--acute admissions H1.3 Mean DRG weight--acute admissions Efficiency H2 Case-mix adjusted length of stay--surgical DRG X H2.1 Case-mix adjusted length of stay H2.2 Case-mix adjusted length of stay--medical DRG H13 Pre-op LOS--planned admissions X Surgical Appropriateness H3 Percentage of medical DRG from surgical X departments H4 Percentage of laparoscopic cholecystectomies X in day surgery or 0-1 day admissions H5 Surgical essential levels of health services X DRG--Standard percentage achieved Medical Appropriateness H6 Medical essential levels of health services X DRG--hospitalization rate H14 Percentage of short medical hospitalizations X T9 Percentage of medical day-hospital admissions X with diagnostic aim Clinical Quality H9 Percentage of cesarean births X H10 Percentage of readmissions within 30 days for X the same MDC corrected by the hosp. rate H10.1 Percentage of readmissions within 30 days for the same MDC H10.1.1 Percentage of readmissions within 30 days for the same MDC-Medical DRG H10.1.2 Percentage of readmissions within 30 days for the same MDC--Surgical DRG H11 Percentage of femur fractures operated within X 2 days H12 Regional outflow X PRIMARY CARE (T) Effectiveness of Chronic Disease Management T2 Hospitalization rate for heart failure X (50-74 years old) T3 Hospitalization rate for diabetes X (20-74 years old) T4 Hospitalization rate for COPD X (50-74 years old) Pharmaceutical Prescription Efficiency AF5 Per capita gross pharmaceutical expenditure X AF5.1 Gap between the distrectual use of pharmaceuticals and national median AF5.2 Percentage of the distrectual expenditure for equivalent pharmaceutical on the total net expenditure AF5.3 Percentage of the distrectual expenditure for equivalent pharmaceutical on the total DDD PREVENTIVE CARE (P) P1 Flu vaccine coverage rate X P2 MPR vaccine coverage rate X P3 Mammography screening extension X P4 Compliance with mammography screening X P5 Colorectal screening extension X P6 Compliance with colorectal screening X EXHIBIT 3 Case-Mix Adjusted Length of Stay Indicator at Regional Level, Year 2008 Italian Regions days Emilia Romagna -0.70 Tuscany -0.64 Marche -0.34 Umbria -0.33 Sicily -0.33 Lombardy -0.24 Trento -0.19 Abruzzo -0.18 Piedmont 0.01 Campania 0.00 Apulia 0.02 Veneto 0.17 Calabria 0.19 P.A. Bolzano 0.21 Friuli Venezia Giulia 0.38 Liguria 0.42 Basilicata 0.50 Molise 0.56 Sardinia 0.61 Aosta Valley 0.87 Lazio 1.10 Note: Table made from bar graph. EXHIBIT 4 Case-Mix Adjusted Length of Stay Indicator by Tuscan Health Providers, Year 2008 Tuscan Healthcare Providers days Az. Ospedaliera Meyer -2,40 Siena -1,79 Versilia -1,58 Arezzo -1,55 Massa Carrara -1,31 Fondazione Cnr-Rt G. Monasterio -1,22 Firenze -1,09 Az. Ospedaliero-Univ. Pisana -0,84 Prato -0,47 Grosseto -0,37 Lucca -0,27 Pisa -0,11 Az. Ospedaliero-Univ. Careggi -0,09 Livorno -0,05 Spedali Riuniti -0,04 Pistoia 0,08 Empoli 0,70 Note: Table made from bar graph.
|Gale Copyright:||Copyright 2012 Gale, Cengage Learning. All rights reserved.|