Multi criteria assessment of county public health capability disparities.
Objective: This paper proposes an analytical approach to the
assessment of geographic population health disparities that are measured
as the consolidation of public health related indicators into
geographic-specific scores, and are representative of the level of
public health capability within counties in the state of Mississippi.
Study Design: A multi-criteria decision model was employed to develop an
additive scoring system that assigns a numerical score of public health
capability disparities for a geographical area (county).
Methods: Routinely collected indicators were used to measure each county's current public health related concerns. These indicators include access, risks, health care quality, and outcomes data. Public health experts rated and ranked indicators to generate indicator weight.
Results: A county score was developed to rank Mississippi counties based on relative public health capability. This scoring system depicts population health disparities among Mississippi counties.
Conclusions: The model is useful and ideal for establishing expectations and benchmarks for reduction or equalization of disparities. This information can be used to manage geographic population health disparities by guiding policy formulation and implementation.
Medical care (Quality management)
Medical care (Demographic aspects)
Medical care (Research)
Zuniga, Miguel A.
Zuniga, Genny Carrillo
Seol, Yoon Ho
Fos, Peter J.
|Publication:||Name: Journal of Health and Human Services Administration Publisher: Southern Public Administration Education Foundation, Inc. Audience: Academic Format: Magazine/Journal Subject: Government; Health Copyright: COPYRIGHT 2009 Southern Public Administration Education Foundation, Inc. ISSN: 1079-3739|
|Issue:||Date: Winter, 2009 Source Volume: 32 Source Issue: 3|
|Topic:||Event Code: 310 Science & research|
|Product:||Product Code: 8000120 Public Health Care; 9005200 Health Programs-Total Govt; 9105200 Health Programs NAICS Code: 62 Health Care and Social Assistance; 923 Administration of Human Resource Programs; 92312 Administration of Public Health Programs|
|Geographic:||Geographic Scope: United States Geographic Code: 1USA United States|
Capacity evaluation and planning of public health resources is increasingly more important for state and local health departments and policy makers (Jia & Borawski, 2004). At present, there are few models that reflect both the public health system and, those aspects beyond its control that influence health status.
Decision analytic techniques are popular for use in a wide range of decision problems (Chatburn & Primiano, 2001). This paper presents an analytical approach to assessing relative population health disparities between counties in the state of Mississippi. The level of public health services within each county was identified through the consolidation of public health related indicators into geographic specific scores. This method allows counties to be ranked according to their relative capability scores as a representation of direct and indirect public health capability. This is important due to the fact that counties do not have the same health profiles, measured as available resources, disease epidemiology, public attitudes, etc. It is assumed that the differential in disparities across counties can be identified and its variation highlighted. This method may serve as a complement to planning, allocating, and implementing primary health care resources in an efficient and cost-effective manner.
The public health delivery system varies by locale and is an important concern for health care planning and policy formulation. Given this inherent variability, the question arises whether a county has the requisite capability to meet the demand for public health services. The concern is whether a local or regional public health system can provide public health services in areas where demand is not under its control. For example, not all of the indicators used in the model can be addressed by a public health system. Unemployment rates, education, and other social issues that have a significant influence on health are examples of concerns that can not be affected by the public health system.
In Mississippi, the State Health Department (MSDH) operates approximately 200 clinics in 82 counties. These clinics are organized into nine administrative health districts. The MSDH provides basic public health services throughout the state, such as immunizations, family planning, health promotion, health education, disease screening, disease surveillance, license and certification of health facilities and health professionals, and environmental health. The MSDH also provides limited medical services in some counties where there is an identified lack of other primary care providers. The array of services provided by the MSDH is dependent on the availability of services from private and other public sector providers.
The capability differentials across counties must be evaluated and understood in order to establish priorities for disparity minimization and equalization intervention targets. Establishment of priorities is intuitive and subjective, requiring insight, experience, and wisdom. This task is best accomplished through a very structured and well-defined procedure.
Public health capability refers to the degree of occurrence of factors associated with access to health promotion and disease prevention services, health care processes and outcomes, and socio-environmental factors that influence the relative well-being and quality of life of populations.
The methodology for geographically determining and ranking relative public health capability involves the development of a scoring system that assigns a numerical score to each county (Raifaa, 1996). This score represents the county's relative capability of basic public health services and allows each county to be ranked relative to all others in the state. This decision analytic methodology, known as multiple attribute utility evaluation, has been used in previous health services research (Fos & Zuniga, 1999; Fos, Miller, Amy, Zuniga, 2004).
The public health capability scoring system is composed of 20 weighted indicators divided into four major categories. Indicator selection was based on the Center for Disease Control and Prevention's Consensus Set of Health Status Indicators for the General Assessment of Community Health Status(CDC, 2004), United Health Group State Health Ranking-Selection of Components (United Health Group, 2007), and the Institute of Medicine Priority Areas for National Action: Transforming Health Care Quality (Adams and Corrigan, 2003). Population halth inputs and outputs are represented by indicators that depict the healthcare delivery system and characterize: access to, and quality of care; health outcomes; and, populations at- risk for disease and/or death (Alciaiti & Glanz, 1996; Last & Logan, 1999).
Model indicators and their impact on county public health capability are presented in Figure 1. There are four major categories of model indicators: (1) access to health care, (2) quality of care, (3) populations at higher risk of disease and/or death, (4) health outcomes. All 20 explanatory indicators used in the scoring model are collected, updated, and available from state and federal agencies.
Indicators that describe access to healthcare characteristics include percent of pregnant women receiving prenatal care in the first trimester, percent of county population covered by designated Health Professional Shortage Area (HPSA), physician to population ratio, percent CHIP enrollment, and percent of population with Medicare enrollment in each county.
Quality is an important issue today in the provision of public health care services. Five quality of care indicators are included in the public health capability scoring model. These include: (1) children immunization coverage; (2) adult immunization coverage; (3) percentage of nursing home residents with infections; (3) mean nursing home nursing staff hours per resident per day; and, (5) cesarean section rates per 1,000 live births.
The socio-economic condition in each county was evaluated using indicators that describe populations at high health risks. Five indicators were included to describe high health risk populations: (1) childhood poverty, measured by the proportion of children under 15 years of age living in families at or below the poverty level, (2) births to adolescents (ages 10-17 years) as a percentage of total live births, (3) prevalence of low birth weight, measured by the percentage of live born infants weighing under 2,500 grams at birth, (4) unemployed population, and (5) high school graduation rate.
The final category of the public health capability scoring model contains mortality and morbidity indicators, essential for evaluating public health care delivery systems. These include cardiovascular disease deaths per 100,000 population, potential years of life lost to age 75, female breast cancer incidence per 100,000 women, race/ethnicity-specific infant mortality rate, and motor vehicle crash deaths per 100,000 population.
SOURCES OF WEIGHTS AND MODEL DATA
The weighting of indicators was based on expert input, a well accepted technique whose procedure has been reported in studies of health care issues (Hankins & Fos, 1989; Fos & McLin, 1990; Edwards & Newman, 1982). Since the assigned relative weights in this study represent value judgments of policy makers and administrators, this is a valid and reasonable approach for planning and policy formulation. Determining value trade-offs of stakeholders is essential in evaluation and policy formulation. The group of experts selected for this procedure was composed of eighteen physicians, health care planners, health care policy makers and public health personnel who are very knowledgeable of public health in Mississippi.
Experts were interviewed in groups and individually by the authors. Interviews ranged from 45 minutes to an hour. Experts completed a structured questionnaire that rank ordered and rated indicators according to their relative importance to (or impact on) the public health capability of a county. The experts individually ranked and rated indicators following the method discussed earlier.
The model does not seek expert consensus to develop the scoring procedures (Morgan, DeKay, Fischbeck, et. al., 2001). The concern for agreement can be evaluated by calculating the construct known as Kendall's coefficient of concordance. Kendall's statistic tests the hypothesis that there is no agreement across and among experts. Table 1 presents the results of these tests.
After the expert group ranked and rated model indicators, individual indicator-specific weights were calculated based on each expert's input across model indicators. These individual weights represent normalized weights of relative priority of each indicator in relation to public health capability. After the individual normalized weights were calculated, the mean values of all the experts' input were computed in order to be used as the model indicator weights. Figure 1 includes indicator weights identified by the expert group.
Model indicator data were obtained from secondary sources and compiled by the authors. Demographic information originated from 2000 United States Census data. Mortality and morbidity data were obtained from existing, routinely collected databases of the MSDH. Additional indicator data were obtained from the Medicaid and Medicare Office in Mississippi. SPSS for Windows version 15 was used to compute public health capability model variables and to test the hypothesis of agreement among experts (SPSS, Chicago, IL, 2006).
DEVELOPMENT OF THE PUBLIC HEALTH CAPABILITY SCORING MODEL
The methodology presented below follows the techniques presented previously by the authors.6 Following identification, each indicator was assigned a specific public health capability score. This capability score indicates the comparison of county-wide indicators to a standard, the indicator value for the state. The indicator-specific score was determined as follows:
IndicatorSpecific _Score = (County _Value/State _Value) x 10
The county value indicates the specific indicator value for the county and the state value indicates the specific indicator value for the state. Ratios of county and state values are then multiplied by 10 to avoid small numbers. This process was repeated for all 20 explanatory indicators for each county.
The technique used in the scoring model to assign weights to indicators is classified as numerical estimation (von Winterfeldt & Edwards, 1986). After all indicators are rated, the ratings are summed, and each individual indicator's rating is divided by the sum of the ratings. This is a computational step which converts indicator ratings into a mathematical representation of the influence of each indicator on public health capability. This summation step assumes an additive function, so the sum of all weights equals 1 . Even though the indicators are not independent, as is the case in all complex problems, an additive function was used because of the documented robustness of additive models, as well as, the relative difficulty in building more comprehensive multiplicative models (Dyer & Sarin, 1979; Keeney & Raiffa, 1996).
After the indicator-specific scores and indicator weights have been determined, the scoring system, which will calculate each county's public health capability score, can be constructed. The scoring system proposed is an additive model, and it is represented as follows:
County _Capability _Score = [n.summation over (i=1)] WiVi
Where, [w.sub.i] = each individual indicator weight, and [v.sub.i] = each indicator-specific score.
Since we have assumed an additive model, the individual weighted indicator scores can be added together to determine the county score. It is important to note that weights of indicators which have a negative effect on public health capability are assigned a negative sign, and weights of indicators which have a beneficial effect on public health capability are assigned a positive sign. This is analogous to positive and negative coefficients in multiple regression statistical techniques.
Figure 1 represents the weighted value judgments of all experts. In this study, model indicators are issues that may influence the health status of a population and may determine the state of public health capability within each county. The preference given to a model indicator by the stakeholder shows the impact (positive or negative) of such attributes on the health status of a community. To distinguish the relative importance of the model indicators, the scores are calculated and normalized to sum 1 (Fos & Zuniga, 1999; Edwards, 1977).
Following the calculation of individual normalized weights, the computation of mean values of all experts' input was possible; these mean values are the model variable weights. The weighting of the four major categories reveals the order of importance given to these categories as they impact public health capability. Health access/resources have a weight of 29%; healthcare quality has a weight of 27%, high risk populations have a weight of 26%; and health outcomes have a weight of 18%. Weighting of individual variables shows the impact of the top five variables on the model. These are: child immunization coverage (15%); child poverty (10%); access to adequate prenatal care (9%); cardiovascular death rate (8%); and percent of population covered by (Note: Acronym needs to be written out--could not find a previos reference to it--Health Public Service Agencies???) HPSAs (7%). Toghether, these variables have an aggregate weight of 49% of the overall model.
Table 2 presents the distribution of overall public health capability scores, depicting the presence and development of public health activities across the state. DeSoto County has the highest public health capability score of 1.358 in relation to all others in the state. This indicates that DeSoto County is the best suited to meet the demand for basic public health services when compared with the rest of the state. Pearl River County follows, with a public health capability score of 1.164. Marshall County has the lowest public health capability score in the state, -5.789. Others with low capability scores include: Humphreys County (-3.687), Holmes County (-3.023), Issaquena County (-2.979), and Tallahatchie County (-2.789).
Geographical Information Systems is a viable method for public health disparity evaluation and planning (Gordon & Womersley, 1997). Figure 2 presents the State of Mississippi according to the statistical distribution of public health capability scores.
[FIGURE 2 OMITTED]
DISCUSSION AND CONCLUSIONS
As in any analytical model which attempts to quantify priorities and individual trade-offs, the results should be viewed as recommendations to health system planners and policy makers. It is important to reiterate that several determinants of public health capability are not under the influence of the public health services delivery system, but are controlled by the private sector and other public entities. However, there are many disparity determinants that the public health delivery system has influence over, to affect some level of improvement. Through the policy making arm of most public health delivery systems, highlighting areas of improvement may have a profound influence on improvements in education, employment status, and other socioeconomics.
Given these limitations of the influence of the public health delivery system, the discussion below will address the uses of the capability assessment model. The strength of the model is that it allows for an aggregate numerical value which represents several aspects of public health systems. In this study, these aspects include 20 indicators, each measured in different units. Traditional decision-making methodologies did not have the ability to combine the effects of these indicators on public health capability. Multi-criteria decision-making approaches allow for one unique numerical value which reflects disparities, alternatives, choices, and preferences associated with the decision situation.
Information derived from this model can be used to (a) identify areas within a state which need targeting for some form of intervention, (b) discriminate between specific areas which can be directly affected by the public health care system and areas in which other agencies have control, (c) identify specific needs of targeted areas, (d) guide budget and resource allocation, and (e) establish expectations for disparity reduction or equalization in a targeted area.
The public health capability assessment model can be used to make absolute or relative comparisons. Absolute comparison uses the county's individual rank as a benchmark. Improvement in model variables can be evaluated with respect to whether a county's score undergoes change. If the county's score moves higher, then this is a reasonable improvement. If a county's score is unchanged, then more thought and discussion should take place before targeting the areas described by the model variables.
Relative comparisons are made by evaluating the public health capability for each individual county compared to all others. Evaluation of an individual county's rank in the State gives a relative view of the county's public health capability. This benchmark can be used to direct improvement efforts in targeted areas indicated by the model that will change the county's rank and thus, equalize disparities. For example, a county may plan to expend resources to increase the percent of children who are immunized. This will change the county's public health capability score, but may or may not improve the county's rank. It is important to mention that the mean performance value is a "moving average" and interpretation of the public health capability score must be carefully considered, examining each of the 20 indicators that comprise the score. For a dynamic online demonstration of the model visit http://www.msdh.state.ms.us/county/CountyPlanningModu le/CPM.html
With only minor exceptions, counties with the highest of public health capability scores and those with the lowest of scores are not randomly distributed throughout the state. In these two categories appear geographic clusters, which may be used to guide policy and planning. Since the Mississippi model employs several parameters which are health status indices, it is possible that some in the upper quartile of scores may be less capable of accessing public health services than is apparent. Neighboring may positively influence each other with respect to public health capability. One county may have more than adequate resources for some services, while the neighboring county may have more than adequate resources for other services. A complementary effect may exist, where residents from neighboring counties may benefit from each other's public health services.
Given the identified need for improvement, and the limited supply of resources and funding, the model can be used to focus attention on areas of least capability. One improvement strategy is to individually review each model parameter for counties. The counties in the 25th percentile scores which are also in the 25th percentile of indicator-specific scores are the counties of concern. After specific counties are targeted, the model has the ability to identify specific areas for improvement in a county. The indicator-specific scores of targeted counties can be reviewed relative to other un-targeted counties and the state as a whole. Indicator specific scores for each parameter in the model are ordered numerically across targeted counties. Those parameters in the targeted counties which ranked low are selected for improvement.
Finally, the model is ideal for establishing expectations and benchmarks for reduction or equalization of disparities. Specific values of selected parameters can be calculated in advance, serving as disparity intervention goals. These "improvement" values are important in deciding whether it is practical or feasible to plan and implement specific interventions. "Improvement" values must be attainable before planning, funding, and implementation of interventions can occur.
Adams K, Corrigan JM, eds. (2003). Priority Areas for National Action: Transforming Health Care Quality. Washington, DC: The National Academies Press.
Alciati M.H. and Glanz K. (1996). Using Data to Plan Public Health Programs: Experience From State Cancer Prevention Programs. Public Health Reports, 111(2):165-72.
Centers for Disease Control and Prevention (1991). Consensus Set Of Health Status Indicators For The General Assessment Of Community Health Status: United States. Morbidity and Mortality Weekly Report. 40(27): E-1-E-3.
Chatburn R.L. and Primiano, Jr., F.P. (2001). Decision Analysis for Large Capital Purchases: How to Buy a Ventilator. Respiratory Care, 46(10):1038-53.
Dyer J.S. and Sarin R.A.(1979). Measurable Multiattribute Value Functions. Operations Research, 22:810-22.
Edwards W. (1977). How to Use Multiattribute Utility Measurement for Social Decision Making. IEEE Transactions on Systems, Man and Cybernetics, SMC-7, 326-346.
Edwards W. and Newman J.R. (1982). Multiattribute Evaluation, Sage University Paper Series in Quantitative Applications in the Social Sciences, Series No. 07-026, Beverly Hills, CA.
Fos P.J. and McLin C.L. (1990). The Risk of Falling In the Elderly: A Subjective Approach. Medical Decision Making, 10(3): 195-200.
Fos P.J. and Zuniga M.A. (1999). Assessment of Primary Care Health Care Access Status: An Analytic Technique for Decision Making. Health Care Management Science, 2(4):229-38.
Fos PJ, Miller DL, Amy BW, Zuniga MA. (2004). Combining the Benefits of Decision Science and Financial Analysis in Public Health Management: A County-specific Budgeting and Planning Model. Journal of Public Health Management Practice. 10(5): 406-412.
Gordon A. and Womersley J. (1997). The Use of Mapping in Public Health and Planning Health Services. Journal of Public Health Medicine, 19(2):139-47.
Hankins R.W. and Fos P.J. (1989). Objectives for a System of Health Care Delivery for HIV-Infected People. Socio-economic Planning Sciences, 23(4):181-193.
Jia H, Muennig P, and Borawski E. (2004). Comparison of Small-Area Analysis Techniques for Estimating County-Level Outcomes. American Journal of Preventative Medicine, 26(5): 453-460.
Keeney R.L. and Raiffa H.(1996). Decisions with Multiple Objectives: Preferences and Value Tradeoffs. New York: John Wiley and Sons.
Last J. and Logan H. (1999). Monitoring, Surveillance And Research Needs. Public Health Planning Priorities and Policy Options. Canadian Journal of Public Health. Revue Canadienne de Sante Publique, 90(6):SU 1-16.
Morgan K.M., DeKay M.L., Fischbeck P.S., Morgan M.G., Fischhoff B., and Florig H.K. (2001). A Deliberative Method For Ranking Risks (II): Evaluation Of Validity And Agreement Among Risk Managers. Risk Analysis. 21(5):923-37.
Stevens A., and Gillam S. (1998). Health Needs Assessment: Needs Assessment: From Theory to Practice. British Medical Journal. 316(7142):1448-1452.
Raiffa H. (1996). The Art and Science of Negotiation. Cambridge, MA: Belknap Press.
Reid W.M., Barnette D.M., and Mahan C.S. (1998). Local Health Departments: Planning For a Changed Role in the New Health Care Environment. Journal of Public Health Management & Practice, 4(5): 1-12.
United Health Group (UHG) (2007). America's Health: State Health Ranking 2007 Edition. St. Paul, MN: UHG.
von Winterfeldt D. and Edwards W. (1986). Decision Analysis and Behavioral Research, Cambridge University Press, Cambridge.
MIGUEL A. ZUNIGA
South Texas Center
Texas A&M University System
Health Science Center
School of Rural Public Health
Texas A&M University System
Health Science Center
YOON HO SEOL
Medical College of Georgia
PETER J. FOS
The University of Texas at Tyler
Table 1 Kendal Coefficient of Concordance Expert Group k df [chi square] p-value W All 18 23 115.10 <0.00000 0.312 Board of Health 7 23 38.18 0.02348 0.331 Members District Health 6 23 62.97 <0.00001 0.456 Administrators Central Office Staff 5 23 43.17 0.00662 0.375 Table 2 Public Health Scores Across All Counties Rank Upper Score Quartile 1 DeSoto County 1.35872 2 Pearl River County 1.16459 3 Hancock County 1.08799 4 Itawamba County 1.03010 5 Lafayette County 0.94927 6 Prentiss County 0.91636 7 Lamar County 0.86815 8 Harrison County 0.57041 9 Newton County 0.50921 10 Rankin County 0.50778 11 Forrest County 0.41914 12 Wayne County 0.40843 13 Amite County 0.33519 14 Madison County 0.33190 15 Smith County 0.29946 16 George County 0.29631 17 Yalobusha County 0.28516 18 Tippah County 0.28121 19 Oktibbeha County 0.23892 20 Lawrence County 0.07274 21 Pontotoc County 0.06704 Rank Mid-Upper Score Quartile 22 Simpson County 0.01561 23 Hinds County -0.00882 24 Calhoun County -0.04571 25 Tate County -0.11152 26 Lee County -0.11702 27 Jones County -0.16877 28 Scott County -0.20517 29 Alcorn County -0.20575 30 Union County -0.21343 31 Neshoba County -0.37476 32 Warren County -0.48666 33 Stone County -0.49765 34 Attala County -0.49929 35 Lincoln County -0.50961 36 Perry County -0.52258 37 Jackson County -0.53995 38 Jasper County -0.54593 39 Lauderdale County -0.70650 40 Panola County -0.72961 41 Walthall County -0.78904 42 Sharkey County -0.82263 Rank Mid-Lower Score Quartile 43 Monroe -0.82702 44 Tishomingo County -0.82924 45 Marion County -0.84707 46 Copiah County -0.87878 47 Sunflower County -1.00571 48 Clarke County -1.04767 49 Chickasaw County -1.06705 50 Bolivar County -1.11142 51 Lowndes County -1.12705 52 Montgomery County -1.12865 53 Leake County -1.30313 54 Covington County -1.35419 55 Adams County -1.35535 56 Pike County -1.37140 57 Benton County -1.50750 58 Claiborne County -1.53612 59 Quitman County -1.54115 60 Grenada County -1.59706 61 Tunica County -1.65324 62 Yazoo County -1.65376 Rank Lower Score Quartile 63 Carroll County -1.67668 64 Winston County -1.71860 65 Greene County -1.74070 66 Wilkinson County -1.76331 67 Jefferson Davis County -1.87106 68 Franklin County -1.93648 69 Jefferson County -2.02680 70 Kemper County -2.28934 71 Coahoma County -2.42653 72 Choctaw County -2.43448 73 Leflore County -2.51697 74 Clay County -2.55039 75 Webster County -2.56901 76 Noxubee County -2.66457 77 Washington County -2.78341 78 Tallahatchie County -2.78910 79 Issaquena County -2.97996 80 Holmes County -3.02318 81 Humphreys County -3.68726 82 Marshall County -5.78989 The county with the highest level of capability is ranked 1, and the county with the lowest level of capability is ranked 82. Figure 1 Value Tree Health Access/ Resources IMPACT 0.2936 Percent Prenatal care|0.086 Positive Percent Population covered by HPSA|0.017 Positive Population to Physician Ratio|0.059 Positive Percent CHIP Enrollment|0.059 Positive Percent Medicare Population|0.020 Positive Healthcare Quality 0.2677 Child Immunization Coverage|0.151 Positive Adult Immunization Coverage|0.048 Positive Percent Nursing Home Residents Negative with Infections|0.030 Mean Nursing Home Staffing per Resident|0.022 Positive Cesarean Section Rates|0.016 Negative High Risk Populations 0.2584 Childhood Poverty|0.095 Negative Births to Adolescents|0.051 Negative Prevalence of Low Birth Weight|0.049 Negative Unemployed Population|0.037 Negative High School Graduation Rate|0.027 Positive Health Outcomes 0.1804 Cardiovascular Deaths|0.080 Negative Potential Years of Life Lost|0.033 Negative Female Breast Cancer Mortality|0.029 Negative Infant Mortality|0.026 Negative Motor Vehicle Crash Death|0.012 Negative
|Gale Copyright:||Copyright 2009 Gale, Cengage Learning. All rights reserved.|