Leading for quality in healthcare: development and validation of a competency model.
Article Type: Survey
Subject: Health care industry (Management)
Medical law (Evaluation)
Medical care (Quality management)
Medical care (Analysis)
Authors: Garman, Andrew
Scribner, Linda
Pub Date: 11/01/2011
Publication: Name: Journal of Healthcare Management Publisher: American College of Healthcare Executives Audience: Trade Format: Magazine/Journal Subject: Business; Health care industry Copyright: COPYRIGHT 2011 American College of Healthcare Executives ISSN: 1096-9012
Issue: Date: Nov-Dec, 2011 Source Volume: 56 Source Issue: 6
Topic: Event Code: 200 Management dynamics Computer Subject: Health care industry; Company business management
Product: Product Code: 9105280 Health Regulation NAICS Code: 92615 Regulation, Licensing, and Inspection of Miscellaneous Commercial Sectors SIC Code: 8000 HEALTH SERVICES
Geographic: Geographic Scope: United States Geographic Code: 1USA United States
Legal: Statute: Patient Protection and Affordable Care Act
Accession Number: 274519826
Full Text: EXECUTIVE SUMMARY

Increased attention to healthcare quality and impending changes due to health reform are calling for healthcare leaders at all levels to strengthen their skills in leading quality improvement initiatives. To address this need, the National Association for Heakhcare Quality spearheaded the development and validation of a competency model to support healthcare leaders in assessing their strengths and planning appropriate steps for development. Initial development took place over the course of several days of meetings by an advisory panel of quality professionals. The draft model was then validated via electronic survey of a national sample of 883 quality professionals. Follow-up analyses indicated that the model was content valid for each of the target samples and also distinguished differing levels of job scope and experience. The resulting model contains six domains spanning three organizational levels.

INTRODUCTION

As delivery of high-quality healthcare continues to grow more complex, so do the roles of the professionals leading these efforts. Recent years have seen increased focus on the leadership elements of the quality professionals' roles; initiatives such as the Comprehensive Unit-Based Safety Program (CUSP; Pronovost et al. 2005), crew resource management, Lean Six Sigma, and Malcolm Baldrige emphasize the key elements of leadership and management needed for success (see Garman et al. 2011). Given the impending changes associated with the Affordable Care Act, leaders are likely to be charged with implementing these quality improvement initiatives within a context of increasing emphasis on resource efficiency. While the opportunities to improve may be tremendous, threading the value needle will likely test the mettle of all leaders of quality efforts in the years to come.

In preparation for this new era for the quality professional, the National Association of Healthcare Quality (NAHQ) began an initiative to investigate the leadership development needs of the profession. Their efforts yielded a competency model that is specific to leadership in quality and holds implications for professionals across the career path. Development and validation of this model are described in the following section.

METHOD

Development of the competency model began in June 2008. Members of the NAHQ board agreed to serve as the advisory panel for developing a leadership model and convened a two-day series of meetings to develop the draft. The meetings proceeded through three phases: clarification of goals, definition of scope, and competency identification. In the competency identification phase, subcommittees were formed to represent the perspectives of presentstate, future-state, and senior leadership. Using the Health Administrators Leadership model (HAL) (Garman, Tyler, and Darnall 2003) as their seed model, the subcommittees reviewed this and other published models against the goals and scope criteria developed during the phase one and two meetings. These separate reviews were then compiled during a large group meeting to form the first draft. These results were summarized overnight and disseminated the following morning for further discussion. The revised model contained 21 competencies organized into six domains. This model was circulated to the committee a third time several weeks after the original meeting, and feedback was collected via teleconference. At that point the board agreed by consensus that the model reflected the elements they believed would be essential to quality leadership moving forward.

With the draft of the model finalized, the next step was to conduct a construct validation study. To accomplish this, an electronic survey was developed for distribution to quality professionals, using methods adapted from Williams and Crafts (1997). The survey asked respondents to review each of the competency descriptions and rate it according to level of perceived importance to their quality leadership role using a five-point scale of relative importance (1 = No importance, 5 = Extremely important). An open-ended question followed each competency, to allow respondents to comment on clarity, word choice, or other concerns with the way the competency was defined. Respondents were also asked to answer a set of demographic questions concerning the nature of their positions and their institutional settings.

Invitations to complete the survey were sent to all members of NAHQ (n = 4,445), using a list maintained by this organization and a list of non-NAHQ members holding the Certified Professional in Healthcare Quality (CPHQ) credential (n = 2,704), which was provided by the Healthcare Quality Certification Board.

RESULTS

A total of 883 quality professionals responded to the survey, for a response rate of 12 percent. Respondent demographics are provided in Exhibit 1. For respondent role, four categories were created to collapse roles to a manageable number: senior executive leadership (CEO, president, chief of staff, chief or VP of medical affairs, chief nursing executive, chief operating or operations officer, chief financial officer); senior quality leadership (president or vice president for quality services, quality improvement, patient safety divisions/ departments, physician quality leader (department or service line); mid-level quality leadership (director or manager of quality, patient safety, and/or risk management); and direct contributor (quality coordinator, quality specialist, data analyst, data abstractor).

To determine the validity of the model and its component competencies, a series of content validity ratios (CVR) (Lawshe 1975)were calculated. CVRs indicate the proportion of respondents who agree that a particular competency is very or extremely important. This analysis indicated that all competencies were above the recommended threshold value of 0.49 (range: 0.66 for "Financial Acumen" to 0.97 for "Professional Ethics").

Model Structure

The association between competencies and domains was examined using principal components factor analysis, a statistical technique that tests the extent to which items on a survey tend to be rated similarly by respondents (higher levels of association suggest that the items hang together more closely, forming associations). To test whether the domains originally defined would map to the competencies we originally specified, the analysis was set up to fit the data to a six-domain solution. Results of this analysis, shown in Exhibit 2, supported the original structure for 17 of the 21 competencies and identified four areas in need of reconciliation. First, the systems thinking competency, which was originally in the organizational awareness domain, loaded more heavily onto the fosters positive change domain. Second, the lifelong learning competency, originally in the professionalism domain, loaded relatively equally onto the self-management and professionalism domains and slightly higher onto self-management. Third, the professional ethics competency, which was originally under the professionalism domain, loaded more strongly onto the self-management domain. Fourth, drive for results, originally under the passion for positive change domain, loaded equally strongly onto two other domains: performance improvement and self-management. These four findings were vetted with the original advisory panel, who by consensus agreed that the survey results made conceptual sense and should be used to guide revision of the model. These final revisions yielded the model shown in Exhibit 3.

Subgroup Analyses

Our next step was to develop a more refined understanding of how the competencies may differ in their relative importance as a function of organization setting, organization size, experience level, and position level.

For organization setting, although the survey contained a question about primary practice setting, within-category response counts were too small for meaningful separate analysis of these results. For this reason, we focused only on respondents who provided a bed count for their organizations in the demographics section of the survey. A total of 267 (31%) respondents indicated "not applicable" and were categorized as nonhospital settings. The remaining 69 percent were categorized as hospital settings. A series of statistical analyses (ANOVAs) was conducted to determine whether there were differences between hospital and nonhospital groups on the importance of the competencies and competency domains. These analyses yielded no statistically significant differences between these groups, providing some evidence for generalizability across organizational settings.

To examine the influence of organization size and experience level, we followed a similar approach. For organization size, we focused only on hospital settings and only considered those respondents who reported bed counts associated with their organizations. These analyses suggested significant differences as a function of organization size for four competencies within the organizational awareness and fosters positive change domains (strategic planning, strategic thinking and alignment, partners for change, and drives for results), as shown in Exhibit 3. Level of experience was also significantly associated with five competencies (strategic thinking and alignment, financial acumen, systems thinking, analytic thinking/knowledge-based decision making, and listening and receiving feedback). For all significant effects, the direction of the effect was for larger organizations and higher experience levels to be associated with higher levels of importance for the competency/domain.

To analyze competencies according to position level, subgroups were formed according to responses to the demographic question "The primary functional role (not necessarily the title) of my current position related to quality or patient safety is:--." Responses were categorized into the four role levels described previously (senior executive leadership, senior quality leadership, mid-level quality leadership, and direct contributor). For the competency analysis, respondents were selected only if they reported a bed count in the demographic section (indicating they worked in a hospital setting). This analysis yielded statistically significant associations for 5 of the 21 competencies: one from the revised organizational awareness domain, two from the revised fosters positive change domain, and one each from the performance improvement and professionalism/professional values domains. Those five competencies were strategic thinking and alignment, drive for results, systems thinking, analytic thinking/knowledge-based decision making, and future focus. In all cases, higher organizational level was associated with greater perceived importance of the competency, suggesting that these five competencies would be particularly appropriate foci for leadership development programs that center on career progression to higher organizational levels.

DISCUSSION AND CONCLUSION

Several important limitations of this work are important to keep in mind when interpreting the results. First, although the sample size obtained for the content validity study was large, the small response rate means the responses could differ in important ways from the population from which they were drawn. Second, because the focus of this project was on leadership in quality, the model should not be considered a complete description of all critical elements of a quality professional's job. Effectiveness in a quality leadership role will typically also require mastery of a considerable knowledge base and technical competencies that are beyond the scope of the present effort. Third, competency models are only as useful as the leadership-development efforts they support. When they are incorporated into development programs based on sound adult learning principles, competency models can be powerful facilitators of individual change, but by themselves these models do little to help people develop. Last, the effectiveness of even the most skilled quality professionals ultimately will be bounded by the level of collaboration they experience from the other leaders and clinicians they work with. Attention to leadership as a team sport and the need for a continuous learning focus will continue to be essential in supporting quality improvement gains in healthcare organizations.

These limitations notwithstanding, results of this study suggest that the Quality Leadership model and its component competencies can be considered content-valid descriptions of the areas quality leaders need to master to be effective in their roles and thus represent a useful model for leadership development within the quality profession. This appears to hold true regardless of setting (hospital or other) or of other professional characteristics such as experience, job level, or organization size. Additionally, the relative importance of a number of the competencies and domains increased as a function of higher position level and larger, more complex organizational settings, suggesting specific areas that may be particularly useful foci for leadership-development efforts.

Based on the patterns of results described previously, we constructed the graphic model depicted in Exhibit 4. The pyramid shape captures the hierarchy of the competencies according to the analyses described in the prior sections. In particular, professionalism/professional values appears as the base to indicate the critical role that this domain (and professional ethics in particular) plays across all quality positions. The next level contains foundational skills that, while still relevant to all positions, start to look different at different organizational levels. The top level, which contains organizational awareness and fostering positive change, is the most closely associated with higher organizational levels.

Taken together, these competencies illustrate several important ways in which successful leadership of quality efforts may differ from a more general definition of leadership effectiveness. In comparison to other widely recognized healthcare leadership competency models (for a review, see Garman and Johnson 2006), there appears to be a greater relative emphasis on developing and maintaining a culture of continuous process improvement (partnering for change, cultivating a quality-supportive climate) and the competencies necessary for the analytic work associated with data-driven decision making (managing data, analytic thinking, developing a knowledge-rich environment). It is perhaps not surprising that both emphases map closely to areas focused on by the Malcolm Baldrige award criteria (National Institute of Standards and Technology 2010).

[ILLUSTRATION OMITTED]

Conversely, competencies with an external focus (e.g., community relations, board relations) or longerterm focus (e.g., fundraising, talent development), or a focus on day-today operations (e.g., human resource management, information technology management) are absent or deemphasized. So the model does appear to trade a narrower focus for greater depth of focus on quality leadership specifically.

In terms of developing current and future quality leaders, competency models can support these efforts in a variety of ways. Competency models are particularly helpful for identifying areas in which further development may provide the greatest relative payoff. For example, the competency definitions can be used as a template for creating developmental 360-degree feedback programs in preparation for on-the-job development. Similarly, the model can be used as a framework for a development planning discussion between leaders and their direct reports, using templates such as those provided by Garman and Dye (2009). Models that are adopted by a hospital or health system's senior leaders can be particularly useful for aligning the developmental agenda of the entire organization. For example, an employee and organizational development department can cross-walk an organization-endorsed competency model against internally provided training and development programs to better communicate these opportunities to employees and to identify potential gaps in their offerings. Additionally, assessments such as 360-degree feedback can be aggregated to the organization level to identify and begin to address broader skill gaps.

In conclusion, this model appears relevant and useful for leaders and educators interested in developing capacity in the area of quality leadership. In support of open dissemination, and in the spirit of viewing quality as everyone's responsibility, all competency descriptions and supplementary material have been made available on the NAHQ website. Our hope is that the availability of this model will help organizations attend to the leadership development needs of their quality professionals.

PRACTITIONER APPLICATION

Janet Holdych, PharmD, CPHQ, director of quality, Catholic Healthcare West, San Francisco, California

Ten years ago the Institute of Medicine's landmark report Crossing the Quality Chasm called out the US healthcare delivery system for not consistently providing high quality care to all people. The report concluded that fundamental changes were needed in order to provide safe, effective, patient-centered, timely, efficient, and equitable care (IOM 2001). Operationalizing this vision has led to a decade of change in the healthcare field as healthcare providers, payers, regulators, and policy makers work toward improving the quality of care and reducing costs through an increased focus on applying evidence to healthcare, implementing information technology, and aligning payment policies with quality improvement (e.g., pay for reporting, value-based purchasing). Healthcare organizations have felt the impact through increased data collection and reporting requirements and the implementation of healthcare quality improvement initiatives. This increased attention to hospital quality performance has driven organizations to make certain their leaders have the skills and tools they need to participate in or lead quality of care improvement work.

The National Association for Healthcare Quality (NAHQ) competency model that Garman and Scribner describe is a useful tool that supports healthcare leaders in assessing their ability to lead and drive performance improvement. Fortunately, Catholic Healthcare West was an early adopter of the model; we used the NAHQ competencies as a starting point toward the development of our organizationwide Quality Leader competency model. Its development was deemed a priority in response to the increased focus on hospital quality and what appeared to be a shortage of qualified quality leaders to fill vacant leadership positions in our hospitals. The Quality Leader competency model follows in the footsteps of other competency models our internal Organizational Development department established for our organization's C-suite positions. We recognized that our success to achieving our mission of providing high-quality, affordable healthcare to the communities we serve was dependent on our people. To best leverage our talent, we needed to clearly understand the specific behaviors, knowledge, competencies, and skills that these leaders must demonstrate in order to be effective in their given role.

We became aware of the draft NAHQ competencies when the survey was released in 2008. Collaboration with NAHQ provided us with a starting point for our competency model and allowed us the opportunity to provide additional responses to their survey and vetting process. Our competency model was designed to assist our organization in developing incumbents' skills, identifying top talent for succession, and assessing the organization's overall bench strength. More important, it allowed us to develop competencies needed as we progressed toward leadership development.

As part of the process for developing our competency model, we solicited participation from job experts (quality leaders within our organization) and executive team members. The process included research and data collection, focus group sessions, drafted competency models for functional and job-specific levels, and validation with job experts, stakeholders, and the executive team. The validation process was thorough, and as a result, we have a fully integrated model. Supplementary documents to support the model were also developed, including a quality leader job description; a quality leader interview guide that contains behavioral interview questions; and a development catalogue consisting of courses, activities, books and periodicals, resources, professional organizations, and other development aids that are applicable to each of the defined competencies. Keeping competencies as the focus of these tools helps ensure that individuals are hired against the same criteria they will be developing and measured on in their jobs. In addition, quality leaders within our organization complete an individual development plan annually that the executive they report to reviews and the system office uses to identify potential development activities for the upcoming year.

Our end product is a competency model that defines the competencies and attributes that the hospital quality leader must demonstrate in order to be effective in their role. We believe that effective leadership and adequate resources in the quality department can lead to an effective program that supports and accelerates progress toward improved patient care. Garman and Scribner conclude that they hope their competency model will help organizations attend to the leadership development needs of their quality professionals, and we can attest that it already has.

REFERENCE

Institute of Medicine (IOM). 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press.

REFERENCES

Garman, A. N., and C. F. Dye. 2009. The Healthcare C-Suite: Leadership Development at the Top. Chicago: Health Administration Press.

Garman, A. N., and M. P. Johnson. 2006. "Leadership Competencies: An Introduction." Journal of Healthcare Management 51 (1): 13-17.

Garman, A. N., A. S. McAlearney, J. Robbins, P. Song, M. McHugh, and M. I. Harrison. 2011. "High-Performance Work Systems in Health Care Management, Part 1: Development of an Evidence-Informed Model." Health Care Management Review 36 (3): 201-13. http://journals.lww.com/hcmrjournal/Fulhext/ 2011/07000/High_performance_work_systems in health_care.1.aspx#.

Garman, A. N., J. L. Tyler, and J. S. Darnall. 2003. "Development and Validation of a 360-Degree Feedback Instrument for Healthcare Administrators." Journal of Healthcare Management 49 (5): 311-25.

Lawshe, C. H. 1975. "A Quantitative Approach to Content Validity." Personnel Psychology 38 (4): 563-75.

National Institute of Standards and Technology. 2010.2010-2011 Health Care Criteria for Performance Excellence. Accessed May 30, 2011. www.nist.gov/baldrige/publications/upload/ 2011_2012_Health_Care_Criteria.pdf.

Pronovost, P., B. Weast, B. Rosenstein, J. B. Sexton, C. G. Holzmueller, L. Paine, R. Davis, and R. Hava. 2005. "Implementing and Validating a Comprehensive Unit-Based SafeW Program." Journal of Patient Safety 1 (1): 33-40.

Williams, K. M., and J. L. Crafts. 1997. "Inductive lob Analysis: The lob/Task Inventory Method." In Applied Measurement Methods in Industrial Psychology, edited by D. L. Whetzel and G. R. Wheaton, 51-88. Palo Alto, CA: Davies-Black Publishing.

For more information on the concepts in this article, please contact Dr. Carman at Andy._N_Carman@rush.edu or agarman@nchl.org.

Andrew Garman, PsyD, MS, CEO, National Center for Healthcare Leadership, and professor, health systems management, Rush University; and Linda Scribner, BA, CPHQ, director of quality and clinical outcomes management, Methodist Dallas Medical Center
EXHIBIT 1

Respondent Demographics

                                                     N    (%)
Professional
  NAHQ member                                       254   (29)
  Certified professional in Health Quality (CPHQ)   160   (18)
  Both                                              469   (53)
Years working in quality
  Less than 1 year                                   14   (2)
  1-2 years                                          46   (5)
  3-5 years                                         117   (13)
  6-10 years                                        169   (19)
  11-15 years                                       137   (16)
  More than 15 years                                400   (45)
Hospital bed size (hospital-based professionals)    604   (68)
  Less than 50                                       61   (10)
  50-99                                              53   (9)
  100-199                                           110   (18)
  200-299                                           102   (17)
  300-399                                            83   (14)
  400-499                                            46   (8)
  500 or more                                       149   (25)
Job level (by categorizable role,                   521   (59)
  respondents in hospital settings)
  Senior executive leadership                        14   (3)
  Senior quality leadership                          44   (8)
  Mid-level quality leadership                      251   (48)
  Direct contributor level                          212   (41)

EXHIBIT 2 Factor Loadings Based on a Principal Components Analysis for
the Competencies on the Original Six Domains

                                        Factors
                                Fosters
                                Positive
Competency                       Change    Communicating

Advocates for and adapts          0.73
to change
Partners for change               0.66         0.27
Cultivates a quality-             0.66
supportive climate
Systems thinking                  0.58
Drives for results                0.41
Verbal communication skills                    0.82
Written communication skills                   0.75
Educating                         0.22         0.58
Listening and receiving           0.29         0.53
feedback
Strategic planning
Strategic thinking and            0.23
alignment
Financial acumen
Professional ethics               0.23
Manages personal limits
Resilience and self-restraint                  0.31
Lifelong learning                 0.25         0.24
Future focus
Consumer advocacy                              0.30
Managing data
Analytic thinking /decision
making
Develops a knowledge-rich         0.25
environment

                                                  Factors
                                Organizational
                                                   Self-      Future
Competency                      Awareness        Management   Focus

Advocates for and adapts                            0.22
to change
Partners for change
Cultivates a quality-                                         0.28
supportive climate
Systems thinking                0.39
Drives for results                                            0.40
Verbal communication skills
Written communication skills                        0.23
Educating                                                     0.34
Listening and receiving                             0.40
feedback
Strategic planning              0.83
Strategic thinking and          0.82
alignment
Financial acumen                0.64                          0.26
Professional ethics                                 0.75
Manages personal limits                             0.62      0.38
Resilience and self-restraint                       0.62      0.31
Lifelong learning                                   0.45      0.44
Future focus                    0.32                          0.68
Consumer advocacy                                             0.67
Managing data
Analytic thinking /decision
making
Develops a knowledge-rich                           0.26      0.39
environment

                                  Factors
                                Performance
Competency                      Improvement

Advocates for and adapts
to change
Partners for change
Cultivates a quality-
supportive climate
Systems thinking
Drives for results                 0.41
Verbal communication skills
Written communication skills       0.26
Educating
Listening and receiving
feedback
Strategic planning
Strategic thinking and
alignment
Financial acumen
Professional ethics
Manages personal limits
Resilience and self-restraint
Lifelong learning
Future focus
Consumer advocacy
Managing data                      0.81
Analytic thinking /decision        0.75
making
Develops a knowledge-rich          0.53
environment

Note: Factor Loadings < 0.20 are not reported.

EXHIBIT 3
Leadership in Quality: Final Model Structure and Associations
with Respondent Demographics

                                        Significant Associations

Domains/
Competencies                            Experience   Job Level

I. Fosters positive change                           **
  Advocates and adapts to change
  Partners for change
  Cultivates a quality-
    supportive climate
  Drives for results                                 **
II. Communicating
  Verbal communication skills
  Written communication skills
  Listening and receiving feedback      **
  Educating
III. Organizational awareness           **
  Strategic planning
  Strategic thinking and alignment      **           **
  Financial acumen                      **
  Systems thinking                      **           **
IV. Self-management
  Professional ethics
  Manages personal limits
  Resilience & self-restraint
V. Professionalism / Professional
values
  Consumer advocacy                                  **
  Future focus
  Lifelong learning
VI. Performance improvement
  Managing data
  Analytic thinking / knowledge-based   **           **
  decision-making
  Develops a knowledge-rich
    environment

                                        Significant Associations

Domains/                                Organization   Interaction
Competencies                            Size           Terms

I. Fosters positive change              **             **
  Advocates and adapts to change
  Partners for change                   **             **
  Cultivates a quality-
    supportive climate
  Drives for results                    **             **
II. Communicating
  Verbal communication skills
  Written communication skills
  Listening and receiving feedback
  Educating
III. Organizational awareness
  Strategic planning                    **             **
  Strategic thinking and alignment      **             **
  Financial acumen                      **             **
  Systems thinking
IV. Self-management                                    **
  Professional ethics                                  **
  Manages personal limits
  Resilience & self-restraint
V. Professionalism / Professional
values
  Consumer advocacy                                    **
  Future focus
  Lifelong learning
VI. Performance improvement
  Managing data
  Analytic thinking / knowledge-based                  **
  decision-making
  Develops a knowledge-rich
    environment

Note: Complete and current descriptions of each of the competencies are
available online: www.nahq.org/membership/leadership/devmodel.html

Group differences statistically significant at p < 0.05
Gale Copyright: Copyright 2011 Gale, Cengage Learning. All rights reserved.