Surgical pathology case reviews before sign-out: a College of American Pathologists Q-probes study of 45 laboratories.
Abstract: Context.--To avoid errors many surgical pathology services mandate review of a case by a second pathologist before reports are released (signed out).

Objective.--To study the extent and characteristics of such review.

Design.--Participants retrospectively examined up to 400 cases to identify a maximum of 30 cases reviewed by at least one additional pathologist before sign-out. For each case, participants documented the organ system, primary disease type, number of additional pathologists consulted, and the reason for case review. The main outcome measure was the fraction of surgical pathology cases that underwent second pathologist review before sign-out.

Results.--From 45 laboratories, examination of 18 032 surgical pathology cases yielded 1183 (6.6%) cases that had been reviewed before sign-out. The median laboratory reviewed 8.2% of cases. Three-fifths of reviews focused on 4 organ systems: gastrointestinal (20.5%), breast (16.0%), skin (12.7%), and female genital tract (10.0%). Malignant neoplasm far exceeded all other categories of disease in reviewed cases (45.3%). Cases were reviewed by one additional pathologist 78% of the time. Two dominant reasons for case review emerged: difficult diagnosis (46.2%) and audit required by departmental policy (43.0%). Most laboratories (71%) had departmental policies regarding review of cases. These laboratories reviewed cases about 33% more often than laboratories without policies (9.6% versus 6.5%).

Conclusions.--Review of selected surgical pathology cases before sign-out is widely accepted with 71% of participant laboratories following policies to this effect. About 1 case in 15 (6.6%) were reviewed with the median laboratory of participants reviewing about 1 in 12 (8.2%).

(Arch Pathol Lab Med. 2010;134:740-743)
Subject: Medical laboratories (Laws, regulations and rules)
Pathologists (Training)
Pathologists (Practice)
Pathologists (Laws, regulations and rules)
Medical errors (Prevention)
Authors: Nakhleh, Raouf E.
Bekeris, Leonas G.
Souers, Rhona J.
Meier, Frederick A.
Tworek, Joseph A.
Pub Date: 05/01/2010
Publication: Name: Archives of Pathology & Laboratory Medicine Publisher: College of American Pathologists Audience: Academic; Professional Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2010 College of American Pathologists ISSN: 1543-2165
Issue: Date: May, 2010 Source Volume: 134 Source Issue: 5
Topic: Event Code: 930 Government regulation; 940 Government regulation (cont); 980 Legal issues & crime; 280 Personnel administration; 200 Management dynamics Advertising Code: 94 Legal/Government Regulation Computer Subject: Government regulation
Product: Product Code: 7397030 Biomedical Testing Labs NAICS Code: 54138 Testing Laboratories SIC Code: 8071 Medical laboratories; 8731 Commercial physical research
Geographic: Geographic Scope: United States Geographic Code: 1USA United States
Accession Number: 230246560
Full Text: Early in training, pathologists are taught to seek the opinion of others when confronted with difficult diagnostic cases. Most pathologists develop particular interest and knowledge in specific organs and tend to seek the advice of others when presented with challenging cases outside this domain of interest and knowledge. Until recently, seeking advice on cases was a voluntary exercise. Recent attention to diagnostic errors in the medical literature and the public press has led to many pathology practices standardizing such case reviews as a plausible measure to prevent errors and to provide assurance of diagnostic accuracy. (1-11)

This trend has often been adopted through policies that require second pathologist review of particular types of case. The content of the case review tends to reflect previous specific experience with misdiagnosis in each surgical pathology practice. We report here a Q-Probes study of practices in 45 surgical pathology laboratories that reviewed cases before they were released (review before sign-out). To our knowledge, this is the first study to measure the frequency and characteristics of case review before sign-out in multiple institutions.

MATERIALS AND METHODS

This study was offered and conducted as a Q-Probes study, the basic mechanism of which has been previously described. (12) Briefly, Q-Probes studies are subscription-based quality assurance studies that are conducted at a certain point in time and have the advantage of peer group comparison to assist in determining level of performance.

This study was offered and conducted in the fourth quarter of 2008. Participants examined accessioned cases retrospectively from August 1, 2008, until 30 surgical pathology cases were identified that had documentation of review by at least a second pathologist before sign-out or until 400 consecutively accessioned cases had been examined.

For each identified, reviewed case, participants documented the organ system and primary disease type involved in the diagnosis, the number of additional pathologists consulted, and the reason for case review. Cases reviewed after sign-out and cases for which the second reviewer was a resident or fellow were excluded from the study. Participant laboratories also provided demographic information describing their laboratory and the institutions at which they serve, as well as practice variables that the study authors hypothesized might be linked to surgical pathology case review.

When data distributions for case review rate were statistically analyzed, the review rate was slightly skewed toward higher review rates, so a rank-based analysis for the univariate tests and log-transformation for the multivariate regression analysis were used. One participant had a 0% review rate and was excluded from the statistical analysis.

Individual associations between the review rate and the demographic and practice variables were investigated using Kruskal-Wallis tests for discrete-valued, independent variables and regression analysis for continuous, independent variables. Variables with significant associations (P < .10) were then introduced into a forward-selection, multivariate regression model. Variables remaining after this procedure would be significantly associated at the P < .05 significance level.

RESULTS

Of the 45 participant surgical pathology laboratories, most (89%) were located in the United States, 3 (7%) were in Saudi Arabia, and 1 (2%) each were in Australia and Canada; 37.8% (17 of 45) considered their laboratories to be teaching hospitals, and 22.2% (10 of 45) maintained pathology residency programs. Most of the laboratories (82.2%; 37 of 45) had been inspected by the Laboratory Accreditation program of the College of American Pathologists within 2 years of the study and 34.1% of institutions (15 of 44) had been inspected by the Joint Commission within the previous 2 years. Table 1 shows institutional demographic information about institutional type, bed size, location, and governmental affiliation. Table 2 shows the distribution of surgical pathology accessioned cases and of full-time pathologists who evaluated cases. The median participant laboratory processed 10 422 cases in 2007 and had 4 surgical pathologists as staff. Table 3 shows the characteristics that we hypothesized might influence case reviews; 32 of 45 laboratories (71.1%) had a policy regarding review of cases before case sign-out. Although most laboratories (24 of 45; 53%) did not target any percentage for review, 7 laboratories (15.6%) targeted 10% of cases for review, and 1 laboratory (2%) targeted 95% of cases.

In aggregate, 1183 cases (6.6%) were reviewed among 18 032 cases examined. The median rate of review was 8.2% with a skewed distribution shown in Table 4. The characteristics of the study sample, the number of cases reviewed per participating laboratory, and the number of accessioned cases examined to find the study sample of reviewed cases is shown in Table 5. Most participants reached the study target of 30 cases in less than 400 cases examined. Table 6 shows the organ systems involved in the cases reviewed before sign-out. The 4 most frequent categories, gastrointestinal, breast, skin, and female genital tract cases, accounted for about 60% of reviewed cases. Table 7 shows the primary disease type, number of pathologists consulted, and the reason for case review before sign-out. As expected, malignant neoplasms (45.3%) were the highest proportion of cases. When combined with premalignant (7.4%), low malignant/ borderline neoplasm (2.8%), and malignant mimics (benign proliferation/hyperplasia/atypia) (15.7%), this accounted for nearly three-quarters of all cases.

In evaluating practice and demographic variables, 4 factors were significantly associated with the level of review. Laboratory policy regarding case reviews before sign-out was associated with a higher percentage of reviewed cases (9.6% versus 6.5%). Nonfederal governmental institutions had a significantly lower rate of review versus nongovernmental and federal governmental facilities (3.5% versus 8.4% and 12.4%, respectively). Laboratories with review policies also reviewed malignancies at a higher frequency than laboratories without such policies (P , .001; 48% versus 36% of reviewed cases). Laboratories with review policies also tended to review breast cases at a higher rate than those that did not (P 5 .003; median rate of 1.2% versus 0.5% of all cases). No such statistical significance was found with other organ systems.

COMMENT

To our knowledge, this is the first multi-institutional study to measure the extent and characteristics of pathology case review before case sign-out. The median laboratory reviewed 8.2% of cases with an aggregate rate of 6.6%. One other publication has attempted to measure the rate of case review before sign-out. Renshaw and Gould (13) measured their institutional rate at 13%.

In our study, the range of the rate of case review was wide, laboratories in the 10th and 90th percentiles reviewed 2.0% and 17.1% of cases. This, however, likely represents a low estimate of the rate of intradepartmental consultation. Because this study only captures documented reviews, we can speculate that a number of cases are routinely shown to other pathologists but are not documented and, therefore, are not captured by a retrospective case review. In a previous Q-Probes study (14) of expert extradepartmental consultation, a median rate of 0.5% was recorded with a range of 0% to 2.0%. In that study, expert extradepartmental consultations only represented consultations that were initiated by the pathologists and did not include cases that are seen at other institutions after sign-out because the patient seeks another opinion or goes to another institution for definitive treatment. The rate described in this report represents the prevalent rate of documented cases reviewed internally before sign-out.

Several quality-assurance guidelines have proposed a 10% review rate as a standard, without demonstrating the effectiveness of this specific rate of review. (15,16) These mandated quality-assurance reviews are typically performed after cases are signed out and are used to evaluate error rates of an individual or a program. (17) So they differ in timing--after sign-out versus before sign-out--and in intention--error detection versus error prevention--from the double-review practice examined in our study. Several studies have hypothesized that review of cases before sign-out reduces error and amended report rates. (13,18,19) Our study did not collect error or amended report rates, so it cannot speak to that hypothesis. If potential errors can be identified before reports are issued, this preventative measure would indeed be an important quality measure. It is surprising to the authors of this study that the effectiveness of this procedure has not been tested. The appropriate rate and pattern of such reviews has not been empirically determined. Our study at least offers a picture of the extent of the practice at present.

Regarding patterns of review, we determined what types of cases were reviewed by study participants. The top 4 organ systems reviewed were gastrointestinal tract (20%), breast (16%), skin (13%), and the female genital tract (10%). Although the proportion of cases seen in a particular laboratory varies depending on the practice setting, GI, skin, and female genital tract specimens tend to be among the highest proportion of cases in most North American practice settings. In general practice, breast specimens tend to account for a small proportion of cases but are probably reviewed at a much higher level than other types of specimens because of the clinical implication of breast biopsy results. This is borne out in our statistical analysis where breast cases were reviewed at a higher median rate (1.2%) in laboratories with a review policy than in laboratories that did not have such a policy (0.5%). With respect to disease type, malignancy accounted for 45% of all reviewed cases and far exceeded any other category by a margin of 3 to 1. Selection of known malignancies for review is easy, but this review criterion leaves aside the issue of detecting cases of neoplasms that potentially could have been missed. These may have been misplaced in the second-most frequent category of benign neoplasm/hyperplasia/atypia. False-negative cases indeed represent the highest proportion (73% and 61%) of cases in 2 reports of legal verdict and settlements against pathologists. (20,21) Some laboratories have resorted to reviewing all potentially problematic specimen types, such as brain biopsies, breast biopsies, pigmented skin lesions, and thyroid fine-needle aspirations or thyroid biopsies. The efficacy of focused reviews is just beginning to be investigated. (22,23)

Among our study's participants, the reason given for review of cases was the difficulty of diagnosis in 46% of cases and a required audit per departmental policy in 43% of cases. It is reassuring that nearly one-half of the cases would have been reviewed even if a policy did not exist. The existence of a policy, however, appears to prompt pathologists to have their cases reviewed by another pathologist: departments that had a policy for review of cases had higher review rates (9.6% versus 6.5%) and a higher percentage of the cases that they reviewed were malignant (48% versus 36%). So the observation made above, about review for malignancy of likely positives versus potentially false-negative cases holds here as well. The levels of review measured in this study seem to be workable. Targeted reviews with subspecialty knowledge appear to be beneficial, workable, and effective. (23,24) A recent publication by Raab et al (23) suggests that a focused review of cases detects errors at a higher level than a random review. If true, then the selection of cases to be reviewed may be at least as important as the rate of cases reviewed before sign-out.

In conclusion, routine case reviews before sign-out occurs at a median rate of 8.2%. The optimal level and case mix for review before sign-out remains unknown and deserves investigation to determine the best strategies for analytic error prevention in surgical pathology.

References

(1.) Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999.

(2.) Sirota RL. The Institute of Medicine's report on medical errors: implications for pathology. Arch Pathol Lab Med. 2000;124(11):1674-1678.

(3.) Fisher B. U of C Hospital sued for error that resulted in removal of breast. Chicago Sun Times. May 11, 2005.

(4.) Goodman E. When doctors make mistakes. Good Housekeeping. April 2001.

(5.) Parker-Pope T. Risk of error may justify second opinion on pathology reports. Wall Street Journal. April 13, 2001:1.

(6.) Parker-Pope T. Why it's hard to get a second opinion (and how to make sure you get one). Wall Street Journal. February 1, 2005:1.

(7.) Renshaw AA, Pinnar NE, Jiroutek MR, Young ML. Blinded review as a method for quality improvement in surgical pathology. Arch Pathol Lab Med. 2002;126(8):961-963.

(8.) Westra WH, Kronz JD, Eisele DW. The impact of second opinion surgical pathology on the practice of head and neck surgery: a decade experience at a large referral hospital. Head Neck. 2002;24(7):684-693.

(9.) Hahm GK, Niemann TH, Luca JG, Frankel WL. The value of second opinion in gastrointestinal and liver pathology. Arch Pathol Lab Med. 2001;125(6):736-739.

(10.) Santoso JT, Coleman RL, Voet RL, Bernstein SG, Lifshitz S, Miller D. Pathology slide review in gynecologic oncology. Obstet Gynecol. 1998;91(5, pt 1):730-734.

(11.) Kronz JD, Westra WH, Epstein J. Mandatory second opinion surgical pathology at a large referral hospital. Cancer. 1999;86(11):2426-2435.

(12.) Bachner P, Howanitz P. Using Q-Probes to improve the quality of laboratory medicine: a quality improvement program of the College of American Pathologists. Qual Assur Health Care. 1991;3(3):167-177.

(13.) Renshaw AA, Gould EW. Measuring the value of review of pathology material by a second pathologist. Am J Clin Pathol. 2006;125(5):737-739.

(14.) Azam M, Nakhleh RE. Surgical pathology extradepartmental consultation practices: a College of American Pathologists' Q-Probes study of 2428 consultations from 180 laboratories. Arch Pathol Lab Med. 2002;126(4):405-412.

(15.) Clinical Laboratory Improvement Amendments of 1988; final rule (42 CFR Part 405, et al). Fed Regist. 1992;57:7001-7186.

(16.) Association of Directors of Anatomic and Surgical Pathology. Recommendations on quality control and quality assurance in anatomic pathology. Am J Surg Pathol. 1991;15(10):1007-1009.

(17.) Nakhleh RE, Fitzgibbons PL, eds. Quality Management in Anatomic Pathology: Promoting Patient Safety Through Systems Improvement and Error Reduction. Northfield, IL: College of American Pathologists; 2005.

(18.) Nakhleh RE, Zarbo RJ. Amended reports in surgical pathology and implications for diagnostic error detection and avoidance: a College of American Pathologists' Q-Probes study of 1 667 547 accessioned cases in 359 laboratories. Arch Pathol Lab Med. 1998;22(4):303-309.

(19.) Novis D. Routine review of surgical pathology cases as a method by which to reduce diagnostic errors in a community hospital. Pathol Case Rev. 2005; 10(2):63-67.

(20.) Kornstein MJ, Byrne SP. The medicolegal aspect of error in pathology: a search of jury verdicts and settlements. Arch Pathol Lab Med. 2007;131(4):615-618.

(21.) Troxel DB. Medicolegal aspects of error in pathology. Arch Pathol Lab Med. 2006;130(5):617-619.

(22.) Zarbo RJ, Meier FA, Raab SS. Error detection in anatomic pathology. Arch Pathol Lab Med. 2005;129(10):1237-1245.

(23.) Raab SS, Grysbicki DM, Mahood LK, Parwani AV, Kwan SF, Rao UN. Effectiveness of random and focused review in detecting surgical pathology error. Am J Clin Pathol. 2008;130(6):905-912.

(24.) Meier FA, Varney RC, D'angelo R, Gandhi A, Zarbo RJ. A taxonomy of amended reports assesses impact of Henry Ford Production System: a lean quality initiative in surgical pathology. Mod Pathol. 2009;22(suppl 1):365A.

Raouf E. Nakhleh, MD; Leonas G. Bekeris, MD; Rhona J. Souers, MS; Frederick A. Meier, MD; Joseph A. Tworek, MD

Accepted for publication July 1, 2009.

From the Department of Pathology, Mayo Clinic Florida, Jacksonville, Illinois (Dr Nakhleh); the Department of Pathology, Phoenixville Hospital, Phoenixville, Pennsylvania (Dr Bekeris); the Department of Statistics, College of American Pathologists, Northfield, Illinois (M. Souers); the Department of Pathology, Henry Ford Health System, Detroit, Michigan (Dr Meier); and the Department of Pathology, St Joseph's Mercy Hospital, Ann Arbor, Michigan (Dr Tworek).

The authors have no relevant financial interest in the products or companies described in this article.

Reprints: Raouf E. Nakhleh, MD, Department of Pathology, Mayo Clinic Florida, 4500 San Pablo Rd, Jacksonville, FL 32224 (e-mail: nakhlehroauf@mayo.edu).
Table 1. Institution Demographics

                                     Institutions,    Institutions,
           Parameters                     No.               %

Institution type, n = 41
  Voluntary, nonprofit                          27              65.9
  Department of Defense                          3               7.3
  County hospital                                2               4.9
  Governmental, nonfederal
    university hospital                          2               4.9
  State acute hospital                           2               4.9
  Other, governmental, federal                   1               2.4
  Other, governmental, nonfederal                1               2.4
  Private, independent laboratory                1               2.4
  Proprietary hospital                           1               2.4
  Veterans hospital                              1               2.4

Occupied bed size, n = 36
  0-150                                         12              33.3
  151-300                                       14              38.9
  301-450                                        5              13.9
  451-600                                        3               8.3
  >600                                           2               5.6

Institution location, n = 41
  City                                          20              48.8
  Suburban                                      11              26.8
  Rural                                          7              17.1
  Federal installation laboratory                3               7.3

Governmental affiliation, n = 41
  Nongovernmental                               29              70.7
  Governmental, nonfederal                       7              17.1
  Governmental, federal                          5              12.2

Table 2. Distribution for the Number of Surgical Specimens Accessioned
in 2007 and Number of Full-Time Equivalent (FTE) Pathologists Who
Sign-Out Surgical Pathology Specimens

                                          All Institutions, Percentiles
Parameters

                                                     50th
                                           10th    (Median)    90th

Surgical specimens accessioned
  in 2007 (n = 44), No.                    5239     10 422    23 698
Pathologist FTEs who sign-out surgical
  pathology specimens (n = 45), No.           2          4         9

Table 3. Practice Characteristics That May Be Related
to Surgical Pathology Case Reviews

                                                   Review,    Review,
Parameter                                            No.         %

Most surgical pathology cases are signed out by
  General surgical pathologists                      37        82.2
  Surgical pathologists with
    subspecialty interests                            8        17.8

Policy regarding review by second pathologist
    before sign-out?
  Yes                                                32        71.1
  No                                                 13        28.9

If a policy, what is minimum percentage
    targeted for review? (a)
  1                                                   1         9.1
  2                                                   1         9.1
  7                                                   1         9.1
  10                                                  7        63.6
  95                                                  1         9.1
  There is no target for case review                 24        100.0

(a) n 5 35; 11 listed 1, 2, 7, 10 and 95% as targets and 24 had no
target for case review.

Table 4. Distribution of the Percentage of Cases Reviewed Before
Sign-Out

                                       All Institution Percentiles

                                               50th
Distribution                 10th    25th    (median)   75th    90th

Surgical pathology cases
  reviewed by a second
  pathologist  before
  sign-out (n 5 45), %       2.0     5.5       8.2      12.4    17.1

Table 5. Distributions for the Number of Cases
Reviewed Before Sign-Out and the Total Number of
Cases Examined During the Study Period

                                      All Institution Percentiles

                                                50th
Distribution                          10th    (median)    90th

Surgical pathology cases reviewed
  before sign-out (n 5 45), No.       13.0      30.0      30.0
Total surgical pathology cases
  examined to obtain 30 reviewed
  cases (n 5 45), No.                175.0     368.0     838.0

Table 6. The Frequency of Organ Systems Listed in
Cases Reviewed Before Sign-Out

Organ System, n = 1183             Listed, No.     Listed, %

Gastrointestinal tract                 242            20.5
Breast                                 189            16.0
Skin                                   150            12.7
Genital tract, F                       118            10.0
Genital tract, M                        79            6.7
Pulmonary system                        72            6.1
Urinary tract                           57            4.8
Head and neck                           49            4.1
Endocrine gland                         41            3.5
Lymph node, spleen                      33            2.8
Soft tissue, retroperitoneum            30            2.5
Liver                                   26            2.2
Bone and joints                         19            1.6
Pleura, pericardium, peritoneum         16            1.4
Bone marrow                             12            1.0
Neuropathology                          7             0.6
Placenta, product of conception         6             0.5
Cardiovascular                          2             0.2
Mediastinum, thymus                     1             0.1
Other                                   34            2.9

Table 7. The Frequency of Disease Type, Number of
Additional Pathologists Consulted, and Reason for
Review of Cases Reviewed Before Sign-Out

                                               Reviewed,    Reviewed,
Parameter                                            No.            %

Primary disease type, n = 1169
  Malignant neoplasm                                 529         45.3
  Benign proliferation/hyperplasia/ atypia           183         15.7
  Inflammatory or immunologic process                139         11.9
  Benign neoplasm                                    109          9.3
  Premalignant neoplasm (e.g., carcinoma
    in situ)                                          86          7.4
  Low malignant potential/borderline
    neoplasm                                          33          2.8
  Infection                                           18          1.5
  Therapy-induced changes                              9          0.8
  Malformation                                         5          0.4
  Congenital or hereditary disease                     2          0.2
  Metabolic disease                                    1          0.1
  Other                                               45          3.8
  Unknown                                             10          0.9

Pathologists consulted (n = 1183), No.
  1                                                  919         77.7
  2                                                  154         13.0
  3                                                   66          5.6
  4                                                   17          1.4
  5                                                    5          0.4
  6                                                   13          1.1
  7                                                    7          0.6
  9                                                    2          0.2

Reason for review, n = 1183
  Difficult diagnosis                                546         46.2
  Required audit as per departmental policy           509           43
  Random case review for quality assurance
    purpose                                            27          2.3
  Other                                               101          8.5
Gale Copyright: Copyright 2010 Gale, Cengage Learning. All rights reserved.