Physician satisfaction with surgical pathology reports: a 2-year College of American Pathologists Q-Tracks study.
* Context.--There are multiple elements that can be measured to
assess the quality of a surgical pathology laboratory. Overall customer
satisfaction is an excellent "global" measure, because it
highlights the unique insight of laboratory performance from the
Objective.--To measure customer satisfaction with surgical pathology reports.
Design.--This study was based on a subscription Q-Tracks study. Voluntary participants were asked to distribute and collect a minimum of 25 surveys per quarter from their clients. Four parameters were graded, which included overall satisfaction, report turnaround time (TAT), completeness, and style on a scale of 1 (poor) to 5 (excellent). Each laboratory submitted quarterly data to the College of American Pathologists, where the data were tabulated and analyzed. Each laboratory could compare their performance in all 4 measures against the entire cohort or a selected subgroup of laboratories. Overall customer satisfaction with surgical pathology reports and 3 subcategories of report TAT, completeness, and style were the main outcome measures.
Results.--This study ran during 2004 and 2005, with 41 and 33 participant laboratories, respectively. The median score for overall satisfaction, TAT, completeness, and style were 4.57, 4.31, 4.62, and 4.64 in 2004, and 4.64, 4.56, 4.65, and 4.68 in 2005, respectively. Most laboratories reported results for 4 quarters or fewer. There was no statistically significant change in overall satisfaction over time.
Conclusions.--Overall satisfaction scores for surgical pathology reports as well as satisfaction with report TAT, completeness, and style were high. ReportTAT received the lowest scores of all parameters.
(Arch Pathol Lab Med. 2008;132:1719-1722)
Pathological laboratories (Standards)
Patient satisfaction (Surveys)
Communication in medicine (Management)
Communication in medicine (Standards)
Nakhleh, Raouf E.
Ruby, Stephen G.
|Publication:||Name: Archives of Pathology & Laboratory Medicine Publisher: College of American Pathologists Audience: Academic; Professional Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2008 College of American Pathologists ISSN: 1543-2165|
|Issue:||Date: Nov, 2008 Source Volume: 132 Source Issue: 11|
|Topic:||Event Code: 200 Management dynamics; 350 Product standards, safety, & recalls Computer Subject: Company business management|
|Geographic:||Geographic Scope: United States Geographic Code: 1USA United States|
The surgical pathology report (either printed or via electronic
distribution) is the final product of surgical pathology laboratories.
Elements necessary for quality in surgical pathology reports include
accuracy, timeliness, and completeness. (1) Each of these elements can,
and should, be measured separately to assess the quality of the surgical
pathology laboratory and its product. However, satisfaction with a
product is often based on elements that are unique to the customer, such
as expectations and individual perceptions. These factors are not as
easily addressed because of the multiplicity and uniqueness of these
expectations, which can vary with various physician customers. (2,3)
Satisfaction surveys are simple tools that can provide a general gauge
of perceptions and expectations of physician customers. Laboratory
accreditation agencies now require that providers of laboratory services
survey the needs, expectations, and level of satisfaction of the
physicians and patients that they serve. (4) In this article, we
describe our experience with a Q-Tracks study designed to determine
physician satisfaction with surgical pathology reports.
MATERIALS AND METHODS
This study was offered and conducted as a Q-Tracks study, the basic mechanism of which has been previously described. (5) Briefly, Q-Tracks studies are subscription-based quality assurance studies that have the advantage of tracking a particular monitor over a prolonged period of time (a minimum of 1 year) with ongoing peer group comparison.
A standardized physician satisfaction survey focused on surgical pathology reports was provided to laboratory participants. The College of American Pathologists (CAP) distributed the studies to the laboratories by standard US postal service. Each laboratory was asked to collect a minimum of 25 surveys but no more than 99 surveys per quarter from physicians who submitted tissue to surgical pathology from either an inpatient or an outpatient setting. Each laboratory independently determined the number of surveys needed to send out in order to receive a minimum of 25 replies. They also independently determined the best method to send the surveys to their physician customers. Some laboratories elected to send a second reminder to achieve an adequate sample. At the end of the quarter, the surveys were returned to each laboratory, where the data were tabulated onto an input form, which was then sent to the CAP for analysis and peer group comparison. Any free text comments written on the surveys were for the benefit of the individual laboratory only and were not centrally collected or analyzed. Laboratories were also surveyed for general items regarding institutional policies and practices related to surgical pathology reports. Laboratories were given the opportunity to update this information on a quarterly basis.
The survey asked physicians who had sent specimens to the laboratory for analysis to rate the services (5 = excellent to 1 = poor) for report turnaround time (TAT), report content, report style, and overall satisfaction. In the survey instructions, the following criteria were included: for report content, 5 means that the report contained all information required. For report style, 5 was defined as a report that was clearly written and easily understood on the first reading. The surveyed physicians were also given the opportunity to make free text comments or suggestions for improvement on report TAT and report content. Physicians could also comment on the laboratory's service and suggest improvements.
The data were tabulated and analyzed by the biostatistics department of the CAP. Each laboratory was able to compare its performance against the entire study population. In addition, each laboratory was also able to define specific cohort comparison groups based on a number of institutional factors, such as the institutional type (governmental vs nongovernmental), institutional location (city, rural, suburban), the existence of research facilities, the existence of training programs, the type of outpatient/outreach specimens, the administrative services provided by the laboratory, the level of service, the hospital complexity, and whether the hospital was a teaching institution. The data were examined for association between physician satisfaction and hospital factors, policies, practices, and various laboratory and institutional characteristics. The data were also examined for trends over time.
A level of .05 was used for statistical significance. All statistical analyses were performed using SAS v9.1 (SAS Institute Inc, Cary, NC).
This Q-Tracks study ran for 2 years in 2004 and 2005. Forty-one institutions participated in the first year, and 33 during the second year. Their institutional characteristics are listed in Table 1. For the years 2004 and 2005, respectively, 89.2% and 93.8% of participant institutions had been inspected by the CAP Laboratory Accreditation Program within the preceding 2 years; 62.2% and 59.4% had been inspected by The Joint Commission; 34.3% and 42.9% described themselves as teaching hospitals; and 27.0% and 31.3% had pathology residency programs.
Over the course of 2 years, a total of 69 laboratories submitted data. Table 2 summarizes the number of quarters that each laboratory submitted data. Four sites with multiple quarter submissions had data gaps of 1 quarter.
The overall report satisfaction scores for 2004 and 2005 are listed in Table 3. Each quarter's satisfaction scores are also listed in Table 3, with percentile distribution of institutional scores. Table 4 contains the cumulative physician satisfaction scores and percentile distribution for TAT, report content, and report style. There was no association of customer satisfaction with any institutional characteristic and the level of satisfaction.
Table 5 shows results of a questionnaire regarding various institutional practices and policies. There was no significant association with any of the practices or policies.
A mixed linear regression model analyzing the quarterly data for the 41 sites that submitted data for more than 1 quarter was performed. There was no significant increase (P = .22) in overall report satisfaction based on the length of participation.
A high level of customer satisfaction is critical for business success. In a competitive environment, customer satisfaction is essential for sustained profitability and viability. Customer satisfaction surveys are important research tools to identify weaknesses and strengths of a product or service and to probe a customer's needs. In surgical pathology, customer satisfaction assessment and customer satisfaction surveys are relatively new phenomena. Accrediting agencies, such as The Joint Commission and the CAP, are requiring that more effective physician to physician communication be implemented, including satisfaction assessment. (6,7) The CAP Laboratory General Checklist and The Joint Commission also require that physician and patient satisfaction with the laboratory service be assessed at least every 2 years. (4) Several studies have been published regarding satisfaction with multiple aspects of the clinical laboratory, but there have been few studies assessing customer satisfaction with surgical pathology. (8-13)
Overall, customer satisfaction scores in this study were high, leading to the conclusion that in general, surgical pathology reports serve their customers well. Scores in this Q-Tracks study were higher than in the Q-Probes study conducted in 2001. (13) The 2001 Q-Probes study represents a single snapshot picture of customer satisfaction, whereas the current Q-Tracks study examined customer satisfaction during a 2-year period. The reasons for higher scores are likely multifactorial. However, a single event may have had a substantial contribution to the overall satisfaction and report completeness scores. In 2004, the Commission on Cancer, a body of the American College of Surgeons, added a requirement for "Cancer Center" designation that more than 90% of surgical pathology reports of cancer resections should include information as defined by the CAP Cancer Protocols.14,15 As a result, there was a clear definition of appropriate and adequate content for a surgical pathology report for malignancy. This had the effect of defining the expectations for clinicians and pathologists and took away the subjectivity of a reviewer. Standardization of surgical pathology cancer reports likely led to higher levels of satisfaction in general for those laboratories that had already incorporated the use of standardized reports. The use of synoptic reporting in pathology has been reported to be associated with a higher level of customer satisfaction. (16) Although we do not know how many laboratories get specimens from a cancer center, we were able to determine that 81.5% of institutions had an oncology service. In this study, we were able to demonstrate increased use of templates for reporting malignancies (64.1%-68.8%) as well as nonmalignant diagnoses (12.8%-30.3%) in a 2-year span. Unfortunately, the baseline level of template use in 2001 is unknown, but it is assumed to be less than that in 2004. We were also able to demonstrate increased use of other technologies in the 2 years for which the study ran. In 2005, there was a higher percentage of institutions using electronic report distribution methods, and a higher percentage of laboratories included images in reports compared with 2004. In the future, the use of these and other technologies has the potential to further enhance customer satisfaction.
The strength of the Q-Tracks and Q-Probes programs is the ability of each laboratory to compare its performance against that of its peers. Each laboratory is able to determine its performance against other laboratories in this study for overall satisfaction and 3 subcategories of report turnaround, report completeness, and report style. Therefore, each laboratory can address its specific problems. Each laboratory also had the opportunity to receive handwritten suggestions and criticism directly from its own clinicians, which may also help improve that laboratory's satisfaction scores.
Report TAT scored the lowest of the 3 parameters surveyed. Although actual TATs are objective, satisfaction with TAT is usually a subjective matter of expectation. If there is an expectation for a 1-day TAT (regardless of any basis for that expectation), the clinician will be unhappy with a 2-day TAT. There are industry standards that should be communicated to clinicians in order to build realistic local expectations. This also highlights the need for ongoing communication regarding specific cases where a possible delay may occur. A pathologist's discussion with the clinician concerning the need to perform special studies or other causes for longer TATs can significantly reduce dissatisfaction by the clinician in regard to TAT and can also enhance the pathologist's role as a consultant. Clinicians more than ever are under pressure from patients to provide timely answers. A clinician that is able to convey or elaborate on the process and provide a reasonable expectation of TAT to his patient is likely to be more satisfied with surgical pathology performance than one who has not received such information. However, it is even better when the clinician is able to convey the process and reasonable expectations for TAT to the patient prior to surgery or other procedures.
In several other Q-Tracks studies, participation in a study for a prolonged period (>1 year) typically led to improvement in performance. (17,18) Speculation as to the reasons for improvement has included the Hawthorne effect (improvement as a result of focusing on a problem) and the possible adoption of best practices. For any one particular monitor, other factors may also play a role. In this Q-Tracks study, we could not demonstrate a measurably significant improvement in physician satisfaction with surgical pathology reports. Most participants conducted the survey for 4 quarters or fewer. In previous studies, improvement could be demonstrated clearly with 2 years of participation, and improvement continued with 4 to 5 years of participation. The short duration of participation could account for our lack of ability to demonstrate any significant improvement. Another factor may have been the relatively low number of participants in this study compared with other Q-Tracks studies. A third possibility for lack of improvement may be the nature of customer satisfaction. For most other studies, the indicator measures the efficacy of a task or process, which tends to improve with added attention. Customer satisfaction monitoring does not directly assess any task or function, and therefore, individuals involved in the process are unlikely to be more careful or change what they do unless specific interventions are initiated. Last, all of the satisfaction scores were initially in the upper range of the evaluation scale, which limits the ability to detect statistically significant improvements. This may be due to satisfaction only due to familiarity by the clinician with the product (ie, report) of the laboratory rather than the true satisfaction with reports, when compared with other laboratories. Future studies may include other, "optimal" reports for comparative purposes in order for the clinicians to have a benchmark to which they can compare their local pathology laboratory reports. Alternatively, more rigorous descriptions of what constitutes an "average" versus "above average" report may provide a better distribution of report satisfaction across the evaluation numeric range.
In summary, this Q-Track study demonstrated the ability for laboratories to survey their "clients" for satisfaction with their product (ie, report). Only by using an ongoing objective assessment of performance is a laboratory able to formulate a plan for meaningful change, by determining both the positive and negative aspects of their surgical pathology reports. The questionnaire provided in this study examines the most basic aspects of the report. Individual laboratories may elect to perform more detailed surveys in order to determine more specific areas of clinician concerns, where the laboratory's improvement efforts are able to provide the greatest return on investment.
Accepted for publication March 21, 2008.
(1.) Nakhleh RE. What is quality in surgical pathology? J Clin Pathol. 2006;59: 669-672.
(2.) Powsner SM, Costa J, Homer RJ. Clinicians are from Mars and pathologists are from Venus. Arch Pathol Lab Med. 2000;124:1040-1046.
(3.) Ruby SG. Clinician interpretation of pathology reports: confusion or comprehension? [editorial]. Arch Pathol Lab Med. 2000;124:943-944.
(4.) College of American Pathologists. Standard: GEN.20368. Laboratory general checklist, Commission on Laboratory Accreditation, Laboratory Accrediting Program. Available at: http://www.cap.org/apps/docs/laboratory_accreditation/ checklists/laboratory_general_sep07.pdf. Accessed December 5, 2007.
(5.) Zarbo RJ, Jones BA, Friedberg RC, et al. Q-Tracks: a College of American Pathologists program of continuous laboratory monitoring and longitudinal performance tracking. Arch Pathol Lab Med. 2002;126:1036-1044.
(6.) National Patient Safety Goals, The Joint Commission. Available at: http://www.jointcommission.org/PatientSafety/NationalPatientSafetyGoals/. Accessed December 5, 2007.
(7.) College of American Pathologists. CAP Laboratory Patient Safety Plan. Available at: http://www.cap.org/apps/cap.portaU_nfpb=true&cntvwrPtlt_action Override=%2Fportlets%2FcontentViewer%2Fshow&_windowLabel=cntvwrPtlt&cntvwr Ptlt%7BactionForm.contentReference%7D = patient_ safety%2Flaboratory_patient _safety_plan.html&_state=maxi Accessed December 5, 2007.
(8.) Jones BA, Walsh MK, Ruby SG. Hospital nursing satisfaction with clinical laboratory services: a College of American Pathologists Q-Probes study of 162 institutions. Arch Pathol Lab Med. 2006;130:1756-1 761.
(9.) Howanitz PJ. Physician Satisfaction With Clinical Laboratory Services: Q-Probes Data Analysis and Critique. Northfield, Ill: College of American Pathologists; 2002.
(10.) Miller KA, Dale JC. Physician Satisfaction With Clinical Laboratory Service. Q-Probes (99-03). Northfield, Ill: College of American Pathologists; 1999.
(11.) Steindel SJ, Howanitz PJ. Physician satisfaction and emergency-department laboratory test turnaround time. Arch Pathol Lab Med. 2001;125:863-871.
(12.) Zarbo RJ. Determining customer satisfaction in anatomic pathology. Arch Pathol Lab Med. 2006;130:645-649.
(13.) Zarbo RJ, Nakhleh RE, Walsh M. Customer satisfaction in anatomic pathology: a College of American Pathologist Q-Probes study of 3065 physician survey from 94 laboratories. Arch Pathol Lab Med. 2003;127:23-29.
(14.) College of American Pathologists. Cancer Protocols and Checklists. Available at: http://www.cap.orgapps/cap.portal?_nfpb=true&cntvwrPtlt_actionOverride= %02Fportlets%2FcontentViewer%2Fshow&_windowLabel=cntvwrPtlt&cntvwrPtlt%7 BactionForm.contentReference%7D=cancer_protocols%2Fprotocols_index.html& _state=maximized&_pageLabel=cntvwr. Accessed December 5, 2007.
(15.) American College of Surgeons, Commission on Cancer. Available at: http://www.facs.org/cancer/coc/standards.html.Accessed December 5, 2007.
(16.) Branston LK, Greening S, Newcombe RG, et al. The implementation of guidelines and computerized forms improves the completeness of cancer pathology reporting: the CROPS project: a randomized controlled trial in pathology. Eur J Cancer. 2002;38:764-772.
(17.) Raab SS, Tworek JA, Souers R, Zarbo RJ. The value of monitoring frozen section-permanent section correlation data over time. Arch Pathol Lab Med. 2006;130:337-342.
(18.) Howanitz PJ, Renner SW, Walsh MK. Continuous wristband monitoring over 2 years decreases identification errors: a College of American Pathologists Q-Tracks study. Arch Pathol Lab Med. 2002;126:809-815.
Raouf E. Nakhleh, MD; Rhona Souers, PhD; Stephen G. Ruby, MD, MBA
From the Department of Pathology, Mayo Clinic, Jacksonville, Ill (Dr Nakhleh); the Department of Statistics, College of American Pathologists, Northfield, Ill (Dr Souers); and the Department of Pathology, Palos Community Hospital, Palos Heights, Ill (Dr Ruby).
The authors have no relevant financial interest in the products or companies described in this article.
Reprints: Raouf E. Nakhleh, MD, Department of Pathology, Mayo Clinic Jacksonville, 4500 San Pablo Rd, Jacksonville, FL 32224 (e-mail: Nakhleh.firstname.lastname@example.org).
Table 1. Demographics of Participating Institutions Percentage of Participant Institutions 2004 (n = 41) 2005 (n = 33) Institutional type Voluntary, nonprofit 70.3 62.5 Governmental, federal 10.8 9.4 University hospital, governmental 2.7 3.1 System/integrated delivery network 2.7 0.0 University hospital 2.7 Group practice 2.7 9.4 Private, independent laboratory 2.7 6.3 Proprietary hospital 2.7 Other 2.7 3.1 Children's hospital 0.0 3.1 State, county, city hospital 0.0 3.1 Institutional location City 73 46.9 Suburban 18.9 31.3 Rural 5.4 15.6 Federal installation 2.7 3.1 Other 3.1 Occupied bed size 0-150 23.5 29.6 151-300 52.9 44.4 301-450 14.7 18.5 451-600 7.4 >600 8.8 Table 2. The Number of Quarters That Laboratories Submitted Data No. of Quarters With Data Frequency Percentage 1 28 40.6 2 7 10.1 3 12 17.4 4 18 26.1 5 0 0 6 1 1.4 7 2 2.9 8 1 1.4 Table 3. Overall Physician Satisfaction Scores (5 = Excellent, 1 = Poor) All Institutions' Percentile Distribution n 10th 25th 50th 75th 90th Cumulative 2004 41 4.15 4.39 4.57 4.68 4.82 First quarter 23 4.40 4.48 4.63 4.77 4.84 Second quarter 27 4.04 4.31 4.52 4.71 4.80 Third quarter 18 3.96 4.54 4.61 4.80 4.91 Fourth quarter 16 4.06 4.44 4.67 4.88 4.95 Cumulative 2005 33 4.20 4.54 4.64 4.76 4.82 First quarter 22 4.24 4.54 4.63 4.75 4.88 Second quarter 25 4.00 4.44 4.64 4.82 4.86 Third quarter 20 4.24 4.49 4.58 4.78 4.84 Fourth quarter 21 4.17 4.45 4.68 4.73 4.83 Table 4. Cumulative Distribution of Physician Satisfaction Scores for Turnaround Time, Report Content, and Report Style (5 = Excellent, 1 = Poor) All Institutions' Percentile Distribut n 10th 25th 50th 75th 90th Report turnaround 41 3.88 4.18 4.31 4.54 4.72 time cumulative 2004 scores * Report content 41 4.32 4.45 4.62 4.83 4.86 cumulative 2004 scores Report style 41 4.37 4.47 4.64 4.76 4.85 cumulative 2004 scores Report turnaround 33 4.01 4.34 4.56 4.66 4.74 time cumulative 2005 scores * Report content 33 4.30 4.46 4.65 4.74 4.81 cumulative 2005 scores Report style 33 4.24 4.47 4.68 4.81 4.84 cumulative 2005 scores * Turnaround time scores are significantly lower than overall satisfaction, report content, and report style scores (P < .001). Table 5. Summary of Institutional Practices and Policies 2004 2005 (n = 39) (n = 33) What is the primary mechanism of surgical pathology report delivery? Paper 35.9 18.2 Electronic computer-based LIS/HIS * 12.8 21.2 Both paper and electronic 43.6 45.5 Facsimile 7.7 15.2 Are images placed in pathology reports? Yes 10.3 15.2 No 89.7 84.8 Are microscopic descriptions routinely used? Yes 59.0 72.7 No 41.0 27.3 Do you report from formatted templates for cases of malignancy? Yes 64.1 68.8 No 35.9 31.3 Do you report from formatted templates for cases of nonmalignancy? Yes 12.8 30.3 No 87.2 69.7 Does the laboratory have specific turnaround time goals for surgical pathology reports? ([dagger]) Yes 92.3 No 7.7 Have the turnaround time goals been communicated to the physician customers? ([dagger]) Yes 61.1 No 38.9 Do you issue a preliminary or partial report before issuing a final report? ([dagger]) Yes 41.0 No 59.0 Has your laboratory instituted any new policies or procedures as a result of monitoring physician satisfaction with surgical pathology reports? ([dagger]) Yes 30.8 No 69.2 * LIS/HIS indicates laboratory information system/hospital information system. ([dagger]) Not asked in 2005.
|Gale Copyright:||Copyright 2008 Gale, Cengage Learning. All rights reserved.|