Analysis of pathology department web sites and practical recommendations.
Abstract: Context.--There are numerous customers for pathology departmental Web sites, including pathology department staff, clinical staff, residency applicants, job seekers, and other individuals outside the department seeking department information. Despite the increasing importance of departmental Web sites as a means of distributing information, no analysis has been done to date of the content and usage of pathology department Web sites.

Objective.--In this study, we analyzed pathology department Web sites to examine the elements present on each site and to evaluate the use of search technology on these sites. Further, we examined the usage patterns of our own departmental Internet and intranet Web sites to better understand the users of pathology Web sites.

Design.--We reviewed selected departmental pathology Web sites and analyzed their content and functionality. Our institution's departmental pathology Web sites were modified to enable detailed information to be stored regarding users and usage patterns, and that information was analyzed.

Results.--We demonstrate considerable heterogeneity in departmental Web sites with many sites lacking basic content and search features. In addition, we demonstrate that increasing the traffic of a department's informational Web sites may result in reduced phone inquiries to the laboratory. We propose recommendations for pathology department Web sites to maximize promotion of a department's mission.

Conclusions.--A departmental pathology Web site is an essential communication tool for all pathology departments, and attention to the users and content of the site can have operational impact.

(Arch Pathol Lab Med. 2008;132:1423-1427)
Article Type: Company overview
Subject: Internet/Web search services (Service enhancement)
Employment services
Online services
Internet
Medical research
Medicine, Experimental
Job hunting
Authors: Nero, Christopher
Dighe, Anand S.
Pub Date: 09/01/2008
Publication: Name: Archives of Pathology & Laboratory Medicine Publisher: College of American Pathologists Audience: Academic; Professional Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2008 College of American Pathologists ISSN: 1543-2165
Issue: Date: Sept, 2008 Source Volume: 132 Source Issue: 9
Topic: Event Code: 360 Services information Computer Subject: Online employment search service; Internet; Internet search software; Internet/Web search service; Text search and retrieval software
Product: Product Code: 8000200 Medical Research; 9105220 Health Research Programs; 8000240 Epilepsy & Muscle Disease R&D NAICS Code: 54171 Research and Development in the Physical, Engineering, and Life Sciences; 92312 Administration of Public Health Programs
Accession Number: 230247131
Full Text: The major product of any pathology department is information. Pathology departments have become adept in their role as information brokers for the hospital, and pathology data play a crucial role in a wide variety of operational and medical decisions. Despite this role of handling information generated by the department, there is often minimal emphasis placed on providing information about the department and its services. However, pathology department Web sites are likely to be an increasingly dominant way that residency applicants, (1-3) clinical staff, and job seekers assess a department, and thus, attention to the content and usability of pathology departmental Web sites is increasingly important.

The ease of obtaining information via the World Wide Web has changed the way individuals obtain accurate and up-to-date medical information. Internet access worldwide is currently at more than 1 billion users. Academic medical centers provide a wide variety of information on their departmental Web sites. Pathology department Web sites typically include information regarding laboratory testing, residency programs, research programs, schedules, and departmental policies. However, the mere creation of online content does not assure that visitors will be able to find and retrieve the information they seek. In the past decade, Internet search engines have dramatically improved the process of finding information, and a search is now a central feature of the online experience. (4,5)

Prior reports focusing on pathology Web sites have tended to examine the general quality and utility of pathology-related sites on the Internet. (6,7) To our knowledge, no prior survey of pathology department Web sites has been reported. We surveyed 20 pathology departmental Web sites to examine the content and functionality of those sites. The goal was to document the commonalities among pathology Web sites and to provide a listing of features and functionality to inform present and future builders of such sites. We also examined the usage and search patterns of our departmental Internet and intranet Web sites to gain insight regarding the users of pathology Web sites. We demonstrate the utility of incorporating site-specific search technologies into a departmental Web site. We further show that increasing the use of pathology Web sites by clinicians can improve operational aspects of a pathology department. Finally, we provide a listing of what we regard as essential features for all pathology departmental sites and provide recommendations for Web site implementations.

MATERIALS AND METHODS

Setting

The Massachusetts General Hospital (MGH) is an 898-bed, academic medical center in Boston, Mass. The MGH Pathology Service supports all of the inpatient medical, surgical, pediatric, and obstetric services of the hospital as well as outpatient practices extending into the greater Boston community. The Pathology Service comprises the anatomic pathology service as well as clinical laboratories, including the core laboratory (chemistry-hematology), microbiology laboratory, blood transfusion services, and various specialty laboratories.

Analysis of Web Sites

The 20 Web sites that were analyzed were identified by selecting pathology departments associated with hospitals listed in "America's Best Hospitals 2006 Specialty: Cancer" in the U.S. News & World Report. (8) We examined the Web sites after finding their departmental Internet Web site using Google (www.google.com). The analysis was performed in January through March 2007. The content categories were derived by performing a preliminary assessment of various pathology department Web sites and creating a list of content areas that would subsequently be assessed systematically for each surveyed site. A Web site was judged to have research information present if a description of the research program and facilities were present. The search functionality of the site was judged to be nonfunctional if an error message appeared (eg, "page not found") or if no results were obtained for common search terms, such as "residency" or the name of a staff member. In the latter instance, it is likely that the search feature was not implemented properly. The first review was conducted by one of the authors, and then, within 2 weeks, a second review (blinded to the results of the first review) was conducted by the other author. Discrepancies were resolved by examining any discrepant results together and reaching a consensus.

Analysis of Usage and Use Patterns of MGH Web Sites

Visitor and Web Site Usage Information.--Web site usage statistics were generated using tools provided by WebTrends (Portland, Ore) and StatCounter (Dublin, Ireland). To reduce uninformative visits, unique visitors were only tallied when they remained on the site for at least 15 seconds. For analysis of our external (Internet) Web site, we studied the usage patterns of 5000 consecutive users during two periods in July 2006 and December 2006. The two sets of results were comparable and only the December 2006 data are shown in this report. For analysis of our intranet Web site, we analyzed 1000 consecutive visitors in December 2006.

Storage of Search Information.--The ability to perform a free text search (eg, user types in text and clicks on the Search button) is supported on all pathology service Web sites. The search results for the MGH Internet and intranet pathology Web sites were provided using search tools from Ultraseek (Sunnyvale, Calif). A detailed description and an explanation of the free text search methodology used with our online laboratory handbook has been previously described. (9) All of our Web pages were constructed such that detailed user information was stored in our database each time a user performed a free text search. The stored data included the user login alias, the type of search (Internet Pathology Service site, intranet Pathology Service site, or Laboratory Handbook), the date and time of activity, the search text (what the user typed in), and a unique session identifier. For intranet users, the login alias and hospital directory were then used to determine the role (eg, nurse, physician, clerical staff) and department of the user. All individual user information was deidentified after the user's role was determined. User information was obtained in accordance with the hospital's online privacy policy. For the analysis of the search using our Internet and intranet Web sites, we analyzed the stored search information from consecutive searches in the period between December 2006 and February 2007.

Survey of Resident Applicants.--Following the completion of the 2007 MGH pathology resident application process, we sent an e-mail to each of the applicants who was interviewed with a link to an anonymous, online survey. The survey was created in house using Macromedia Flash (Adobe Corporation, San Jose, Calif) and a back-end, structured query language database (SQL Server 2000, Microsoft Corporation, Redmond, Wash). The survey questions were determined by our residency program committee.

Data Analysis

Data analysis was performed using Microsoft Access and Excel (Microsoft).

RESULTS

Survey of Pathology Web Sites

We analyzed 20 pathology department Web sites for their content and usability. The major content categories of the sites are listed in Table 1. Most (90%-100%) of the Web sites had a complete listing of staff, research information, information about their residency programs, and clinical contact information. In terms of fundamental Web site features, most sites had an e-mail link for feedback (80%), whereas a minority had attributes such as a privacy policy (30%) or an indication of when the Web pages were last updated (35%). Notably, although most of the sites contained more than 100 different Web pages, only 55% (11/20) of the sites had any search capability built into them. Moreover, the search capabilities of those Web sites that had a search option were often nonfunctional (18%; 2/11) or searched the entire hospital's Web site (36%; 4/11), which resulted in poor search results. Only 5 of the 11 sites that had a search function had site-specific searches that facilitated the rapid location of content within the departmental Web site.

Although pathology Web sites may play an important role within the department, another major role of a pathology departmental Web site is the "marketing" of the department to the outside world. One primary group of outside consumers of departmental information is prospective residency applicants (28% of all external Internet visitors to our institution's site; see Figure 1). The various categories of residency program content and their prevalence in the examined Web sites are presented in Table 2. All 20 Web sites contained residency program material. Most sites (90%-100%) had program descriptions and application instructions. There was more variation in the provision of other supporting information, including benefits (60%), sample schedules (55%), research opportunities for residents (50%), current resident list (40%), city information (30%), and an alumni list (25%). The quality of the residency information varied greatly among the sites. For example, several sites had not been updated in more than 18 months and listed application due dates that were no longer applicable. Others had only a short description of the program but included no information on resident schedules, benefits, or program alumni. At our institution, following the completion of our residency selection process, we send an electronic survey to applicants whom we have interviewed. In the 2007 survey, the response rate was 47% (41/87 applicants), and 97.6% (40/41) of the respondents indicated that online information was useful for obtaining residency information. Survey respondents also indicated that the most helpful sections of our online material were the status of program alumni (present on only 25% of the 20 residency sites surveyed), information regarding benefits (present on 60%), and sample schedules (present on 55%).

Use and Usage Patterns for MGH Web Site

Analysis of External Traffic.--Our pathology department Internet Web site (http://www.massgeneral.org/pathology/) attracts 600 to 1000 unique visitors per day. We analyzed 5000 consecutive visitors in December 2006 to assess the usage of our site. External traffic (visitors from outside the hospital network) comprised 56% of all visitors to the MGH Pathology Service Web site. Web site visitors were primarily from within the United States (87%). Most commonly, visitors arrived at the MGH pathology Web site after an Internet search (75%). The remainder arrived from sites that directly link to the MGH pathology site (eg, the MGH main hospital site, the Partners HealthCare site). As seen in Figure 1, for external visitors, the most highly visited areas of the Web sites were the residency information pages (28%) and the individual staff Web pages (22%). From analysis of the clickstreams (the stored sequence of Web pages clicked through by each visitor), it was evident that approximately one third of the visits to individual staff pages were likely by residency applicants, further underscoring the heavy use of the site by pathology residency applicants. On each staff member's Web page, we provide a link to PubMed that includes a list of that staff member's publications. From analysis of the clickstreams of individual users, we find that many of our users click on the provided PubMed link (32/100 user clickstreams examined).

Analysis of Internal Traffic.--Overall, internal visitors (those within the hospital network) accounted for 44% of Internet site traffic. For our internal traffic, the most frequently visited section of our Internet Web site was the laboratory handbook, accounting for 52% of internal visitors (Figure 1). By examining the login aliases for 200 intranet users, we could determine that the clinical staff (physicians and nursing staff) comprised 84% of laboratory handbook users. The other sections of the Internet Web site that were highly visited included the pathology faculty information pages (14% of internal visitors) and the residency program information pages (12% of internal visitors).

The departmental intranet pages are available to all users within the hospital network. Traffic on the departmental intranet pages was assessed in December 2006 and was approximately 30 to 70 visitors per day with the highly visited areas being resident education (23%), schedules (22%), and policies and procedures (21%). By examining the login aliases for 200 consecutive users, we could determine that 94% of users were from the Department of Pathology.

Analysis of Search.--Search capabilities are supported from all major pages of the MGH Pathology Service Internet Web site. From the home page of the departmental Internet Web site (www.massgeneral.org/pathology), the ability to search the Internet site, the intranet site, or the laboratory handbook site is supported. The search engine for the Pathology Service Internet site (not including the laboratory handbook) is used 20 to 40 times per day. We analyzed the stored search information from 1000 consecutive Internet searches during December 2006 to February 2007; 81% of the searches were performed by individuals outside the hospital's network. This was expected because most of the recurring users of the Internet pathology site are familiar with its content and navigation. As seen in Figure 2, outside searches were most commonly performed to obtain staff information (34% typing in a faculty member's name), residency program information (17% typing in key words associated with the residency program), or test information (15% typing in a test name). Users typically entered 1 to 2 words (91% of searches) in the search box before clicking the Search button. The few intranet searches performed (1-5 per day) were predominantly from pathology employees looking for policies and procedures (61% of a sample of 200 consecutive searches).

Reduction in Laboratory Customer Service Inquiries.--The MGH Pathology Service laboratory handbook is a major source of laboratory information for our clinicians, with more than 300 unique visitors each day and with more than 90% of visitors from within the MGH network. As seen in Figure 1, the laboratory handbook accounts for 52% of all internal Internet visitors to the pathology Web site. Since the launch of the revised online laboratory handbook in March 2005, we have linked the handbook to the various clinical Web sites in the hospital and have communicated information about the handbook's existence in numerous clinical forums. These ongoing marketing efforts have resulted in a steady increase in the number of unique, daily users of the online laboratory handbook (Figure 3). The MGH Pathology Service operates a customer service call center that is the central source of phone information regarding laboratory testing, including testing requirements and test availability. From January 2003 to March 2005 (the time of the launch of the online handbook), call center monthly volume routinely averaged from 7500 to 9000 calls. As seen in Figure 3, starting in March 2005, we observed a reduction in the number of calls to our customer service center, which appears to be related to the increase in the number of daily users of the laboratory handbook. This reduction in call volume has continued to the present day. In addition, because we have linked the laboratory handbook to numerous clinical Web sites and put links to the handbook into the electronic medical record, there has been an increased proportion of use by physicians and nurses compared with clerical staff (63% clinical users in May 2005 vs 84% in December 2006, data not shown).

COMMENT

In the past decade, sophisticated Internet search technology has been developed that permits the rapid and accurate retrieval of a wide variety of content. Web sites serve as an important information outlet for a pathology department. There are numerous internal and external customers for a pathology Web site, including pathology staff, clinicians, prospective researchers, residents, colleagues, referring pathologists, and those seeking continuing education. With the burgeoning use of the Internet, pathology department Web sites are likely to become the dominant method that individuals use to obtain information regarding a pathology department.

Prior studies have shown that residency applicants use the information obtained from program Web sites to help in deciding which residency programs to apply to. (1-3) For external visitors, the residency program section of our Internet Web site was the most frequently visited portion of the site (28% of all external visitors). In a 1999 survey of emergency medicine residency applicants, 71% of respondents either strongly agreed or agreed that they used departmental Web sites to decide where to apply to. (2) In a 2005 survey, 78% of residents reported that the information provided in residency program Web sites influenced their application decisions, and 41% of residents indicated they did not apply to at least one program based on the quality of the residency Web site.1 Important content features for emergency medicine residency Web sites that were identified included curriculum, current resident information, alumni information, board pass rates, faculty, and salary and benefits. (1) In the present survey of pathology Web sites, the majority of the sites did not contain many of the elements reported as important in resident surveys.

[FIGURE 3 OMITTED]

The faculty information pages of our Internet pathology Web site were highly visited, accounting for 22% of all external Web site traffic. In fact, for most faculty in our department, their departmental staff Web page is in the top 3 links retrieved when searching the entire Web with Google (Google Inc, Mountain View, Calif) for their first and last names. Faculty information pages contain a link that takes the user directly to PubMed to provide users with a list of the staff member's publications. From analysis of the clickstreams of individual users, we find that many of our users click on the PubMed link that we provide. Using the PubMed link approach, rather than presenting a list of faculty publications, may be preferable because it requires no maintenance and PubMed will always be up-to-date.

Each hospital's intranet provides the department with a platform to make available a wide variety of information to employees. Policies, schedules, lectures, resident education, and online atlases can all be readily housed on a department's intranet. Through analysis of our Web site statistics, we found that our resident education, schedule, and policy and procedure areas of our intranet sites all had high traffic. It is important for a pathology department to regularly examine its Web visitor statistics for all of its sites because that analysis provides a useful assessment of which parts of the Web site its customers use and value.

In this report, we demonstrate that the presence of an accurate, up-to-date laboratory handbook may reduce the number of phone calls to the laboratory customer service center. The integration of the laboratory handbook and customer service center may represent an important strategy to efficiently provide and maintain clinical information. At present at our institution, we are working to create a single working group responsible for all customer service interactions with clinicians. In addition to staffing the customer service center, this group will be expected to play a major role in maintaining and expanding our laboratory handbook.

The ever-increasing amount of material that pathology departments are making available online heightens the importance of information-finding tools such as the search function. Despite the increasing amount of content on pathology sites, only 55% of the surveyed Web sites had implemented the search functionality. A search option is likely to be especially important for external visitors to a site; external visitors comprised 56% of all visitors to our site. In fact, the search features of our Web site were predominantly used by external visitors, who accounted for 81% of all searches performed. Information finding is greatly improved with a coherent and well-thought-out design that uses Web page design elements, such as templates and style sheets, to enforce consistency. However, once a site gets sufficiently large, there are limitations to simple navigation and design strategies, and thus, a robust search mechanism is highly desirable. Site-specific search engines are readily available (Google Inc; Yahoo Inc, Sunnyvale, Calif; Ultraseek, Santa Clara, Calif; and many others), and the addition of this function to a department's Web pages is straightforward.

By storing and analyzing searches, a database of requested information can be generated, thus arriving at an understanding of how and why users interact with a search engine. (4) We have previously demonstrated (9) that analysis of user searches can be performed to understand and improve the usage of an online laboratory handbook. In the present report, we extend this concept to searches performed on a pathology Web site. A pathology department can improve its Web site through insights gained by collecting information about each user's activity and searches. For example, an abundance of searches for testing information led us to increase the prominence of links and navigation to our laboratory handbook. Another example was the finding that many of our users were searching for point-of-care testing information on our site. Knowledge of these searches prompted us to create point-of-care testing pages and to move some of the key point-of-care documents to our Web sites.

Essential features of a pathology departmental Web site include a feedback link, site-level search functionality, an indication of when each page was last updated, privacy and copyright policies, and the ability to track Web site usage. Content essentials will vary by the customers and goals of a given pathology department, but it is clear from residency applicants that there are high expectations for residency program information and that this information can influence application decisions. (1-3) It is likely that this phenomenon is operant for many areas of the department and increasingly an outsider's first contact with a pathology department will be via its online Web site.

As more sophisticated search techniques begin to provide health care workers with flexible and robust access to testing information, we envision pathology Web sites becoming the primary portals for most inquiries to the department. Clinician queries may be generated when a test is ordered, when results are initially viewed, when results are interpreted, or when a consult is requested. Linkage to pathology Web sites can occur directly from computerized physician order entry applications, hospital clinical information systems, outpatient clinical information systems, and hospital Web sites. A well-visited pathology Web site provides an efficient channel for disseminating information regarding pathology services to clinical staff.

References

(1.) Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ. Aspects of residency programs' Web sites important to student applicants. Acad Emerg Med. 2005;12: 89-92.

(2.) Winters RC, Hendey GW. Do Web sites catch residency applicants? Acad Emerg Med. 1999;6:968-972.

(3.) Embi PJ, Desai S, Cooney TG. Use and utility of Web-based residency program information: a survey of residency applicants. J Med Internet Res. 2003;5: e22.

(4.) Battelle J. The Search: How Google and Its Rivals Rewrote the Rules of Business and Transformed Our Culture. New York, NY: Penguin Group; 2005.

(5.) Brin S, Page L. The anatomy of a large-scale hypertextual Web search engine. Comput Netw ISDN Syst. 1 998;30:107-117.

(6.) Boehm J. Best of the web in pathology: a practical guide to finding specific pathology resources on the Internet [published online ahead of print May 11, 2007]. J Clin Pathol. 2008;61:225-232.

(7.) Talmon G, Abrahams NA. The Internet for pathologists: a simple schema for evaluating pathology-related Web sites and a catalog of sites useful for practicing

pathologists. Arch Pathol Lab Med. 2005;129:742-746.

(8.) Social and Statistical Science Division, RTI International. America's best hospitals: 2006; and Best hospitals 2006 specialty: cancer. Research Triangle Park, NC: U.S. News & World Report; July 17, 2006.

(9.) Blechner M, Kish J, Chadaga V, Dighe AS. Analysis of search in an online clinical laboratory manual. Am J Clin Pathol. 2006;126:208-214.

Christopher Nero, MD; Anand S. Dighe, MD, PhD

Accepted for publication February 14, 2008.

From the Department of Pathology, Massachusetts General Hospital, Boston.

The authors have no relevant financial interest in the products or companies described in this article.

Reprints: Anand S. Dighe, MD, PhD, Department of Pathology, Massachusetts General Hospital, Bigelow Building, Room 510, 55 Fruit St, Boston, MA 02114 (e-mail: asdighe@partners.org).
Table 1. Features Present on Pathology Department
Web Sites

                                 Feature Present,
Feature                          % (No.) *

General
  Department description         100 (20)
  Faculty                        100 (20)
  Residency information          100 (20)
  Research                        95 (19)
  Department history              20 (4)
Administrative
  Policies/procedures             75 (15)
  Calendar                        55 (11)
  Vacancies/job information       40 (8)
  News                            35 (7)
  Organizational chart            10 (2)
Clinical
  Clinical contact information    90 (18)
  Online laboratory handbook      50 (10)
Web site attributes
  Web master feedback             80 (16)
  Search                          55 (11)
  Last updated                    35 (7)
  Privacy policy                  30 (6)

* Total sample size: N = 20.

Table 2. Residency Program Features Present on
Pathology Department Web Sites

                          Feature Present,
Feature                   % (No.) *

Program description       100 (20)
Application information    90 (18)
Benefits                   60 (12)
Sample schedules           55 (11)
Research opportunities     50 (10)
Current residents          40 (8)
City information           30 (6)
Alumni list                25 (5)

* Total sample size: N = 20.

Figure 1. Analysis of Web site traffic to Massachusetts
General Hospital Pathology Department
Web sites. Visitors within the hospital
network (internal) are denoted with
black bars, and visitors from outside the hospital
network (external) are denoted with
white bars.

Percentage of Visitors

                                           Internal   External
                                           Visitors   Visitors

Laboratory Handbook                           52         10
Residency Information                         12         28
Faculty Pages                                 14         22
Research Information                           6         15
Educational Conferences                        5          7
Department of Laboratory Descriptions          3          6
Other                                          8         12

Figure 2. Categories of free text searches
employed by Internet users on the Massachusetts
General Hospital Pathology Department
Web site.

                          Percentage of searches

Faculty Name                        34
Test Name                           15
Residency-Related Term              17
Laboratory Name                     11
Subspecialty Name                    7
Other                               16
Gale Copyright: Copyright 2008 Gale, Cengage Learning. All rights reserved.