Laboratory performance in Neonatal Bilirubin testing using commutable specimens: a progress report on a College of American Pathologists study.
* Context.--In 2003 the Chemistry Resource Committee of the College
of American Pathologists introduced a commutable specimen in the
Neonatal Bilirubin Surveys. This specimen was intended to help evaluate
all bilirubin methods.
Objective.--To evaluate the effect of commutable specimens on the performance of selected clinical analyzers in measuring neonatal bilirubin from 2003 through 2006.
Design.--A human serum-based specimen enriched with unconjugated bilirubin in human serum has been included since 2003 in the Neonatal Bilirubin Surveys. The bilirubin values of these specimens were determined by the reference method and used to evaluate results reported by various chemistry analyzers.
Results.--Coefficients of variation for College of American Pathologists All Data ranged from 4.9% to 6.2% for the Neonatal Bilirubin Survey. However, coefficients of variation for the 4 major instrument groups (Dimension, Olympus, Synchron, and Vitros), which report 65% of all results, varied from 2% to 3%. College of American Pathologists All Data mean bilirubin values were within 0.46 mg/dL (7.8 [micro]mol/L) of the reference method mean in 2003; in subsequent years these differences became larger, peaking at 1.87 mg/dL (32 jjimol/L) in 2005.
Conclusions.--The large systematic error of bilirubin measurements is due primarily to failure of instrument manufacturers to produce reliable bilirubin calibrators. Primary calibrators should consist of human serum enriched with unconjugated bilirubin. Bilirubin values must be assigned by the reference method, the performance and robustness of which are reported in this article. Secondary calibrators distributed to users must be traceable to primary calibrators.
Infants (Newborn) (Health aspects)
Pathological laboratories (Standards)
Infants (Newborn) (Medical examination)
Infants (Newborn) (Methods)
Infants (Newborn) (Standards)
Lo, Stanley F.
Doumas, Basil T.
|Publication:||Name: Archives of Pathology & Laboratory Medicine Publisher: College of American Pathologists Audience: Academic; Professional Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2008 College of American Pathologists ISSN: 1543-2165|
|Issue:||Date: Nov, 2008 Source Volume: 132 Source Issue: 11|
|Topic:||Event Code: 350 Product standards, safety, & recalls|
|Geographic:||Geographic Scope: United States Geographic Code: 1USA United States|
In 2003, the Chemistry Resource Committee of the College of
American Pathologists (CAP), acting on a report regarding inaccuracies
in the measurement of bilirubin in clinical laboratories, (1) introduced
a new specimen in the Neonatal Bilirubin Surveys (NB-Surveys). This
specimen, human serum enriched with unconjugated bilirubin, was expected
to be commutable (free of matrix effects) because it closely resembles
specimens drawn from healthy neonates. Properly calibrated clinical
analyzers were expected to be accurate, providing total bilirubin (TBIL)
values close to those obtained by the reference method. The
noncommutable or conventional NB-Survey specimens consisted of
unconjugated bilirubin and ditaurobilirubin in bovine serum. (1)
A guideline issued by the American Academy of Pediatrics recommended a blood exchange transfusion for infants older than 48 hours when bilirubin levels near 25 mg/dL (428 [micro]mol/L) cannot be reduced by phototherapy. (2) Because this is considered a critical bilirubin value, the CAP Chemistry Resource Committee decided to adjust the bilirubin level in the human serum-based specimen near 25 mg/dL (428 [micro]mol/L).
Results from the 2003 human serum-based specimen (NB-02) were very encouraging. The grand mean value for CAP All Data was only 2.4% higher than the reference method mean and the coefficient of variation (CV) % of 5.7.
This communication is a progress report on the performance of laboratories participating in the NB-Surveys for the years 2004 through 2006. The practice of including the special, commutable NB-Survey specimen in the Chemistry Survey (C-Survey), as specimen C-96 or C-97, continued until the end of year 2006.
MATERIALS AND METHODS
The preparation and distribution to participating laboratories of the human serum-based specimens has been described previously. (3) All NB-Survey and C-Survey specimens (from 2003 through 2006) were analyzed for TBIL by the reference method4 at the Reference Standards Laboratory of the Children's Hospital of Wisconsin, Milwaukee. All other bilirubin results shown in this article have been reported in the participant summary report. Instruments specifically considered in this report include the BR2 (Advanced Instruments Inc, Norwood, Mass), Advia (Siemens Medical Solutions Diagnostics, Tarrytown, NY), Aeroset and Architect (Abbott Diagnostics, Abbott Park, Ill), Cobas Integra, Roche Modular/Hitachi (Roche Diagnostic Systems, Branchburg, NJ), Dimension (Dade Behring, Newark, Del), Olympus (Olympus America Inc, Melville, NY), Synchron (Beckman Coulter Inc, Fullerton, Calif), Unistat (Leica Microsystems Inc, Buffalo, NY), and Vitros (Ortho-Clinical Diagnostics, Raritan, NJ).
According to the participant summary report, methods based on the coupling of bilirubin with a diazo compound are used in the Advia, Aeroset, Architect, Cobas Integra, Dimension, Roche Modular/Hitachi, Olympus, Synchron, and Vitros (TBIL slide). Methods based on the oxidation of bilirubin are used in the Advia (vanadate oxidation) and Synchron (bilirubin oxidase). Methods based on direct spectrophotometry are used in the Aeroset, Architect, Advanced BR2, Unistat, and Vitros neonatal bilirubin (BuBc) slide, which measures the sum of unconjugated bilirubin and conjugated bilirubins (bilirubin monoglucuronide and diglucuronide). According to Abbott, both the Aeroset and Architect methods use a diazo method (diazotized 2,4-dichloroaniline), but for the NB-Survey users prefer direct spectrophotometry.
Table 1 shows CAP All Data mean bilirubin values and CV% for the NB-Survey and C-Survey specimens consisting of unconjugated bilirubin in pooled human sera sent to participating laboratories during 2003-2006; also shown are values obtained by the reference method. For each year from 2003 through 2006, one commutable sample was prepared and sent out as a specimen in the NB-Survey and C-Survey; thus, in 2003, NB-02 was identical to C-96; in 2004, NB-06 was identical to C-97; in 2005, NB-05 was identical to C-96; and in 2006, NB-01, C-97, and NB-11 were identical. Shown in Figure 1 are differences between mean values of CAP All Data and those of the reference method for NB-Survey and C-Survey specimens 20032006. The best overall laboratory performance was achieved in 2003. The CAP All Data mean value for NB 02 was identical to that for C-96 and the difference, 0.46 mg/dL (7.9 [micro]mol/L) or 2.4%, between the CAP All Data mean and that of the reference method was the smallest in 4 years. (3)
In 2006, the commutable specimen was mailed to participants on 3 occasions, in April and November with the NB-Survey (specimens NB-01 and NB-11, respectively) and in June with the C-Survey (specimen C-97). The unusually large difference, 0.64 mg/dL (10.9 [micro]mol/L), between the NB-01 and NB-11 CAP All Data bilirubin mean values (Figure 2) is due to large shifts in the mean values reported by the Cobas Integra, Dimension, Roche Modular/Hitachi, and Vitros, which constitute 60% of the participating laboratories. Discrepancies between NB-01 and C-97 (Figure 2) are primarily due to the large percentage of Vitros participants in the C-Survey.
[FIGURE 1 OMITTED]
Figure 3 shows differences between bilirubin mean values for selected field methods and the reference method. With 3 exceptions, Advanced BR2, Aeroset (2003-2004), and Architect (2006), mean bilirubin values for all other instrument groups are higher than those of the reference method. Note that there were no data from the Abbott instruments in 2005, and both Abbott instruments use the same diazo and direct spectrophotometry methods. The positive bias ranges from less than 1 mg/dL to 4 mg/dL (17.1 to 68.4 [micro]mol/L). The Synchron group consistently shows a small positive bias. The negative bias of the BR2 has decreased over time and is smallest in 2006.
Reference methods are considered the gold standard against which field methods are compared and evaluated. It is important, therefore, to establish the long-term performance characteristics of the reference method for serum TBIL. From September 2003 to April 2007, the Reference Laboratory of the Children's Hospital of Wisconsin prepared 11 bilirubin standard solutions by adding National Institute of Standards and Technology Bilirubin, Standard Reference Material 916, to pooled human sera to a concentration of 20 mg/dL (342 [micro]mol/L). Following preparation, these solutions were analyzed by the reference method and the molar absorptivity (e) of the alkaline azopigment was compared with the values established in 2 round robin studies with participation from national and international laboratories. (5,6) Collaborating laboratories in the 1988 round robin study were from the United States (3) and The Netherlands (2); in the 1998 round robin study, laboratories were from the United States (3), Germany (2), and The Netherlands (1). Data from the 2 round robins and from the Children's Hospital of Wisconsin Reference Laboratory, shown in Table 2, indicate that the established molar absorptivity of the alkaline azopigment, because of its reproducibility and narrow range, can and should be used to assess the accuracy of bilirubin calibrators consisting of unconjugated bilirubin in human sera.
The 20-mg/dL (342-[micro]mol/L) stock solutions are dis pensed in airtight plastic tubes and kept at--70[degrees]C until use. Each time the reference method is used, the stock solution is analyzed to verify that its [epsilon] value is within the accepted limits (mean value [+ or -] 3 SD). Next, the stock solution and 3 dilutions (2.5 mg/dL [42.7 [micro]mol/L], 5.0 mg/ dL [85.5 [micro]mol/L], and 10 mg/dL [171 [micro]mol/L]) are analyzed to construct a calibration curve, the regression equation of which is used to calculate the results of the specimens analyzed. Shown in Table 3 are molar absorptivities, slopes, and intercepts of calibration curves obtained during a 3.5-year period, and precision data on controls prepared in our laboratory. The data in Tables 2 and 3 demonstrate the impressive stability and robustness of the reference method in US and European laboratories.
[FIGURE 2 OMITTED]
[FIGURE 3 OMITTED]
In 2003, the CAP All Data mean values for NB-02 and C-96 were identical and within 2.4% of the reference value (Figure 1). In our opinion this was due to the use of calibrators with accurately assigned bilirubin values. In subsequent years accuracy deteriorated. Results for 2006 are strange; there is good agreement for CAP All Data means between specimens C-97 and NB-11, but the mean value for NB-01 is greater than NB-11 by 0.64 mg/dL (10.9 [micro]mol/L). The lower mean bilirubin value for NB-11 was due to the downward shift of 4 instrument group mean values (Cobas Integra, Dimension, Hitachi/Roche Modular, and Vitros, which represent 59% of the reported results) (Figure 2). The close agreement between the mean values for C-97 and NB-11, 22.44 mg/dL (383.7 [micro]mol/L) and 22.32 mg/dL (381.7 [micro]mol/L), indicates a small, if any, loss of bilirubin, in the 6-month period (June-November). The most logical explanation for the difference between NB-01 and NB-11 would be a change in calibrators or reassignment of bilirubin values in the calibrators used by these instruments. Consistent with this explanation, Dade Behring and Roche Diagnostics were among the diagnostic companies that lowered the assigned values to their bilirubin calibrators (Diane Stille, Dade Behring Inc, Newark, Del, written communication, May 7, 2006, and Melanie Swartzentuber, Roche Diagnostics Corporation, Indianapolis, Ind, written communication, August 18, 2006). In addition, the Roche line of instruments revealed dramatic improvement from 2005 to 2006 (Figure 3) and is consistent with an earlier change to lower assigned bilirubin calibrator values (Melanie Swartzentuber, Roche Diagnostics Corporation, written communication, October 5, 2005). The large difference between C-97 and NB-01 mean bilirubin values for the Vitros indicates a lapse in the calibration of the TBIL and neonatal bilirubin (Bu + Bc) methods; because NB-01 contained only unconjugated bilirubin, one would expect the TBIL and neonatal bilirubin values to be very close.
Coefficients of variation for CAP All Data increased in 2004 and 2005, but in 2006 returned to the level of 2003 (Table 1). Lack of improvement is attributed to the large differences in the mean bilirubin values reported by the various instrument groups (Figure 3). It should be noted that the CVs for the major instrument groups (Dimension, Olympus, Synchron, and Vitros), which report about 65% of all results, vary from 2% to 3%.
The goal of limiting the total error to [+ or -] 10% of the reference method value (3) can be realized only by improving accuracy. Since 2003 the gap between the reference values and the CAP All Data has progressively increased (Table 1); this applies to most instrument groups with notable exceptions of Synchron, and lately Advanced BR2 and Cobas Integra. To state simply: Calibrators used by most manufacturers are inaccurate, that is, the assigned values are much higher than the actual bilirubin concentrations. Because for most major instrument groups precision is quite good (CVs, 2%-3%), further improvement in accuracy would depend on the use of accurate calibrators.
Instruments from the same manufacturer using methods based on different principles exhibit the widest variation in results. Examples are as follows: (1) By direct spectrophotometry (neonatal bilirubin method), differences between the reference method results and those reported by the Vitros are 2.74 mg/dL (49.9 [micro]mol/L) (NB-01) and 2.06 mg/dL (35.2 [micro]mol/L) (NB-11), respectively, but only 0.36 mg/dL (6.1 [micro]mol/L) by the TBIL diazo method; the reference method and the mean TBIL Vitros values for C-97 were 21.04 mg/dL (360 [micro]mol/L) and 21.40 mg/dL (366 [micro]mol/L), respectively. (2) Synchron values obtained by the diazo method differ by less than 1.0 mg/ dL from those of the reference method while those of the bilirubin oxidase method differ (except for 2003) from 1.97 mg/dL (33.7 [micro]mol/L) to 2.96 mg/dL (50.6 [micro]mol/L) (Figure 3). (3) According to Abbott package inserts, 2 methods are available for TBIL for both the Aeroset and Architect analyzers. One is a diazo method (diazotized 2,4-dichloroaniline) and the other a direct spectrophotometry. However, because these are ''open platform'' instruments, the users may have chosen a variety of other methods (listed in the participant summary report as ''JG w/blank,'' "Spectrophot w/o blank" or "w/blank," "DZ Salt," "oxidation," "vanadate oxidation"), which yield very variable results. It is important to emphasize that calibrators containing ditaurobilirubin or made in a protein base other than human serum may not be suitable for calibrating all bilirubin methods, (1) that is, assigned values to calibrators may have to be different for methods based on different principles. A calibrator suitable for all methods is one made with human serum enriched with unconjugated bilirubin. This is because it has the same composition as specimens obtained from healthy neonates.
Because of the importance of accuracy in the measurement of bilirubin in neonates, we proposed setting a goal for limiting the total error (inaccuracy plus imprecision) to [+ or -] 10% of the reference method value at bilirubin concentrations greater than 10 mg/dL (171 [micro]mol/L). (3)
In 2003, the first year of the introduction of the commutable survey specimen consisting of human serum enriched with unconjugated bilirubin, that goal appeared to be within reach for at least 80% of the reported results. That was presumably due to better bilirubin calibrators used in that year as judged by the small difference between the CAP All Data and reference method mean value. In 2006, for specimen NB-01, only the Advanced BR2 and the Synchron CX4/5CE met the upper limit of the [+ or -] 10% goal; all instruments met the lower limit except the Architect. The [+ or -] 20% error allowed by the Clinical Laboratory Improvement Amendments of 1988 implies that acceptable values for a specimen having bilirubin concentration of 25 mg/dL (428 [micro]mol/L) would be from 20 mg/ dL (342 [micro]mol/L) to 30 mg/dL (512 [micro]mol/L);this very large window should be unacceptable to pediatricians and neonatologists. To improve bilirubin assays, manufacturers of clinical analyzers and reagent kits will need to im prove the quality of bilirubin calibrators. A bilirubin reference method and reference material, as well as an established molar absorptivity for the alkaline azopigment, can be used to assess the accuracy of calibrators made with unconjugated bilirubin in human serum. These calibrators may then be used to assign values to secondary calibrators for field methods.
Accepted for publication April 2, 2008.
(1.) Lo SF, Doumas BT, Ashwood ER. Performance of bilirubin determinations in US laboratories--revisited. Clin Chem. 2004;50:190-194.
(2.) American Academy of Pediatrics, Provisional Committee for Quality Improvement and Subcommittee on Hyperbilirubinemia. Practice parameter: management of hyperbilirubinemia in the healthy term newborn. Pediatrics. 1994;94: 558-565.
(3.) Lo SF, Doumas BT, Ashwood ER. Bilirubin proficiency testing using specimens containing unconjugated bilirubin and human serum: results of a College of American Pathologists study. Arch Pathol Lab Med. 2004;128:1219-1223.
(4.) Doumas BT, Poon PKC, Perry BW, et al. Candidate reference method for determination of bilirubin in serum: development and validation. Clin Chem. 1985;31:1779-1 789.
(5.) Doumas BT, Perry BW, McComb RB, et al. Molar absorptivities of bilirubin (NIST SRM 916a) and its neutral and alkaline azopigments. Clin Chem. 1990;36: 1698-1701.
(6.) National Institute of Standards and Technology. Certificate of Analysis. Standard Reference Material 916a, Bilirubin. Gaithersburg, Md: National Institute of Standards and Technology; 2001.
Stanley F. Lo, PhD; Bernadine Jendrzejczak, BA; Basil T. Doumas, PhD
From the Reference Standards Laboratory, Department of Pathology and Laboratory Medicine, Children's Hospital of Wisconsin, Milwaukee (Drs Lo and Doumas and Ms Jendrzejczak); and the Department of Pathology, Medical College of Wisconsin, Milwaukee (Drs Lo and Doumas).
The authors have no relevant financial interest in the products or companies described in this article.
Reprints: Stanley F. Lo, PhD, Reference Standards Laboratory, Children's Hospital of Wisconsin, 9000 W Wisconsin Ave, Wauwatosa, WI 53226 (e-mail: firstname.lastname@example.org).
Table 1. College of American Pathologists (CAP) All Data and Reference Method Mean Bilirubin Values and Coefficient of Variation (CV)% for the Neonatal Bilirubin Survey and Chemistry Survey Specimens CAP All Data Reference Mean Bilirubin Method Value, No. of Specimen Value, mg/dL CV% mg/dL Laboratories 2003 NB-02 19.90 5.7 19.44 1743 C-96 19.92 5.6 19.44 4982 2004 NB-06 23.69 5.2 22.06 1784 C-97 23.41 6.4 22.06 4757 2005 NB-05 26.25 6.2 24.38 1848 C-96 26.35 7.0 24.38 4552 2006 NB-01 22.96 5.2 21.19 1890 C-97 22.44 5.5 21.19 4775 NB-11 22.32 4.9 21.04 1718 Table 2. Molar Absorptivity of Bilirubin Alkaline Azopigment Results From 2 Round Robins and the Reference Laboratory of Children's Hospital of Wisconsin (CHW) * n [epsilon] (L-[mol.sup.1]* SD [cm.sup.-1]([dagger]) Round Robin 1988 (5 labs) 10 76 490 610 Round Robin 1998 (6 labs) 12 76 640 601 Reference Lab CHW 11 76 869 589 * n indicates number of determinations; [epsilon], molar absorptivity at 598 nm. ([dagger]) Corrected for purity (98.3%) of bilirubin. Table 3. Long-term Calibration Stability and Precision of the Reference Method for Total Bilirubin Calibration Curve (September 15, 2003-April 23, 2007) [epsilon] (L-[mol.sup.-1] Slope Intercept *[cm.sup.-1]) * Mean 76 554 0.0770 -0.0003 SD 377 0.0004 0.0018 ([dagger]) N 45 46 46 Min 75 579 0.0759 -0.0038 Max 77 257 0.0782 0.0043 * [epsilon] indicates molar absorptivity. ([dagger]) One standard deviation for a single measurement.
|Gale Copyright:||Copyright 2008 Gale, Cengage Learning. All rights reserved.|