Comparison of validation methods for the diagnosis of occlusal caries in primary molars.
AIM: To evaluate the reproducibility of 7 validation methods used
for caries diagnosis in primary teeth. METHODS: Seventy-two occlusal
sites were selected on 40 primary molars. The sites were evaluated
independently by 3 experienced examiners using validation methods that
involved direct assessment, i.e. by using a (1) magnifying glass (8x)
and (2) stereomicroscope (35x), or indirect assessment i.e. by using (3)
photographs, (4) slide projections of photographs, (5) stereomicroscope
(35x) photographs, (6) stereomicroscope (35x) slide projections, and (7)
projections of polarised light microscope slides. Cohen's kappa
coefficients were calculated and subjected to the Kruskal-Wallis test at
a significance level of 5%. RESULTS: The mean inter-examiner kappa
values for the validation methods were 0.31-0.51. There were
statistically significant differences (p<0.05) between methods 1 and
3, 1 and 4, 2 and 4, 4 and 5, 4 and 6, and 4 and 7. Moderate agreement
was observed for all methods except methods 1 and 4, for which the
agreement was fair. CONCLUSIONS: The inter-examiner agreement for all
validation methods for caries diagnosis was moderate, except for the
method based on indirect assessment by slide projection, which showed
Key words: Primary molar teeth, diagnosis, occlusal caries
Dental caries (Research)
Pinto, A.C. Guedes
|Publication:||Name: European Archives of Paediatric Dentistry Publisher: European Academy of Paediatric Dentistry Audience: Academic Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2012 European Academy of Paediatric Dentistry ISSN: 1818-6300|
|Issue:||Date: April, 2012 Source Volume: 13 Source Issue: 2|
|Topic:||Event Code: 310 Science & research|
Despite a clear decline in the incidence of dental caries in most developed countries, there has been an increase in the occurrence of caries lesions on the occlusal surfaces of primary and permanent teeth [Mejare et al., 1998; Costa et al., 2008; Ferro et al., 2009], which poses challenges in diagnosis [Shi et al., 2000].
Early caries diagnosis in primary teeth is of great importance as the primary enamel thickness promotes rapid caries progression [Goel et al., 2009]. Moreover, a correct and early diagnosis prevents extensive caries, which can lead to aesthetic, functional, and phonetic problems. Many studies have tested the performance of diagnostic methods mainly used for occlusal surfaces. Most have been laboratory studies that used permanent teeth, while few have been performed with primary teeth [Costa et al., 2008; Apostolopoulou et al., 2009; Goel et al., 2009]. Using in vitro studies, results are compared with those of several "gold standard" methods, and the most common method for diagnosing occlusal caries is the histological evaluation of sections.
The validation of diagnostic methods is important for determining their performance. The available validation methods should meet 3 basic criteria: they should be accurate and, therefore, reproducible, they should establish the diagnosis according to criteria that define the disease, and they should be independent from the diagnostic method under evaluation [Hintze et al., 1995].
Because of the diversity of validation methods, studies have been performed to test between-method correlations and inter-examiner reproducibility [Deery et al., 2000; Hintze and Wenzel, 2002; Costa et al., 2007; Valera et al., 2008; Jablonski-Momeni et al., 2009]. Most of these studies have shown wide variation in the results of several validation methods. Therefore, it is difficult to define reliable parameters for comparisons of the performance characteristics of different diagnostic methods. Furthermore, the differences between primary and permanent teeth, as well as between the characteristics of caries in these teeth, may be responsible for differences in the results of validation methods.
Thus, the aim of this study was to evaluate the reproducibility of 7 validation methods used for caries diagnosis in primary teeth. The null hypothesis was that there was no difference among these validation methods.
Materials and methods
Forty-two exfoliated primary molars (14 first and 28 second molars) were selected according to the clinical criteria of sound occlusal surfaces or suspected caries lesions (incipient and pigmented caries lesions, regardless of activity) at one or more sites, without frank cavitation. The teeth were cleaned and stored in de-ionised water until use. The teeth were embedded in orthophthalic resin and sectioned (mesial to distal) with a diamond disc under water refrigeration in a Labcut 1010 machine (Extec Co., Enfield, CT USA) into approximately 0.5-mm-thick sections.
One to three sites were selected in each of 58 sections for an overall assessment of 72 sites representing a range of appearances, from sound to pigmented caries. The sites were evaluated independently by three experienced examiners with expertise in cariology and paediatric dentistry by using the histological classification system described in Table 1.
The validation methods were based on direct or indirect assessment of sections.
Direct assessment of sections
1) Magnifying glass (8x): Sections were placed against a burgundy coloured background and examined using a magnifying glass (Waltex Magnifier, China): sections remained hydrated until analysis and immersed again in de-ionised water after the procedure was completed.
2) Stereomicroscope (35x): Sections were positioned directly under a stereomicroscope (Zoom Stereomicroscope, Olympus Optical Co., Japan) and analysed under incident light independently by 3 examiners.
Indirect assessment (images)
3) Photographs: Sections were placed against a burgundy coloured background and photographed individually from a standardised distance (20 cm) by using Nikon FM2 equipment (Japan) and Medical Nikkor 120-mm lenses: The assessment performed using photographs on a 10 x 15 cm sheet of bright paper.
4) Slides: Images obtained in method 3 were projected (Kodak Carousel Slide Projector, Eastman Kodak Company, USA) against a white background.
5) Stereomicroscope photographs (35x): Sites were evaluated using pictures obtained under the stereomicroscope in method 2.
6) Projections of stereomicroscope slides (35x): Images obtained in method 5 were projected against a white background, as in method 4.
7) Slides of polarised light microscopic images: Sections were manually ground and polished with silicon carbide paper (#400, 600, 1200, and 4000 grits) to about 100-150 um, then immersed in distilled water and photographed under a microscope with transmitted polarised light (Zoom Stereomicroscope, Olympus Optical Co., Japan). The images were analysed after they were projected as in methods 4 and 6.
The scores were compared to obtain Cohen's kappa coefficient and subjected to the Kruskal-Wallis test at a significance level of 5%.
The kappa values for inter-examiner reproducibility for all validation methods were 0.31-0.51. Table 2 presents the inter-examiner kappa values obtained for comparing the validation methods. There were statistically significant differences (p<0.05) between methods 1 and 3, 1 and 4, 2 and 4, 4 and 5, 4 and 6, and 4 and 7. Indirect assessment of sections by slide projection (method 4) was the least effective validation method, as the kappa value (0.31) indicates. According to the evaluation performed using the Landis and Koch  score system, the results of the direct assessment of sections using magnification (method 1) and indirect assessment by slide projection (method 4) were fair, and those of the other methods were moderate.
Many studies have been conducted to verify the performance of validation methods for the diagnosis of caries lesions, specifically on occlusal surfaces. Most studies focused on the validation of methods for diagnosis in extracted teeth, but it is necessary to validate the methods that will be used at the time of diagnosis. Validation is very important for determining the performance characteristics of an assessment method and defining its sensitivity and specificity [Wenzel et al., 1994]. However, very few studies have involved validation methods for the diagnosis of caries in primary teeth [Costa et al., 2008; Apostolopoulou et al., 2009; Goel et al., 2009]. In addition, comparisons with permanent teeth are inappropriate because of the different substrate characteristics and differing rates of progression of carious lesions [Goel et al., 2009].
In this study, we compared seven validation methods for diagnosing occlusal caries lesions in primary molars. The null hypothesis was partially rejected. The evaluation of the kappa values showed that a validation system based on the projection of images obtained by polarised light microscopy produced the highest mean inter-examiner kappa value, i.e., 0.51. In a report comparing validation methods, Ricketts et al.  suggested that polarised light microscopy is difficult to perform because of the need to prepare very thin specimens (average thickness, 100-150 [micro]m). However, they showed the possibility of diagnosing lesions in their initial stages when this technique was used. In fact, the number of specimens was reduced to 40 because of the difficulties encountered in sample preparation. Nevertheless, the inclusion of this technique is important because together with the criteria used, it allows for the detection of lesions in their initial stages of development.
The comparison of mean inter-examiner kappa values showed that inter-examiner agreement was low for the method based on the evaluation of projected images (b2). Previous studies that have evaluated validation methods did not use the same projection equipment, making it difficult to directly compare projection-based validation methods. In our analysis, the mean inter-examiner kappa values for methods in which photographs were obtained and slides viewed using Nikon equipment, were 0.48 and 0.30, respectively. As the same equipment was used, the results were expected to be similar; the differences in the results could be attributable to the differences in the projection of images, e.g. the magnification at which they were shown. It should be emphasised that we used 0.5mm thick tooth sections. Wenzel et al.  stressed the importance of the use of thin sections and suggested that only hemi-sections did not provide clear information for correct validation. This assertion applies to our case, as a lesion was classified on the basis of the stratification of the tooth [Ekstrand et al., 1997]. The separation of teeth into only two specimens does not provide sufficient area for analysis and thus may not reveal the true status of the tooth.
Another relevant issue is the criteria for the classification of lesions in evaluated specimens. In the study by Wenzel et al.  4 criteria were used without subdividing the enamel lesions into stages: initial and dentino-enamel junction. Similarly, the studies by Hintze et al.  which used 3 scoring criteria, and those by Ricketts et al.  and Jablonski-Momeni et al. , which used the ICDAS-II criteria, did not evaluate the subdivision of enamel into stages. In this study, as well as in the investigation published by Pereira et al. , the criteria used by Ekstrand et al.  were adopted. There are different approaches to classify the stages of lesions; it is important to define criteria that include the subdivision of the enamel and to determine whether the dentino-enamel junction is involved. Furthermore, it is difficult to obtain greater inter-examiner agreement.
The importance of validation is in assuring the precision and sensitivity of a given procedure. The establishment of a gold standard will enhance our ability to develop and assess more reliable diagnostic methods, thus improving the clinical detection of caries lesions on occlusal surfaces in primary teeth.
The inter-examiner agreement was moderate for all validation methods for caries diagnosis, except the method based on projected images, which showed low agreement.
Apostolopoulou D, Lagouvardos P, Kavvadia K, Papagiannoulis L. Histological validation of a laser fluorescence device for a occlusal caries detection in primary molars. Eur Arch Paediatr Dent 2009;10:11-15.
Costa AM, Bezerra AC, Fuks AB. Assessment of the accuracy of visual examination, bite-wing radiographs and DIAGNOdent on the diagnosis of occlusal caries. Eur Arch Paediatr Dent 2007;8:118-122.
Costa AM, Paula LM, Bezerra AC. Use of Diagnodent for diagnosis of non-cavitated occlusal dentin caries. J Appl Oral Sci 2008;16:18-23.
Deery C, Ferrier S, Longbottom C, Nugent ZJ, Pitts, NB. The effect of diagnostic criteria and validation system on the interpretation of caries diagnostic results. Caries Res 2000;34:322-323.
Ekstrand KR, Ricketts DN, Kidd EA. Reproducibility and accuracy of three methods for assessment of demineralization depth of the occlusal surface: an in vitro examination. Caries Res 1997;31:224-231.
Ferro R, Besostri A, Olivieri A. Caries prevalence and tooth surface distribution in a group of 5-year-old Italian children. Eur Arch Paediatr Dent 2009;10:33-37.
Goel A, Chawla HS, Gauba K, Goyal A. Comparison of validity of DIAGNOdent with conventional methods for detection of occlusal caries in primary molars using the histological gold standard: An in vivo study. J Indian Soc Pedod Prevent Dent 2009;27:227-234.
Hintze H, Wenzel A, Larsen MJ. Stereomicroscopy, film radiography, microradiography and naked-eye inspection of tooth sections as validation for occlusal caries diagnosis. Caries Res 1995;29:359-363.
Hintze H, Wenzel A. Influence of the validation method on diagnostic accuracy for caries. A comparison of six digital and two conventional radiographic systems. Dentomaxillofac Radiol 2002;31:44-49.
Jablonski-Momeni A, Ricketts DN, Stachniss V, et al. Occlusal caries: Evaluation of direct microscopy versus digital imaging used for two histological classification systems. J Dent 2009;37:204-211.
Landis JR, Koch GG. The measurement of oberver agreement for categorical data. Biometrics 1977;33:159-174.
Mejare I, Kallestal C, Stenlund H, Johansson H. Caries development from 11 to 22 years of age: a prospective radiographic study. Prevalence and distribution. Caries Res 1998;32:10-6.
Pereira AC, Eggertsson H, Martinez-Mier EA et al. Validity of caries detection on occlusal surfaces and treatment decisions based on results from multiple caries-detection methods. Eur J Oral Sci 2009;117:51-57.
Ricketts DN, Watson TF, Liepins PJ, Kidd EA. A comparison of two histological validating techniques for occlusal caries. J Dent 1998;26:89-96.
Shi XQ, Welander U, Angmar-Mansson B. Occlusal caries detection with KaVo DIAGNOdent and radiography: an in vitro comparison. Caries Res 2000;34:151-158.
Valera FB, Pessan JP, Valera RC, Mondelli J, Percinoto C. Comparison of visual inspection, radiographic examination, laser fluorescence and their combinations on treatment decisions for occlusal surfaces. Am J Dent 2008;21:25-29.
Wenzel A, Verdonschot EH, Truin GJ, Konig KG. Impact of the validator and the validation method on the outcome of occlusal caries diagnosis. Caries Res 1994;28:373-377.
M.D.M. Oliveira *, T.K. Tedesco *, T.L. Lenzi *, A.C. Guedes Pinto **, R.O. Rocha *
* Dept. of Stomatology, Federal University of Santa Maria, Santa Maria;
** Dept. of Orthodontics and Paediatric Dentistry, University of Sao Paulo, Sao Paulo, Brazil.
Postal address: Dr R.O. Rocha, Departamento de Estomatologia, Universidade Federal de Santa Maria, Rua Floriano Peixoto, 1184 sala 109 Santa Maria, RS, Brazil.
Table 1. Criteria for caries assessment of histological tooth sections Score Criteria 0 No demineralisation 1 Enamel demineralisation limited to the outer half (50%) of the enamel layer 2 Enamel demineralisation extending into the inner half of the enamel layer and dentino-enamel junction 3 Dentine demineralisation in its initial layer 4 Frank dentine demineralisation Table 2. Inter-examiner reproducibility (kappa values) according to validation methods for diagnosis of caries in primary molars. Validation methods (groups) Examiners 1 2 3 4 1 x 2 0.44 0.35 0.45 0.28 1 x 3 0.39 0.58 0.51 0.33 2 x 3 0.35 0.46 0.51 0.32 Mean kappa value 0.39a 0.47a,b 0.49b 0.31c Validation methods (groups) Examiners 5 6 7 1 x 2 0.42 0.33 0.75 1 x 3 0.50 0.49 0.39 2 x 3 0.56 0.46 0.38 Mean kappa value 0.49a,b 0.43a,b 0.51a,b * Different superscript letters indicate statistically significant differences (p<0.05) between groups; Validation method groups: (1) Magnifying glass (8x), (2) Stereomicroscope (35x), (3) Photographs, (4) Slide projections of photographs, (5) Stereomicroscope (35x) photographs, (6) Stereomicroscope (35x) slide projections and (7) Projections of polarised light microscope slides.
|Gale Copyright:||Copyright 2012 Gale, Cengage Learning. All rights reserved.|