Document Detail

A graphical judgmental aid which summarizes obtained and chance reliability data and helps assess the believability of experimental effects.
MedLine Citation:
PMID:  16795609     Owner:  NLM     Status:  PubMed-not-MEDLINE    
Interval by interval reliability has been criticized for "inflating" observer agreement when target behavior rates are very low or very high. Scored interval reliability and its converse, unscored interval reliability, however, vary as target behavior rates vary when observer disagreement rates are constant. These problems, along with the existence of "chance" values of each reliability which also vary as a function of response rate, may cause researchers and consumers difficulty in interpreting observer agreement measures. Because each of these reliabilities essentially compares observer disagreements to a different base, it is suggested that the disagreement rate itself be the first measure of agreement examined, and its magnitude relative to occurrence and to nonoccurrence agreements then be considered. This is easily done via a graphic presentation of the disagreement range as a bandwidth around reported rates of target behavior. Such a graphic presentation summarizes all the information collected during reliability assessments and permits visual determination of each of the three reliabilities. In addition, graphing the "chance" disagreement range around the bandwidth permits easy determination of whether or not true observer agreement has likely been demonstrated. Finally, the limits of the disagreement bandwidth help assess the believability of claimed experimental effects: those leaving no overlap between disagreement ranges are probably believable, others are not.
J C Birkimer; J H Brown
Related Documents :
23124479 - In silico multi-scale model of transport and dynamic seeding in a bone tissue engineeri...
3839739 - The measurement of interobserver agreement based on categorical scales.
15898539 - Texture and haptic cues in slant discrimination: reliability-based cue weighting withou...
1412519 - Criterion of reliability as the function of variability and significance of biological ...
15587489 - The validity and reliability of the global index of safety (gis).
17852959 - Activity-based assessment (bia)--inter-rater reliability and staff experiences.
20232149 - Combined method for reduction of high frequency interferences in surface electroenterog...
17136359 - Trunk biomechanical models based on equilibrium at a single-level violate equilibrium a...
18629809 - Møller-plesset perturbation energies and distances for hec(20) extrapolated to the comp...
Publication Detail:
Type:  Journal Article    
Journal Detail:
Title:  Journal of applied behavior analysis     Volume:  12     ISSN:  0021-8855     ISO Abbreviation:  J Appl Behav Anal     Publication Date:  1979  
Date Detail:
Created Date:  2010-06-29     Completed Date:  2010-06-29     Revised Date:  2010-09-14    
Medline Journal Info:
Nlm Unique ID:  0174763     Medline TA:  J Appl Behav Anal     Country:  United States    
Other Details:
Languages:  eng     Pagination:  523-33     Citation Subset:  -    
University of Louisville.
Export Citation:
APA/MLA Format     Download EndNote     Download BibTex
MeSH Terms

From MEDLINE®/PubMed®, a database of the U.S. National Library of Medicine

Previous Document:  Comparison of observational recordings in vivo, via mirror, and via television.
Next Document:  Back to basics: Percentage agreement measures are adequate, but there are easier ways.