Stats.Con: How We Have Been Fooled By Statistics-Based Research in Medicine.
Article Type: Book review
Subject: Books (Book reviews)
Author: Miller, Clifford G.
Pub Date: 06/22/2011
Publication: Name: Journal of American Physicians and Surgeons Publisher: Association of American Physicians and Surgeons, Inc. Audience: Academic Format: Magazine/Journal Subject: Health care industry Copyright: COPYRIGHT 2011 Association of American Physicians and Surgeons, Inc. ISSN: 1543-4826
Issue: Date: Summer, 2011 Source Volume: 16 Source Issue: 2
Topic: NamedWork: Stats.Con: How We Have Been Fooled By Statistics-Based Research In Medicine (Nonfiction work)
Persons: Reviewee: Penston, James
Accession Number: 267810804
Full Text: Stats.Con: How We Have Been Fooled By Statistics-Based Research In Medicine, by James Penston, softcover, 319 pp, $12.99, ISBN 978-2-907313-33-2, UK, London Press, 2010.

This book is a "must-read" for all AAPS members: practitioners, researchers, administrators, and managers: an essential textbook of at least first reference and more.

It is impressive and an intellectual joy in just over 300 pages. The author is a consultant physician and gastroenterologist in a National Health Service district general hospital in the north of England, with a background in the pharmaceutical industry and clinical research. He displays a deep under-standing of the many subjects that come together in this thoroughly researched, comprehensive, intellectually stimu-lating, and--most bizarrely for a book on statistics--enjoyable text. The author explains complex subject matter absent the jargon and opaque flowery language that academics sometimes use to disguise their inability to explain their own subject. And the typeface is comfortable also.

This is not merely a book on medical statistics. It has a breadth and depth of coverage of a multiplicity of subjects that the author skillfully melds into a cohesive and cogent whole. History, philosophy, politics, corruption, and fraud, all find their place. This is not simply a case of statistical treatments of significance and confidence in numerical results--which often fail to reveal or account for the variety of methodological and other errors inherent in such studies. It is rather an elegant, deep, thorough deconstruction of current practice, revealing confusion and muddle of fundamental concepts by researcher authors of and readers of formally published medical studies.

This comprehensive treatment achieves far more than communication of its core messages: the absence of justification for the trust placed in modern epidemiological studies and large-scale randomized clinical trials, particularly where therapeutic or other claimed effects are small. The author's arguments illustrate how and why our trust in the P-values and confidence intervals, as measures of reliability of results, is misplaced and false.

The author's accessible style, without sacrificing rigor of treatment or accuracy of terminology, allows full rein to excite and stimulate the reader's mind. It is like a full-course gourmet meal rich in flavor. The reader comes away satisfied and wanting to return another time.

Penston argues that the grounds for causal inference in statistics-based research are lacking. Not only can we can never know whether the results of a randomized controlled trial (RCT) apply either to a particular patient or to a specified group, but the essential inference of the presence of a causal relationship, drawn from small differences in outcomes revealed by RCTs, is highly questionable. The frequentist approach to statistics used in nearly all medical research, which is based solely on the frequency with which a characteristic occurs, is unsound, and serious criticisms of it remain unanswered. The external validity of RCTs, namely the reliability of any generalization from results of an individual study to the wider population of patients, is always open to question. The size of the treatment effect in large-scale studies is very small. The true size of an effect can be, and deliberately is concealed by some researchers and others with a vested interest in the outcome of the studies. Needed close and independent examination is rarely accorded, so the dubious worth and doubtful meaning of statistical studies goes unnoticed.

The conditions for internal validity of RCTs are rarely if ever satisfied by randomization, allocation concealment, double-blind administration of treatment, the handling of withdrawals and drop-outs, and the statistical tests.

With the seeming increase in research fraud, lack of ability to confirm results is concerning. Neither the results of individual RCTs nor the statistical methods used can in general be tested independently because RCTs involve heterogeneous samples with unknown mixtures of constituents.

Penston argues that even if we were to accept the validity of causal inference in such circumstances, and to dismiss concerns about independent testing, we would still face a distasteful reality that the product of statistics-based research is of little value.

Penston shows how statistics-based research has become accepted over the past 50 years. He argues that advocates have used every means to spread a flawed methodology, and their views have infiltrated the thinking of generations of researchers, practicing physicians, and others involved in the care of patients. The implications extend beyond medicine.

While the author touches on the importance of clinical research and how statistical studies have overshadowed it, he misses answering two important questions: What is and should be the place of clinical research, evidence, and experience? And how can their rightful place in medicine be regained?

Chapter 7, "Fraud in Medical Research," might be usefully supplemented by separate treatments of the psychologies of fraud, of author biases, and reinforcements of belief in expected outcomes.

Clifford G. Miller, Esq.

Beckenham, Kent, England
Gale Copyright: Copyright 2011 Gale, Cengage Learning. All rights reserved.