On Bem and Bayse.
|Publication:||Name: The Journal of Parapsychology Publisher: Parapsychology Press Audience: Academic Format: Magazine/Journal Subject: Psychology and mental health Copyright: COPYRIGHT 2011 Parapsychology Press ISSN: 0022-3387|
|Issue:||Date: Fall, 2011 Source Volume: 75 Source Issue: 2|
|Geographic:||Geographic Scope: United States Geographic Code: 1USA United States|
A key objective of parapsychologists has always been to gain
acceptance of the existence of psi from mainstream scientists. Crucial
to that effort is support from prominent scientists in mainstream
fields. In recent years, the most important of these allies has been
Daryl Bem, a distinguished social psychologist who recently retired from
Cornell. Bem burst on the scene 17 years ago when he authored with
Charles Honorton a report of a series of successful ganzfeld experiments
conducted at Honorton's laboratory (Bem & Honorton, 1994). The
article was published in a major psychology journal, Psychological
Bulletin. Then earlier this year, Bern (2011) published a successful
series of precognition experiments in another prestigious psychology
journal, the Journal of Personality and Social Psychology (JPSP). In
both cases, critical responses were published in the same issue of the
journal. However, much more so than the ganzfeld paper, the precognition
paper led to a firestorm of other negative comments from the
"establishment." The intensity of the comments suggests that
certain segments of the mainstream scientific community feel threatened
by the publication of Bem's results. For example, the normally
tactful Ray Hyman, who had refereed and approved the publication of the
ganzfeld paper in Psychological Bulletin, was quoted by Carey (2011) in
the online New York Times as proclaiming the publication of the
precognition paper to be "... craziness, pure craziness. I
can't believe a major journal is allowing this work in. I think
it's just an embarrassment to the entire field." A consistent
theme in these commentaries has been the danger that the publication of
Bem's research posed to science generally. I can think of two
reasons why scientists would be more threatened by the precognition
experiments than the ganzfeld experiments. First, Bem conducted the
precognition studies himself. Second, he employed a research methodology
modeled on a procedure widely adopted in mainstream psychology.
I want to comment in more detail on one of these commentaries, written by cognitive psychologist Douglas Hofstadter (2011). He stated, for example, that if psi were real, it "... would go so profoundly against the laws of physics as we know them that our entire scientific worldview would be toppled ..." Of course, a number of physicists/paraphysicists would disagree with this statement (e.g., Walker, 1975), but let's assume for the sake of argument that current physical theories cannot account for psi. Hofstadter's remark implies that we would have to abandon or drastically revise these theories if psi were true. He doesn't tell us why having to do so would be such a terrible thing, but as far as the laws of physics are concerned he doesn't have anything to worry about. Hidden in Hofstadter's remark is the premise that fundamental theories of nature (such as quantum mechanics) are refuted if there is something (in this case, psi) that they cannot account for. However, as philosopher/ parapsychologist Steven Braude pointed out many years ago (Braude, 1986), we don't abandon theories in such cases but merely redefine their domains or boundaries. In fact, none of our current physical theories explains all the accepted facts of nature; the theory of everything, as physicists call it, is still a pipe dream. Does any thoughtful scientist seriously believe that we would abandon a powerful and widely successful theory such as quantum mechanics just because scientists had to admit that psi was real? That sounds as crazy to me as psi does to them. What is threatened by psi is not the validity of our current theories of nature but their collective universality. Moreover, the establishment of psi in the eyes of the mainstream would not even require us to necessarily abandon materialism; psi could be just an epiphenomenon of the brain.
Turning to the controversy about Bem's precognition paper, I was surprised to see that the one published critique was couched in an appeal to psychologists to abandon their traditional methods of statistical hypothesis testing and replace it with an application of Bayse's theorem (Wagenmakers, Wetzels, Boorsboom, & van der Haas, 2011). Briefly, the crucial characteristic of Baysean statistics is that the calculated probability that an experimental hypothesis will be confirmed (technically, the null hypothesis rejected) in an experiment is influenced by the a priori (prior) probability that the experimental hypothesis is true, based, presumably on past data, theory, or even metaphysics. The problem is that there are no ground rules for establishing these a priori probabilities. Of necessity, they are to some extent subjective and they can even be arbitrary. I see great potential for abuse if Baysean statistics were ever to become the norm in psychology. Specifically, it is an excellent way for psychologists to protect their pet theories from refutation. This is especially true in light of the discovery that certain prominent effects in psychology have been subject to the "decline effect" that we have been burdened with in parapsychology (Lehrer, 2011). Here's how it might work. Let's say that the initial experiments testing theory x are very promising, but a decline effect sets in and subsequent tests yield only chance results. All the supporters of theory x have to do is reanalyze the "failed" replications using Baysean statistics, assigning a very high a priori probability for theory x being true based on the earlier significant results. If the assigned a priori probability is high enough (and the sky's the limit), the "failed" replications become statistically significant "successful" replications. Voila! Problem solved.
If you don't think any psychologist would go to such lengths in practice, consider Wagenmakers et al.'s reply to Bern. In a sense, the problem here is the exact opposite of the scenario described above. Whereas in that case the Baysean applicator wants the experimental hypothesis (theory x) to be true, Wagenmakers et al. want the experimental hypothesis (psi) to be false. However, the important principles are the same in both cases.
Serendipitously, we have a member of the parapsychological community, Jessica Utts, who is an expert in Baysean statistics. Along with a statistician colleague from her university, she coauthored Bem's response to Wagenmakers (Bem, Utts, & Johnson, 2011). Wagenmakers et al. made two Baysean arguments aiming to show that Bem's significant results were really nonsignificant, but space limitations precluded Bern et al. from responding to both in detail. Bern et al. chose to focus on Wagenmakers et al.'s second argument, which is the one the latter put the most weight on. The arguments on both sides are rather complicated, and I won't review them here, except to note that the questionable premise of Wagenmakers et al.'s case is that if psi is real, the effects must be very large (in statistical jargon, this means a high effect size). Although I understand why Bem et al. aimed their fire where they did, I think Wagenmakers et al.'s first argument also needs to be addressed. For one thing, because it is simpler, readers of the debate are more likely to fully understand it and relate to it. Thus, soon after I saw Wagenmakers et al.'s reply, I wrote up a critique of their first argument. As I soon realized that my chances of getting my remarks published in the JPSP are virtually zero, I decided to take advantage of my position as editor of the JP to copy them here. Because the points are more philosophical than statistical, I believe I can say something useful even though I'm not a statistician. First, I will reproduce the essential passages in Wagenmakers et al.'s first Baysean argument:
As a first reason, consider that Bern ... acknowledges that there is no mechanistic theory of precognition.... This means, for instance, that we have no clue about how precognition could arise in the brain.... Note that precognition conveys a considerable evolutionary advantage (Bem, 2011), and one might therefore assume that natural selection would have led to a world filled with powerful psychics.... This is not the case, however.... The believer in precognition may object that psychic abilities, unlike all other abilities, are not influenced by natural selection. But the onus is squarely on the believer in psi to explain why this should be so. Second, there is no real-life evidence that people can feel the future (e.g., nobody has ever collected the $1 million available for anybody who can demonstrate paranormal performance under controlled conditions...). To appreciate how unlikely the existence of psi really is, consider the facts (a) casinos make profit, and (b) casinos feature the game of French roulette.... In this context, even small effects of psi result in substantial payoffs. For instance, suppose a player with psi can anticipate the correct color in 53.1% of cases--the mean percentage correct across participants for the erotic pictures in Bem's (2001)Experiment 1.... After accounting for the house advantage ... the probability that the psi player will win 1 million [euro] ... equals 48.6%.... This means that ... the expected profit ... equals $485,900.... Clearly, Bem's psychic could bankrupt all casinos on the planet.... This analysis leaves us with two possibilities. The first possibility is that ... psi effects are not operative in casinos, but they are operative in psychological experiments on erotic pictures. The second possibility is that the psi effects are either nonexistent or else so small that they cannot overcome the house advantage. Note that in the latter case, all of Bem's (2011)experiments overestimate the effect ... the above reasons motivate us to assign our prior belief in precognition a number very close to zero. For illustrative purposes, let us set P([H.sub.1]) [probability that the psi hypothesis is true] = [10.sup.-20], that is, 00000000000000000001. (Wagenmakers et al., 2011, p. 428)
Wagenmakers et al. offer both theoretical and empirical arguments to justify their figure of [10.sup.-20] for the prior probability of precognition. Their theoretical argument is that there is no "mechanistic theory" of precognition, by which they mean mechanisms sanctioned by mainstream science. Their two examples are brain processes and evolution. Fifty years ago we had no clue where many cognitive processes were localized in the brain, but I don't recall any scientist arguing that therefore these processes didn't exist, and it would be ludicrous for them to do so. There is every reason to assume that psi is at least mediated by the brain, even if one assumes we have immaterial minds. Eventually we will learn what role the brain plays in the production of psi effects, just as we have done (and will do) for other cognitive manifestations. As for evolution, Wagenmakers et al. argue in effect that precognition can't be real because if it were, it would have been favored by evolution and we "would have a world filled with powerful psychics." But just because Wagenmakers et al. can't think of a logical reason why precognition wouldn't be favored by evolution doesn't mean there isn't one. My own speculation is that precognition was selected against because it is inherently less reliable than normal sensory mechanisms, and it had to be suppressed by the evolution of the brain for these mechanisms to function properly; the psi ability that remains is vestigial. But let's say for the sake of argument that whatever precognition remains cannot be accounted for within an evolutionary framework. In that case, it is tantamount to a refutation of evolutionary theory, or more accurately, a refutation of its universal applicability. Wagenmakers et al. would have us reject the evidence for precognition because of its inconsistency with evolutionary theory, which we "know" to be true. How do we know evolutionary theory is true? A major reason is that there is no evidence against it. The reason there is no evidence against it is that the experiments ostensibly providing such evidence (e.g., Bem's) can't survive a Baysean analysis for which the virtual truth of the theory of evolution is a premise. Unless Wagenmakers et al. want to maintain that some theories can't be refuted by new evidence (a patently unscientific position), their argument is an example of circular logic. In any case, the application of this principle would have the practical effect of protecting all "well-established" theories from refutation, as I discussed above.
The second part of Wagenmakers et al.'s argument for their prior probability of precognition is empirical. They use the lack of big winners in casinos as the prime example to make their point. Radin and Rebman (1998) offer a cogent explanation of why psi does not lead to monetary success in this psi-inhibitory environment. I won't dwell on this matter, because it is really a side issue. The key claim is the more general one that "there is no real-life evidence that people can feel the future." If this claim is to justify their [10.sup.-20] probability, Wagenmakers et al. must assume, with a certainty corresponding to the inverse of [10.sup.-20], that all the myriad examples of precognition experiences (mostly dreams) in the real world (which far exceed what might or might not be going on in casinos) have conventional scientific explanations. How can they justify this assumption? If they appeal to their theoretical arguments, we are back to the problems discussed above. They surely can't prove empirically that all these cases have conventional explanations. If they still want to invoke their [10.sup.-20] probability, they have to admit that it is nothing more than personal prejudice.
I will close this editorial by combining my points about physics theories and Baysean statistics as they relate to psi. In my opinion, the strongest philosophical or theoretical reason mainstream scientists and philosophers reject psi is that they see it as incompatible with our well-grounded theories of the physical universe. I expect many scientific critics of parapsychology would attach an astronomically low a priori probability to psi simply on the grounds of its incompatibility with these theories. Such an action would strike me as profoundly misguided. The fact that physicists have been unable to come up with a credible "theory of everything" in over a hundred years of inquiry (to paraphrase critics of parapsychology) strongly suggests that the totality of events in the universe are caused not by a single fundamental process, but rather by multiple such processes that are incommensurate with one another. Therefore, I conclude this editorial with the bold statement that in estimating the a priori probability of psi (or any other fundamental class of events in nature), the weight that should be given to its alleged incompatibility with currently accepted theories of physics, either individually or collectively, is zero.
Bem, D.J., & Honorton, C. (1994). Does psi exist? Replicable evidence for an anomalous process of information transfer. Psychological Bulletin, 115, 4-18.
Bem, D.J. Utts, J., & Johnson, W. O. (2011). Must psychologists change the way they analyze their data? Journal of Personality and Social Psychology, 101, 716-719.
Braude, S. E. (1986). The limits of influence: Psychokinesis and the philosophy of science. New York: Routledge and Kegan Paul.
Carey, B. (2011, January 6). Journal's paper on ESP expected to prompt outrage. New York Times. Retrieved from http://community.nytimes. com/comments/www.nytimes.com/2011/01/06/science/06esp. html
Hofstadter, D. (2011, January 6). A cutoff for craziness. New York Times. Retrieved from http://www.nytimes.com/roomfordebate/2011/01/06/ the-esp-study-when-science-goes-psychic/a-cutoff-for-craziness.
Lehrer, J. (2010, December 13). The truth wears off: Is there something wrong with the scientific method? New Yorker. Retrieved from www.newyorker.com/reporting/2010/12/13/101213fa_fact_ lehrer?printable=true
Radin, D. I., & Rebman, J. M. (1998). Seeking psi in the casino. Journal of the Society for Psychical Research, 62, 193-219.
Wagenmakers, E.J., Wetzels, R., Boorsboom, D., & van der Haas, M. (2011). Why psychologists must change the way they analyze their data: The case of psi. Journal of Personality and Social Psychology, 100, 426-433.
Walker, E. H. (1975). Foundations of paraphysical and parapsychological phenomena. In L. Oteri (Ed.), Quantum physics and parapsychology (pp. 544-568). New York: Parapsychology Foundation.
|Gale Copyright:||Copyright 2011 Gale, Cengage Learning. All rights reserved.|