Why you should publish the repeat studies of Bem

The new scientist informed us today that the Journal of Personality and Social Psychology rejected a paper by Richard Wiseman, Chris French and Stuart Ritchie. They did what every decent researcher in the field should do, when they heard about the results of a paper by Daryl Bem presenting evidence for precognition: They went to their respective labs, tried to repeat the experiments and when they failed, they wanted to publish the negative results anyway. But then the editor from the JPSP stepped in, and refused to publish their data without even sending it out for peer review. And this is where the real problem starts.

I will not repeat the arguments against the original study by Bem, as Nicholas Gauvrit has done an excellent job to present the problems with this study in writing and as a podcast. And I will not insist, that the initial study should not have been published as it was, given the doubtful statistical methodology.

What I rather would like to stress once more, is the refusal of the Journal of Personality and Social Psychology to publish the repeat study. If you look at the original paper, the correct title (or at least a subtitle) should have included the words “a pilot study”. Bem conducted a large amount of experiments (or at least he published them, he might have conducted even more) in a fishing expedition. So the decent journal would be more than happy to publish the studies, that are trying to repeat these results – because THESE experiments, are the ones that really test the hypotheses, that were formulated after the initial pilot study.

So the refusal of the JPSPs refusal to publish these results, really shows, that they crossed over to the dark side. They just wanted the publicity of the initial Bem study, they did not care about the validity of the study or the statistical results (a lot of comments criticized the publication last year) and they don’t want their papers to be tarnished by negative repeat experiments.

This is one of the most obvious cases of a publication bias: negative results do not get published. And the comment by the JPSP, that they might publish a meta-analysis, makes it even worse. If you only publish the positive results, your meta-analysis is as worthless as the initial publications – GIGO. So either you publish negative results as well, or you should consider rebranding your publication to Journal of Junk Research – formerly Journal of Personality and Social Psychology.

Advertisements

About brainfisch

Some unnecessary musings on life, the universe and everything.
This entry was posted in brainscience, ideas, Skeptic, Uncategorized and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s