“Truthiness” & “sciency” in academy


4th of Obtober Science published an article which was described as a “sting operation” against open access journals. The aim was to create an fake article with obvious methodological and scientific problems and then send the article to a number of open access journals to see which would publish the article. The problem with the study is that the researcher (John Bohannon, biologist and science journalist) knew that the oa-journals the article was sent to were predatory open access journals and that he did not have a control group of traditional toll access journals.

The article has created a lot of discussions and even some well-written satirical blog posts, e.g. Dr. Mike Taylors Anti-tutorial: how to design and execute a really bad study. Mike Taylor is  paleontologist in the Department of Earth Sciences at the University of Bristol, UK.

Taylor writes”…It has that sciencey quality. It discusses methods. It has supplementary information. It talks a lot about peer-review, that staple of science. But none of that makes it science. It’s a maze of preordained outcomes, multiple levels of biased selection, cherry-picked data and spin-ridden conclusions. What it shows is: predatory journals are predatory. That’s not news.”

Bohannon’s article does not help us to form an objective opinion on oa journals because of its flaws and that it is trying to describe something as the truth but instead gives  “truthiness”, as Gary F. Daught describes it, a face. “Truthiness” is defined in Wikipedia  as “a quality characterizing a ‘truth’ that a person making an argument or assertion claims to know intuitively ‘from the gut’ or because it ‘feels’ right without regard to evidence, logic, intellectual examination, or facts”.

This is not the first time the peer review process is under scrutiny. In 1996 Alan Sokal tested Social Text, a highly regarded journal in Sociology, and in just some weeks ago it was reported that some serbian researchers had tested peer review process. The article was not even up to par but it had also cited among others B. Sagdiyev (Borat), A. S. Hole and Michael Jackson.

Next time someone conducts a study like this it would be great if they made a serious try: tested both highly regarded oa journals and ta journals, and then drew conclusions which the results allow and not spinn them into someting unrecognizable.

Pieta Eklund