Posted by Curt on 11 January, 2015 at 6:43 pm. 2 comments already!



The worst paper ever published has competition. I was going to mock this, but it has all rather slipped beyond the Plains of Derision and sunk in a parallel universe. Researcher Jose Duarte is flummoxed, he simply can’t explain why a paper so weak was written, but moreso why it was ever published, and why everyone associated with it is not running for cover.  It’s not so much about the predictable flaws, biased questions, and mindless results, it’s now about why UWA, The Uni of Bristol, PLOS, and the Royal Society are willing to wear any of the reputational damage that goes with it.

Lewandowsky, Gignac and Oberauer put out a paper in 2013 which was used to generate headlines like “Climate sceptics more likely to be conspiracy theorists”. The data sample is not large, but despite that, it includes the potential Neanderthal, as well as a precocious five year old and some underage teenagers too. The error was reported on Lewandowsky’s blog over a year ago by Brandon Shollenberger, then again by Jose Duarte in August 2014. Nothing has been corrected. The ages are not just typos, they were used in the calculations, correlations and conclusions. The median age was 43 but the mean age was a flaming neon 76. One wildly old person in the data skewed the correlation for age with nearly everything:

That one data point – the paleo-participant – is almost single-handedly responsible for knocking out all the correlations between age and so many other variables. If you just remove the paleo-participant, leaving the minors in the data, age lights up as a correlate across the board. Further removing the kids will strengthen the correlations.

Duarte remarks he would not sleep if he knew his work had a problem as major as this:

I don’t understand how anyone could let a paper just sit there if they know the data is bad and specific claims in the paper are false. No credible social psychologist would simply do nothing upon discovering that there were minors in their data, or a five-digit age. I’d be running to my computer to confirm any report that claims I’d made in a peer-reviewed journal article rested on bad data, fake participants, etc. I wouldn’t be able to sleep if I knew I had something like that out there, and would have to retract the paper or submit a corrected version. You can’t just leave it there, knowing that it’s false.

In any case, something is very wrong here. The authors should explain how the 32,757-year-old got into their data. They should explain how minors got into their data. They should explain why they did nothing for more than a year. This is a very simple dataset – it’s a simple spreadsheet with 42 columns, about as simple as it gets in social science. It shouldn’t have taken more than a few days to sort it out and run a correction, retraction, or whatever action the circumstances dictated. These eight purported participants allowed them to claim that age wasn’t a factor. It allowed them to focus on the glitzy political stuff, allowed them to focus on finding something negative to pin on conservatives.

They don’t tell you until late in the paper that conservatism is negatively correlated with belief in conspiracies – the exact opposite of what they claimed in the earlier scam paper that APS helped promote. Also note that we already know from much higher quality research that Democrats are more likely than Republicans to believe in the moon hoax, though it’s a small minority in both cases (7% vs. 4%), and that Democrats endorse every version of the JFK conspiracy at higher rates. I think some journals might be unaware that the pattern of these conspiracy beliefs across the left-right divide is already well-documented by researchers who have much higher methodological standards – professional pollsters at Gallup, Pew, et al. We don’t need junky data from politically-biased academics when we already have high-quality data from professionals.

Duarte describes how unusual this kind of simple mistake is. The software used to run the survey specifically asks researchers to set age boundaries and it automatically checks to make sure respondents fit within the accepted range.

What’s also exceptional is that Lewandowsky, Gignac and Oberauer managed to delete and filter out more than a quarter of their participants, yet somehow left in the person born in 30,000 BC. They ruled out one in four responses yet allowed minors to be included, which they don’t have ethical approval for, and given the political conclusions, don’t have a scientific reason to include either.

Duarte wonders where the accountability is:

Read more

0 0 votes
Article Rating
Would love your thoughts, please comment.x