JoNova:
Richard Tol has an excellent summary of the state of the 97% claim by John Cook et al, published in The Australian today.
It becomes exhausting to just list the errors.
As Tol explains, the Cook et al paper used an unrepresentative sample, can’t be replicated, and leaves out many useful papers. The study was done by biased observers who disagreed with each other a third of the time, and disagree with the authors of those papers nearly two-thirds of the time. About 75% of the papers in the study were irrelevant in the first place, with nothing to say about the subject matter. Technically, we could call them “padding”. Cook himself has admitted data quality is low. He refused to release all his data, and even threatened legal action to hide it. (The university claimed it would breach a confidentiality agreement. But in reality, there was no agreement to breach.) As it happens, the data ended up being public anyhow. Tol refers to an “alleged hacker” but, my understanding is that no hack took place, and the “secret” data, that shouldn’t have been a secret, was left on an unguarded server. The word is “incompetence”, and the phrase is “on every level”.
The hidden timestamps of raters revealed one person rated 675 abstracts in 72 hours, with much care and lots of rigor, I’m sure. It also showed that the same people collected data, analyzed results, collected more data, changed their classification system, and went on to collect even more data. This is a hopelessly unscientific process prone to subjective bias and breaches the most basic rules of experimental design. Tol found the observations changed with each round, so the changes were affecting the experiment. Normal scientists put forward a hypothesis, design an experiment, run it, and then analyze. When scientists juggle these steps, the results influence the testing. It’s a process someone might use if they wanted to tweak the experiment to get a specific outcome. We can’t know the motivations of researchers, but there is a reason good scientists don’t use this process.
My problem with taking the Cook paper seriously is that it is so wholly, profoundly, unscientific from beginning to end that it’s hard to muster any mental effort to unpack a pointless study that will never tell us anything about the atmosphere on Earth.
As I have said from the start, studies on consensus are a proxy for funding, not a proxy for Truth — and funding is as monopolistic as ever. The government gives grants to researchers to find a crisis, and we get what we paid for. If we pour $30 billion into finding reasons to fear CO2, and $0 into finding holes with that theory, it is entirely predictable that we will get 90+ percent of papers that support the theory. There are plenty of ways to write irrelevant, flawed, unrelated, or repetitive material. (What’s remarkable is that there are so many skeptical papers that manage to get written without much funding and get past the gatekeepers in “peer review”.)
But many harried, busy people, untrained in logic, seem to find these consensus papers compelling, so it is worth pointing out the flaws.
The most important issue here is not the inept study authors (who are beyond help) but the response of the University of Queensland, and the editors of Environ. Res. Lett.. Richard Tol has informed the journal of the problems and suggested his reply should be published and the paper should be retracted. Editor Daniel Kammen chose not to publish Tol’s analysis, though he sent it to reviewers. Peer review has become so farcical, one ERL reviewer suggested Tol should rewrite his submission and should conclude that Cook’s paper was an example of “exemplary scientific conduct”. That says a lot about scientific standards at ERL.
