JoNova:
The BOM Technical Advisory Forum report is out. Finally there is the black and white admission that the BOM “adjusted” dataset cannot be replicated independently, has not been replicated by any other group, and even more so, that the BOM will not provide enough information for anyone who wants to try.
As we have said all along, the all new ACORN wonder-data was not created with the scientific method. Adjustments to Australian temperature data were done with a black box mystery technique that only the sacred guild at the BOM are allowed to know. Far from being published and peer reviewed, the methods are secret, and rely on — in their own words — a “supervised process” of “expert judgment” and “operator intervention”. In other words, a BOM employee makes their best guess, ruling in or out the “optimal” choices, making assumptions that are not documented anywhere.
It’s a “trust us” approach. Would we let an ASX company audit their own books? Would you buy shares in such a company, or let it inform national policy on billion dollar schemes?
Here is the entire section on replication from page 9 and 10 (below). This is what any semi-skilled PR operative would write if they were trying to justify keeping their methods secret. My translations included.
Ability to reproduce findings
Only BOM staff are smart enough to understand “scientifically complex” thermometers (this is something that engineers, astrophysicists, aeronautics experts and physicists would not be able to do, is that what they are saying?):
The Forum considers that the algorithms and processes used for adjustment and homogenisation are scientifically complex and a reasonably high level of expertise is needed to attempt analysis of the ACORN-SAT data. For this reason the Forum had some queries about the ability to reproduce findings by both experts and members of the public.
Thinly veiled put-down coming:
It would be useful for the Bureau to provide advice about the necessary level of end-user expertise (notwithstanding a likely tendency for end-users to feel qualified to attempt such an analysis).
It might be more “useful” if the BOM staff provided their personal exam results in fluid dynamics, heat flow, mathematics and statistics. Or even just their resumes? We’ll find people who outscored them. OK?
Here’s the statement that no one has replicated the Australian temperature set:
The Forum felt that reproducing the Bureau’s ACORN-SAT daily analyses would be a very onerous task, and advice was supplied at the Forum meeting day that, while international groups have provided independent data homogenized at the monthly time-scale, no groups other than the Bureau are known to have attempted to produce or analyse an homogenized daily data set for Australia. One option would the Bureau to work with local and international collaborators with the appropriate skill set to broadly assess the ACORN-SAT daily homogenisation methodologies.
Here is the statement that no one can replicate them because only the BOM knows how it was done (my bolding):
The Forum noted that the extent to which the development of the ACORN-SAT dataset from the raw data could be automated was likely to be limited, and that the process might better be described as a supervised process in which the roles of metadata and other information required some level of expertise and operator intervention. The Forum investigated the nature of the operator intervention required and the bases on which such decisions are made and concluded that very detailed instructions from the Bureau are likely to be necessary for an end-user who wishes to reproduce the ACORN-SAT findings. Some such details are provided in Centre for Australian Weather and Climate Research (CAWCR) technical reports (e.g. use of 40 best correlated sites for adjustments, thresholds for adjustment, and so on); however, the Forum concluded that it is likely to remain the case that several choices within the adjustment process remain a matter of expert judgment and appropriate disciplinary knowledge.
The process can’t be “automated” — which means it can’t be described by a set of rules other people, or other computers could follow. It’s a bit of a red herring: skeptics have never demanded “automation”. We just want explanations. The crux of science is replication, not automation. If ad hoc judgements were part of the process, they need to be recorded and their impact on the numbers included in the processing from raw data to final product. Justifications can come afterwards; let’s first establish what happened.
Ooooooo!!!!
They must be smart!
They are the only ones who can understand both homogenization and higher math!
Leave it to them.
Just trust them.
OK, sarcasm off.