Climatic Research Unit data is valid – don’t let the sceptics tell you otherwise

RT @leftfootfwd: Climatic Research Unit data IS valid – don’t let the sceptics tell you otherwise:- http://is.gd/51R5J

RT @leftfootfwd: Climatic Research Unit data IS valid – don’t let the sceptics tell you otherwise: http://is.gd/51R5J

hahahahahahahahahahahahahahahah!!!!!!!!!!!!!!!!!!!!!!

Papers presented for publication are subject to peer review. Experts are not peer-reviewed. For a rather more significant revelation of internal documents, a recent disclosure in a Federal law suit in the USA showed that the major contrarian campaign knew very early on that their arguments did not stand up. Here is the link:

http://www.nytimes.com/2009/04/24/science/earth/24deny.html?_r=1&hpw

A lot more significant than the kind of background chatter in the CRU emails.

'Climategate' filleted. http://bit.ly/6iaxWt

Forlornehope – fair point on the ‘peer reviewed’ thing. I hope it was obvious what I meant, i.e. that these are REAL climate scientists with papers on climate science that have been peer reviewed as opposed to blogs on pseudo science written by non-climate scientists.

Fantastically constructive comment from Gracie the Collie. Well done on engaging with this debate in such a mature manner.

Joss Garman – so you think it’s OK not to give raw data to people who request it and then smear and lie about people who don’t share their “opinion” on the reasons for global warming (which as you know isn’t happening – oh sorry I should have said climate change).

By forcing poor countries to accept this nonsense on global warming is just cruel but then I suppose you people don’t need to grow a lot of crops in Kensington.

And another supremely constructive contribution from Anon there, once again accusing others of smearing people while doing the same thing himself. And the idea, the very idea that the Right-wingers and the deniers are claiming to speak on behalf of poor countries is too contemptible for words.

There are several things to note about this “leak”.

The first thing is that they are all text files. That is not surprising because internet mail is exchanged as text. However, depending on the mail server, email may be stored in an RDMS database, or as individual files, or some proprietary flat file system (or maybe a combination of all three). Email clients’ message stores will be one of these mechanisms too. However, the general standard for emails stored as individual text files is that they have the extension eml, yet these have the extension txt. The extension indicates to me that they were created on a Windows machine (where the erroneous idea that extensions have to be three characters still persists). It is also the extension used by the standard text editor on Windows (Notepad).

As mail is passed between mail servers (at least two, the SMTP server of the sender and the SMTP server of the receiver) headers are added. Further headers are added to give the message a unique ID and also to give the ID of the message replied to (which allows threading by mail clients). Also MIME headers are added so that attachments (as additional bodies) can be added to the email. Usually an email message will have no less than 20 lines of headers.

Yet we find that the CRU emails only have To, From, Date and Subject headers (no Message-ID, no Received, no Return-Path, no References, no In-Reply-TO and no MIME information). Since all email messages will have far more than these four headers it is incontrovertible that these emails have been edited – if only to remove the extra headers, but who knows what else was changed. As I mentioned above, the editing probably was carried out using Notepad.

Without the headers that were removed it is difficult to see any threading in the emails, but a cursory inspection shows that they appear to be a jumble of emails, 1073 of them. How many emails do you think that the CRU mail server has archived – and remember the “leaked” emails are from 7 Mar 1996 to 12 Nov 2009? My guess is hundreds of thousands, if not millions. Yet somehow only 1073 of them were leaked. Even if the emails came from a user’s mailbox, do you think that a user would have saved just 1073 emails in a period of 13 1/2 years? I have tens of thousands of emails on my hard disk.

I know from the time I worked in a university that much of my inbox was in the form “going for a drink on friday?”, yet I couldn’t find any messages like that.

Now imagine you are a hacker and you have broken the security of some outfacing server and you see the storage folders of the email server, or perhaps a user’s mailbox. What would you do? Most likely you would copy everything. Then what would you do? Well since you have “dynamite” you would want to post this stuff as quickly as possible (partly to save your ass, so that you can remove the evidence from your machine, partly because you are so excited about the “dynamite”). So you would post everything as you found it to the internet.

That is not what has happened. Someone has carefully edited the emails to remove the extra headers (that admittedly get in the way of reading the emails when using Notepad, but most hackers would have an email client anyway), they have also removed attachments (why?) and they have sifted through them and given just a few emails, ones that tell a particular story. All of this takes time. Someone has been very methodical about this.

This is not the result opf some random hacker breaking into a web server, it is a concerted attack from someone who knows what to look for in the emails. The rather cack-handed editing of emails makes me think that it was not done by the hacker, but instead by whoever it is sifting the emails.

Anyway, one email caught my attention (1256353124.txt). First, bear in mind that the bane of the climate scientists is someone called Steve McIntyre who tried to “prove” that the hockey stick graph “was not true” (the US National Research Council convened to investigate McIntyre’s claims basically said that McIntyre was wrong, being publicly denounced like that is a bitter blow to any climate change denier). The following CRU email is about a preliminary paper posted on the public website, it reads:

I’m not thinking straight. It makes far more sense to have password-protection rather than IP-address protection. So, to access those pages

Mike

I am assuming that the steve mentioned here is one Steve McIntyre!

[...] this earlier post on Left Foot [...]

Shamik – Who did I smear?

Accusing people of things they haven’t done when you are a moderator on this blog gives you every advantage and is not fair.

And on the subject of poor countries your party has tried to bomb Iraq and Afganistan back to the stone age – and as a Labour voter don’t call me Right Wing – your New Labour party backed George Bush.

The people claiming that this is the end of the AGW debate are foolish but so are those saying that CRU data is completely valid. We don’t know that and the information in this leak casts things in more doubt. At the very least, these e-mails show concerted efforts to get around the FOIA processes to keep their data from being reviewed by outsiders and suggestions that the peer review process was being gamed. This lack of transparency is going to cast doubt in peoples’ minds and I can’t understand why most on the left don’t have an issue with it. How do we know that HADCRUT3 database is accurate? We haven’t seen any of the code or procedures in place for the adjustments. It can’t be extremely off (like the right would like people to think) since we have GISS to compare to, but would it shock me if the way they pick sites to include and how they fill in missing was not exactly scientific? Not at all after seeing this.

Specific comments to the original author’s piece.

“These climate deniers seem to think the CRU, like the Royal Society, NASA and the US National Academy of Sciences, are agents of a clandestine global movement against the truth. ”

Way to smear whole groups of people. Keep using the right’s tactics…that’ll win people over.

“For the best reaction to this hyped up story making its way around the right wing echo chamber – see Realclimate’s reaction. (This is a website run by some of the world’s pre-eminent climate scientists.) Their team of peer-reviewed climate experts conclude: ”

You fail to mention that RC is run by some of the same people captured in these e-mails. They are probably not the best place to go to get an unbiased reaction. Not that I’m claiming sites like WUWT are either but saying RC has the best reaction is showing your bias.

“Today Professor Phil Jones, the director of the CRU, whose emails are at the centre of the story, said he wanted to put the record straight, saying he saw the hack “caused a great deal of ill-informed comment”, and Kevin Trenberth, another leading climate change scientist whose private emails were also among those stolen, said the leaks may have been aimed at undermining next month’s global climate summit in Denmark”

There’s a very easy way this could have all been avoided. Release the data. We don’t need all of these e-mails, but for every peer reviewed paper, everything that would be needed to allow others to reproduce the paper’s results should be public. This would allow all people, no matter what their biases are, to make informed conclusions. If their papers and conclusions are correct, the data will show it and they truly won’t have to worry about anyone except the Fox News crowd. Until this happens, I’ll remain skeptical of the exact conclusions of climate science. Especially around the true uncertainty in our understanding about past climate and future projections.

Sorry Will I was still laughing at http://di2.nu/foia/HARRY_READ_ME-30.html and I came across your blog you lost me when you said “is alight with what David Cameron’s old University friend James Delingpole hysterically asserts” it’s always tribal with you lot. This will be spun in to “fake but accurate”

grace the collie,

Can you see the irony in accusing others of being ‘always tribal’ and then immediately referring to them as ‘you lot’?

http://www.leftfootforward.org/2009/11/climatic-research-unit-data-is-valid-dont-let-the-sceptics-tell-you-otherwise/ #CRU #climatechange

The data is NOT valid.

Told you they were fiddling the data.

Forget the emails. Anyone who writes computer programs in a high level language, such as C, puts in comments to remind themselves of what the code is dong and to allow future programmers to make changes to the code.

There are two files normally generated, a “Source Code” file (where the comments are) and the compiled file or “Object” (.obj) file or .asm file.

(I’m a 1980′s Z80 man so this may be dated but you get the point)

The file, after compiling, has the comments and other stuff removed and it is basically (no pun there) the “computer code” for the machine to run on.

So the climate change programmer has left his comments here:

http://wattsupwiththat.com/2009/11/22/cru-emails-may-be-open-to-interpretation-but-commented-code-by-the-programmer-tells-the-real-story/

Personally since they don’t include the effects of the sun I can’t take the data seriously but I’m surprised that this terrific blog, being progressive as it claims, hasn’t picked up on this.

See also http://www.di2.nu/200911/21b.htm and http://bit.ly/4RMkXh and http://rupertsread.blogspot.com/2009/11/scandal-over-hacked-climate-science.html for useful posts discussing this.

Don’t know much about “irony” but when I see a blog entry entitled “Climatic Research Unit data is valid – don’t let the sceptics tell you otherwise” alarm bells go off. It’s already decided that this is the line and there is no need to question, don’t need to think and don’t even think at looking at the source code. The data is not valid.

Rupert Read – You have, deliberately or otherwise, missed the point here.

Let me spell it out for you. The software that is used by the Climate Change scientists to make their predictions and claims, is programmed incorrectly and cannot give valid data. Fact.

Not only that Rupert but smoke screens about emails are being used to cover this up. What concerns me is that you are not concerned about that software yourself. Why not?

Any claims you make regarding data about temperature rises cannot be substantiated by this software and are therefore an act of faith and not scientific proof.

The majority of people do not believe you are right Rupert and there is no proof you are. Is that way comments on your blog are blocked?

Talk about missing the point. The unknown part of climate change is what is climate sensitivity.

We know what a doubling in the atmosphere will do directly: 0.7oC rise (yes, really). What we don’t know is how the sum of secondary effects (positive and negative feedbacks) works out. If climate sensitivity is, say, 2oC, then we’ve not much of a problem. If it’s 6 oC then we’re in shit street.

What CRU is supposed to be working out is what is climate sensitivity. It’s the most important result we want. And if they’ve been hiding raw data, indulging in a lot of hand waving and in general pushing a line rather than doing their job, research, then we’ve all got really rather a large problem.

Tim – My point is not the temperature, increase or otherwise – my issue is with the reasons for it. Leave aside the bogus data being pedaled by the CRU there is NO EVIDENCE, bogus or otherwise that (man made) CO2 causes global warming.

I have to say; when George Monbiot says it looks bad, it’s got to be pretty damning. He’s calling for Phil Jones’ resignation and a re-evaluation of the evidence referred to in the emails. As for his assertion that the timing of this hack/leak with Copenhagen isn’t a coincidence, I’m sure he’s just being cynical. It obviously is, just like CRU’s discovery last week that temperatures will rise by 6oC by 2100.

Can I also suggest that, for an evidence-based blog, a link to the leaked/stolen material might be more appropriate than a link to a Guardian story?

http://www.eastangliaemails.com/index.php

For the latest on this, goto; http://www.opendemocracy.net/ourkingdom/rupert-read/real-scandal-in-hacked-climate-change-e-mails-controversy . [See also #cru on twitter]

Rupert – The real scandal is the fact the software is bogus so the evidence is incorrect and even though you know that to be true you persist to pedal this unproven myth.

George Monbiot in the Guardian seems more skeptical now than you are and since you hold public office as a councilor for the Green Party I find it very concerning that you seem not to be interested in this matter that affects our lives. You continually repeat the same (now) unproven stuff.

Monbiot seems angry about this I wonder why it doesn’t bother you. Your site says you: “specialise(s) increasingly in political and environmental philosophy”.

What philosophy includes denial of data when presented with it? Worrying.

@torybear One link for you http://bit.ly/6iaxWt

Check this out:
http://www.realclimate.org/index.php/archives/2009/11/the-cru-hack-context/ An update, to help understand in techie detail why the CRU ‘scandal’ ain’t much of one

In answer to brave Anon E Mouse: Monbiot called on Phil Jones to resign to take resposibility for ethical lapses revealed by the emails, NOT because of any fundamental weakness in the climate science. Ditto for Monbiot’s call for re-analysis of some data: precisely so that the climate sceptics can be openly rebutted, NOT because there is any reason to doubt the actual data or its standard interpretation.
Monbiot is not ‘sceptical’ at all – that’s just another of your crude distortions. He is simply more extremely vexed than I am with the ethical lapses revealed in the emails. My own view is that it is premature to call for Jones’s resignation: let the independent review report first.

Rupert Read – Thanks for calling me brave. Not sure why a person who shares the majority opinion in this country should be called ‘brave’ but I accept your compliment to me in the spirit it was sent.

My point with Moonbat was to illustrate the fact that (amazingly) he is less dogmatic than you seem to be and unlike your postings both here and on your blog (where comments are not allowed – surprise surprise Rupert) he disagrees with your Stalin like tendencies to prevent opinion and free speech that differs from your own.

The fact you don’t believe that claims made using bogus software and that you believe the suppression and manipulation of data is a fair way to behave not a serious matter makes me wonder just how serious you are about this subject.

Worrying.

Joss – you had me at “anti-scientific”

I suggest you read this web site in depth

You’ll find the web-site is riddled with scientific analysis of this issue.

Do you think you do your cause any good by labelling people “anti-scientific”, “blowhards” or “head bangers”?

Personally I’m all for looking after the planet, but I hate the way extremists are following these two routes:

- Telling us that there is scientific consensus (and that if we are sceptical about anything in any way, that we are “headbangers” etc)

- Using the Green movement to unleash political ideals on the world (you know, the sort of Lisbon Treat gone global stuff)

I object to being labelled anti-scientific.

From Billy Blofeld BSc Chemical Engineering (Hons).

[...] There is something to be said for claiming to be an evidence based blogger. You can use “evidence” to prove whatever the hell you like. [...]