Some Final Thoughts on The NASA Temperature Restatement

I got a lot of traffic this weekend from folks interested in the US historical temperature restatement at NASA-GISS.  I wanted to share to final thoughts and also respond to a post at RealClimate.org (the #1 web cheerleader for catastrophic man-made global warming theory).

  1. This restatement does not mean that the folks at GISS are necessarily wrong when they say the world has been warming over the last 20 years.  We know from the independent source of satellite measurements that the Northern Hemisphere has been warming (though not so much in the Southern Hemisphere).  However, surface temperature measurements, particularly as "corrected" and aggregated at the GISS, have always been much higher than the satellite readings.  (GISS vs Satellite)  This incident may start to give us an insight into how to bring those two sources into agreement. 
  2. For years, Hansen's group at GISS, as well as other leading climate scientists such as Mann and Briffa (creators of historical temperature reconstructions) have flaunted the rules of science by holding the details of their methodologies and algorithm's secret, making full scrutiny impossible.  The best possible outcome of this incident will be if new pressure is brought to bear on these scientists to stop saying "trust me" and open their work to their peers for review.  This is particularly important for activities such as Hansen's temperature data base at GISS.  While measurement of temperature would seem straight forward, in actual fact the signal to noise ration is really low.  Upward "adjustments" and fudge factors added by Hansen to the actual readings dwarf measured temperature increases, such that, for example, most reported warming in the US is actually from these adjustments, not measured increases.
  3. In a week when Newsweek chose to argue that climate skeptics need to shut up, this incident actually proves why two sides are needed for a quality scientific debate.  Hansen and his folks missed this Y2K bug because, as a man-made global warming cheerleader, he expected to see temperatures going up rapidly so he did not think to question the data.  Mr. Hansen is world-famous, is a friend of luminaries like Al Gore, gets grants in quarter million dollar chunks from various global warming believers.  All his outlook and his incentives made him want the higher temperatures to be true.  It took other people with different hypotheses about climate to see the recent temperature jump for what it was: An error.

The general response at RealClimate.org has been:  Nothing to see here, move along.

Among other incorrect stories going around are that the mistake was due
to a Y2K bug or that this had something to do with photographing
weather stations. Again, simply false.

I really, really don't think it matters exactly how the bug was found, except to the extent that RealClimate.org would like to rewrite history and convince everyone this was just a normal adjustment made by the GISS themselves rather than a mistake found by an outsider.  However, just for the record, the GISS, at least for now until they clean up history a bit, admits the bug was spotted by Steven McIntyre.  Whatever the bug turned out to be, McIntyre initially spotted it as a discontinuity that seemed to exist in GISS data around the year 2000.  He therefore hypothesized it was a Y2K bug, but he didn't know for sure because Hansen and the GISS keep all their code as a state secret.  And McIntyre himself says he became aware of the discontinuity during a series of posts that started from a picture of a weather station at Anthony Watts blog.  I know because I was part of the discussion, talking to these folks online in real time.  Here is McIntyre explaining it himself.

In sum, the post on RealClimate says:

Sum total of this change? A couple of hundredths of degrees in the US
rankings and no change in anything that could be considered
climatically important (specifically long term trends).

A bit of background - surface temperature readings have read higher than satellite readings of the troposphere, when the science of greenhouse gases says the opposite should be true.  Global warming hawks like Hansen and the GISS have pounded on the satellite numbers, investigating them 8 ways to Sunday, and have on a number of occasions trumpeted upward corrections to satellite numbers that are far smaller than these downward corrections to surface numbers. 

But yes, IF this is the the only mistake in the data, then this is a mostly correct statement from RealClimate.org..  However, here is my perspective:

  • If a mistake of this magnitude can be found by outsiders without access to Hansen's algorithm's or computer code just by inspection of the resulting data, then what would we find if we could actually inspect the code?  And this Y2K bug is by no means the only problem.  I have pointed out several myself, including adjustments for urbanization and station siting that make no sense, and averaging in rather than dropping bad measurement locations
  • If we know significant problems exist in the US temperature monitoring network, what would we find looking at China? Or Africa?  Or South America.  In the US and a few parts of Europe, we actually have a few temperature measurement points that were rural in 1900 and rural today.  But not one was measuring rural temps in these other continents 100 years ago.  All we have are temperature measurements in urban locations where we can only guess at how to adjust for the urbanization.  The problem in these locations, and why I say this is a low signal to noise ratio measurement, is that small percentage changes in our guesses for how much the urbanization correction should be make enormous changes (even to changing the sign) of historic temperature change measurements.

Here are my recommendations:

  1. NOAA and GISS both need to release their detailed algorithms and computer software code for adjusting and aggregating USHCN and global temperature data.  Period.  There can be no argument.  Folks at RealClimate.org who believe that all is well should be begging for this to happen to shut up the skeptics.  The only possible reason for not releasing this scientific information that was created by government employees with taxpayer money is if there is something to hide.
  2. The NOAA and GISS need to acknowledge that their assumptions of station quality in the USHCN network are too high, and that they need to incorporate actual documented station condition (as done at SurfaceStations.org) in their temperature aggregations and corrections.  In some cases, stations like Tucson need to just be thrown out of the USHCN.  Once the US is done, a similar effort needs to be undertaken on a global scale, and the effort needs to include people whose incentives and outlook are not driven by making temperatures read as high as possible.
  3. This is the easiest of all.  Someone needs to do empirical work (not simulated, not on the computer, but with real instruments) understanding how various temperature station placements affect measurements.  For example, how do the readings of an instrument in an open rural field compare to an identical instrument surrounded by asphalt a few miles away?  These results can be used for step #2 above.  This is cheap, simple research a couple of graduate students could do, but climatologists all seem focused on building computer models rather than actually doing science.
  4. Similar to #3, someone needs to do a definitive urban heat island study, to find out how much temperature readings are affected by urban heat, again to help correct in #2.  Again, I want real research here, with identical instruments placed in various locations and various radii from an urban center  (not goofy proxys like temperature vs. wind speed -- that's some scientist who wants to get a result without ever leaving his computer terminal).  Most studies have shown the number to be large, but a couple of recent studies show smaller effects, though now these studies are under attack not just for sloppiness but outright fabrication.  This can't be that hard to study, if people were willing to actually go into the field and take measurements.  The problem is everyone is trying to do this study with available data rather than by gathering new data.

Postscript:  The RealClimate post says:

However, there is clearly a latent and deeply felt wish in some sectors for the whole problem of global warming to be reduced to a statistical quirk or a mistake.

If catastrophic man-made global warming theory is correct, then man faces a tremendous lose-lose.  Either shut down growth, send us back to the 19th century, making us all substantially poorer and locking a billion people in Asia into poverty they are on the verge of escaping, or face catastrophic and devastating changes in the planet's weather.

Now take two people.  One in his heart really wants this theory not to be true, and hopes we don't have to face this horrible lose-lose tradeoff.  The other has a deeply felt wish that this theory is true, and hopes man does face this horrible future.  Which person do you like better?  And recognize, RealClimate is holding up the latter as the only moral man. 

Update:  Don't miss Steven McIntyre's take from the whole thing.  And McIntyre responds to Hansen here.

  • Josh

    "flaunted the rules"

    That would be flouted, good sir.

  • Reformed Republican

    There were however some very minor re-arrangements in the various rankings (see data). Specifically, where 1998 (1.24 ºC anomaly compared to 1951-1980) had previously just beaten out 1934 (1.23 ºC) for the top US year, it now just misses: 1934 1.25ºC vs. 1998 1.23ºC. None of these differences are statistically significant.

    So they admit that trumpeting 1998 as the hottest year on record was an exaggeration? Even without the correction, it was a dishonest statement.

  • Kat

    Is it bad that I'm skeptical on model based predictions because they can't tell me if it's going to rain tomorrow (with any sort of real accuracy) but they KNOW that in 2009 we will see the beginnings of a major warming period. It seems like skepticism is good for any science and there are real concerns, to bad it's become so politicized. I thought the only real truths in science and life is we all will die and pay taxes who know that global warming would be added to the list.

  • Joe Citizen

    This is the reason that people like you have such a hard time finding traction with reasonable people:

    "If catastrophic man-made global warming theory is correct, then man faces a tremendous lose-lose. Either shut down growth, send us back to the 19th century, making us all substantially poorer and locking a billion people in Asia into poverty they are on the verge of escaping, or face catastrophic and devastating changes in the planet's weather."

    This is simply not true. It is not necessary to "shut down growth" or to send us back to the 19th century. You people love to paint the Al Gore's of this world as fear-mongers, but you are being Al Gore on steroids here. Even he, the great fear monger has never made apocalyptic statements like this. The way out of this problem is through cleaner energy amd industrial technologies, in all its many guises. Its no wonder that so many people tend to see you skeptics as little more than paid, or unpaid advocates for dinosaur technologies.

    "Now take two people. One in his heart really wants this theory not to be true, and hopes we don't have to face this horrible lose-lose tradeoff. The other has a deeply felt wish that this theory is true, and hopes man does face this horrible future. Which person do you like better? And recognize, RealClimate is holding up the latter as the only moral man."

    Actally I like the third person better - the one you don't mention. The one who really hopes in his heart to figure out exactly what is going on with regard to the heat-balance of the earth. Some call such people "scientists". Once again, I find it instructive that you seem not to even consider the existence of such people.

    And a fourth person, who may be the same as the third - the person with great optimism who sees enormous opportunity in building the technologies of the future, for the benefit of the planet, and also for themselves.

    The ironies here are just mind-spinning. The "skeptics" lambaste the realists for fearmongering, but fearmongering also seems to be the core strategy in encouraging average people to be skeptics.

    Claiming that if GW is true, then the only possible future would be horrendous, then coupling that with an accusation that GW realists "wish" for such a disaster makes them out to be monsters. Who wants to associate with the ideas of monsters? That seems to be the operative strategy here.

    If you are such a good person, who really hopes in his heart that we dont have to face a warming future, and you are also a rational person, then you should accept that the only way forward is, despite your wishes, to actually arrive at the truth of the matter. If GW is happening, then your wishes are irrelevant - and to be a good person you should focus on finding ways to deal with a warming world that minimizes the pain.

    If GW is not happening it wont be because you wished such an outcome. So my bottom line advice would be - forget this "wishing" frame, and just focus on discovering reality, and be prepared, emotionally, to accept the things that are learned. Then you might actually begin to resemble a scientist.

  • dearieme

    In most walks of life, people who behaved like Hansen would be assumed to be crooks unless there were strong evidence to the contrary.

  • You said, "NOAA and GISS both need to release their detailed algorithms and computer software code for adjusting and aggregating USHCN and global temperature data. Period." But no bureaucrat needs to do anything unless he, his job, or some other interest of his is at stake. From your account of these matters, one may judge that the men at NOAA and GISS aren't simply moved by love of truth, but more by pride, manifesting itself here as pride of professional reputation. Yet their protection of their professional reputation is served more by hiding and downplaying their failings than by openness and candid assessment. Right now, the men at NOAA and GISS need to hide their failings; it's up to others to find ways to make them need to be open. One must find the screws, and one must have the cruelty to turn them.

  • Joe Citizen, do you apply the same standard to Christy and Spencer at UAH? Once their algebraic error in the UAH satellite data analysis was removed, it added about 50% to the global warming trend. In contrast, Hansen's error has added less than 1000th of a degree to the global warming trend.

    For the record, I think both Christy's and Hansen's errors were honest mistakes - both are immensely embarrassing though.

  • Joe Citizen, apologies, my comment was addressed to dearieme's comment about Hansen behaving like a crook (I got the names confused).

  • McE

    "1 NOAA and GISS both need to release their detailed algorithms and computer software code for adjusting and aggregating USHCN and global temperature data"

    Here you go, enjoy: http://www.giss.nasa.gov/tools/

  • McE

    "Is it bad that I'm skeptical on model based predictions because they can't tell me if it's going to rain tomorrow (with any sort of real accuracy) but they KNOW that in 2009 we will see the beginnings of a major warming period."

    There's a statistical fallacy in the above statement. I can easily predict that I'm going to die sometime in the next 40-60 years ago with a very high degree of certainty. It's much harder to predict whether or not I'm going to die tomorrow with anywhere near the same level of certainty. Statistics usually works this way, the longer the period, the greater the predictability.

  • TTT

    Now take two people. One in his heart really wants this theory not to be true, and hopes we don't have to face this horrible lose-lose tradeoff. The other has a deeply felt wish that this theory is true, and hopes man does face this horrible future. Which person do you like better? And recognize, RealClimate is holding up the latter as the only moral man.

    Don't you mean, "which person do you want to have a beer with?"

    Truth is not determined by how pleasant things are. AGW's existence is independent of your feelings. I don't "like" someone who argues that the Holocaust couldn't have happened because people shouldn't be so mean.

  • twv

    McE is right: Truth is not determined by pleasantness. Climate change happens regardless of how we think of it. It is not socially constructed, though it is likely (to what extent quite uncertain) socially influenced. There's a difference.

    But to pretend that only one side or the other (as if there were only two sides!) in the current debate recognizes this truth about truth is goofy. Most of my environmentalist friends feel quite morally smug about the issue, though only a few know anything about statistics, chaos, chemistry, etc. Or history. They just "know," because on faith they believe that human civilization must be in dangerous disequilibrium with natural processes.

    Similarly, my anti-enviro neighbors are generally optimistic about man's relationship to nature.

    Neither disposition -- both reflected in their hopes and moralities -- has much scientific merit.

    And while Al Gore may see catastrophic global warming as an "inconvenient truth," there are good reasons for his opponents to see the scenario as a "convenient conjecture" for his political allies. If you like regulation, you'll love the scenario.

    And it does stand to reason. Regulation is very bad at making social co-operation work better. But it is very good at making it work worse, and if human co-operation in industrial society is making the world warmer, then . . .

  • JDE

    A question: how many of the global temperature monitoring organizations are having their data analyzed through the original NASA algorithms? I would suspect that, while not submitting their underlying code for peer review, NASA has made these data analysis tools available to other organizations and states for their use in monitoring "Global Warming". Is there some way to determine if such a distribution has taken place and who may be using the flawed tools?

    Finally, should the original flawed NASA work be renamed "AlGorerithms"? (Sorry, couldn't resist).

  • ALLAN AMES

    Kralizec notes “But no bureaucrat needs to do anything unless he, his job, or some other interest of his is at stake.”. This is the heart of the problem. There presently is no advantage for U.S. federal agencies to be transparent. Prescriptive releases of information like item 1 of Coyote’s recommendations can always be practically circumvented by the bureaucracy by burying the information elsewhere. Case in point is the note by John Cook. Few people can scan the 130000+/- lines of GISS 1E code and extract anything useful from it. I have said elsewhere, understanding what Fortran code does by studying it is rather like trying to fathom the U.S economy from watching trailer trucks go by on the interstate highway. All you know is that a truck went by; you do not know what was in it, where it came from, and where, or even if, it is going. It is probably up to us to get our representatives to make NASA scientists more open to external review.

  • ALLAN AMES

    Kralizec notes “But no bureaucrat needs to do anything unless he, his job, or some other interest of his is at stake.”. This is the heart of the problem. There presently is no advantage for U.S. federal agencies to be transparent. Prescriptive releases of information like item 1 of Coyote’s recommendations can always be practically circumvented by the bureaucracy by burying the information elsewhere. Case in point is the note by John Cook. Few people can scan the 130000+/- lines of GISS 1E code and extract anything useful from it. I have said elsewhere, understanding what Fortran code does by studying it is rather like trying to fathom the U.S economy from watching trailer trucks go by on the interstate highway. All you know is that a truck went by; you do not know what was in it, where it came from, and where, or even if, it is going. It is probably up to us to get our representatives to make NASA scientists more open to external review.

  • Mike

    I live in Vancouver, home of the Standartenfuehrer of the Reichsministerium fur Klimaanderung, David Suzuki and thousands of other profiteering True Believers. It has been rainy, cloudy, cool, wet, clammy, and otherwise unseasonably COLD all summer. Conveniently, I'm sure (though I can't be bothered asking) the explanation is that INSTABILITY is the key feature of AGW - not just the random fluctuations we can expect of a non-doom-oriented climate scenario. Make the data fit into your scenario - that's like real good science, dudes! >:P

  • Mike

    Speaking of Suzuki, you should really check out his defense of AGW at his foundation's site: http://www.davidsuzuki.org/Campaigns_and_Programs/Climate_Change/News_Releases/newsclimatechange07130401.asp.

    Let's break this down, chunk it as we say in the writing biz:

    1) Vancouver has been keeping temp records for one human lifetime (since 1937).
    2) In that time, we have had two SECOND-warmest Junes. They were 25 years apart.
    3) We are not told what the WARMEST June on record during one human lifetime in this corner of the planet was; I assume that's because it would make an already-meaningless statistic look even more so.

    Lies, damn lies, and statistics - the unholy trinity of the AGW religion 🙂