Strength in Many Peers

Strength in Many Peers :

(This article is about climate science, but, again, you have to make it through the beginning.)

Last week an article that linked autism and measles vaccinations was retracted by the medical journal, The Lancet. (Story in LA Times, Story in San Francisco Chronicle, Retraction in The Lancet) The retraction is very short. The newspaper articles report that many of the co-authors of the paper had stated that the study was flawed. This paper has been used as a rationalization by many people to not have their children vaccinated, leading to a resurgence of measles. Ultimately there have been many studies and reviews investigating the link between vaccines and autism, and they have not found any link. (see Discover Magazine and National Academy of Science Report) Despite many studies that discredited the research linking autism and vaccination, the controversy about vaccines has been kept alive, and, in fact, laws have been passed prohibiting the use of a certain preservative in vaccines for young children.

This single paper provided evidence that suggested risk. Many studies suggested that the paper was wrong. Actions were taken to address the suggested risk – actions that are usually published with statements that there was no basis in the scientific literature to justify the action. Millions of dollars and human talent are spent refuting this research. Celebrities take counter positions to the scientists. Children go unvaccinated; a public health risk, infectious disease, is increased. A single paper is retracted - apparently a paper whose results proved unverifiable by independent researchers.

In December there was the publication in the news that there was a problem with a claim and a reference in the IPCC reports. This problem is a statement in the report that the glaciers in the Himalaya were melting at such a rate that they might disappear by 2035. Here is the article in the London Times that was the first press report (I think).

The London Times article states that this statement followed from a news story, not a journal article. More recent news stories in the The Times of India and newKerala.com report the scientist at the center of the assertion, Syed Hasnain, denying that he made the statement in the interview or that he has every published said conclusion in peer reviewed literature. If you look around you can find stories about transcription errors, creative journalism by the original reporter in the news story, and speculation during the interview. Of course, you can also find many stories about this is evidence of the conspiracy by climate scientists to mislead the world in order to achieve some undefined goal that would benefit climate scientists or Al Gore.

If you examine the “crisis,” disruption, or controversy that follows from this revelation and the controversy that arose around the vaccines and autism, you will find many of the same elements. There are accusations of prejudice and conspiracy; there is advocacy by people based on want they believe and want; there is fear and perception of risk. There is a dance of people defending their positions, and attempts to clarify information that appears in shards in the press, as well as in blogs, good and bad. The crisis takes on a life of its own. There are those who become vested in the crisis and the perpetuation of the crisis. It is naïve to assume that all parties are interested in the resolution of the crisis. It is even more naïve to presume that people desire a knowledge-based resolution of the crisis.

If you look at these controversies from a distance, they are alike. However, if you dig a little deeper, if you are truly interested in carrying out your own verification, then there will be, I assert, a significant difference. That difference is the Intergovernmental Panel on Climate Change, the IPCC. As I have stated before, the IPCC process is known and published. It provides a measure of transparency. This figure is a schematic of the process:




Figure 1: Schematic of the Process to Generate an IPCC Assessment Report

Note there are review processes. The “experts” are different from the “authors.” If you study the process more thoroughly, then there is a step where comments are taken from people all over the world, not just identified experts and government officials. It is required that all comments be addressed. This level of review would be extraordinary as a “policy,” but to provide accountability, the draft documents and the review comments are available for public inspection. (Here they are!)

If you can slog through these, then you will find that concerns about the now controversial statement were raised in review. Now the fact is that in this case the review failed, and an unverified statement about glaciers melting made it through. That statement did not go unnoticed, and as the London Times reported “glaciologists find such figures inherently ludicrous.” A nuance pointed out to me is that in terms of water resources, snow cover is more of an issue than glaciers; therefore, there is perhaps a natural blurring of terms as information is reported in the press. It is safe to say, that amongst expert scientists, the presence of the statement in the IPCC report was worrisome.

Again to be clear, here is an example of a mistake, an error, of a statement making it into the IPCC report that was not correct. (Here is a far more detailed analysis of the mistake: climatesciencewatch.org) However, it is possible to trace some of the heritage of that mistake because a log has been kept of a fundamentally open review process. This, as Daniel Farber argues, (link) is one of the features that makes climate science more robust than a single article that has caused the turmoil in the world of vaccines and autism.

There is a point that I want to make about the IPCC Reports. Over my career the practice of providing assessment reports about global environmental issues has become routine. I started working on Ozone Assessment Reports in the 1980s, and the 1987 Report was a comprehensive review of stratospheric science. There are assessments of climate change other than the IPCC Assessment Reports; for example, the reports of United States Global Change Research Program. When I was at NASA, the production of these reports was viewed as a “tax” on the community; namely, to work on these reports came at the expense of “research.” I have learned in recent years that these reports are more than a tax on or a service function of the community; they are an essential part of scientific research. One part of scientific research is the reduction of complex problems to small enough pieces to isolate cause and effect. Another essential part is the integration of those pieces into a system as a whole, because it is the system as a whole that we are really studying. These assessments both directly and indirectly motivate this integration. The fact that there are a number of assessments, again, improves robustness as there are checks and balances.

In my previous entry, I noted that, like a market, the scientific community requires processes and practices to put in checks and balances. In the assessment process, these checks and balances reduce the inclusion of results that stand by themselves. Some of those lone-standing results will be correct, and hence, the report will appear “conservative.” Some of those lone-standing results will be incorrect, and the report will appear robust and correct. The process is not perfect, but it minimizes the inclusion of incorrect statements. Therefore, the consequence of an isolated failure, as with the statement about the melting of the glaciers, will have minimal impact on the science-based conclusions of the assessment. There is strength in many peers; this is not the same as consensus.

The consequences of an isolated failure might, however, have a large impact on politics. It is obvious that there are people who are vested in discrediting the science of climate change. They want to disrupt, diminish, and guide any actions that we might, collectively, take. Disruption is a strategy, and it is deliberate behavior. It is a process that is not interested in knowledge-based resolution of scientific disagreement. It is a process that is interested in disruption; hence, one response by the scientific community does nothing but to generate the next disruption. Scientists are trained to produce knowledge and to develop knowledge-based arguments. Scientists by human temperament like to be right, and they defend their behavior – we all do. But to respond to a disruptive process by defense, careful exposition of arguments, and accusations of who is right and who is wrong plays into the hand of the disruptor. It maintains and spreads the crisis. If the scientists of the IPCC do this, they will ultimately discredit and undermine a stunning and important body of knowledge that they created.

r

Some other related blogs:

“Have you no sense of decency, sir, at long last?”


Trust, but Verify


Science, Belief and the Volcano

Opinions and Anecdotal Evidence




And here is

Faceted Search of Blogs at climateknowledge.org