Friday, January 02, 2004

Viewing "Pop Science" With a Skeptical Eye

Pop science is one of my biggest pet peeves. You know what I mean. Pop science is medical studies you read about in popular magazines that tell you to eat this or avoid that for better health. It's also those Discovery Channel documentaries such as "The Real Eve," where the narrator blathers on for TWO HOURS about one study of human mitochondrial DNA. There are actors portraying ancient humans walking dramatically into a body of water as they migrate out of Africa. This is Hollywood science. What the narrator doesn't tell you is that the conclusions of this study are disputed by several other research groups. To explain it briefly, multiple but equally valid phylogenetic trees (diagrams that cluster genes based on relatedness) can be developed from the analysis of the mitochondrial DNA markers, some of which don't support the out of Africa hypothesis. The narrator never uses the words theory or possibly. All is presented as indisputable fact and is quite misleading to the average non-scientist.

I ran across this speech by writer Michael Crichton that eloquently expresses the problems that arise in scientific inquiry when theories are based on poor scientific method and consensus rather than rigourous experimental science (via Instapundit). The title is great!

Aliens Cause Global Warming
A lecture by Michael Crichton
Caltech Michelin Lecture
January 17, 2003

Here's some of my favorite excerpts:

...Even as a child I believed that science represented the best and greatest hope for mankind. Even to a child, the contrast was clear between the world of politics-a world of hate and danger, of irrational beliefs and fears, of mass manipulation and disgraceful blots on human history. In contrast, science held different values-international in scope, forging friendships and working relationships across national boundaries and political systems, encouraging a dispassionate habit of thought, and ultimately leading to fresh knowledge and technology that would benefit all mankind. The world might not be avery good place, but science would make it better. And it did. In my lifetime, science has largely fulfilled its promise. Science has been the great intellectual adventure of our age, and a great hope for our troubled and restless world.

But I did not expect science merely to extend lifespan, feed the hungry, cure disease, and shrink the world with jets and cell phones. I also expected science to banish the evils of human thought---prejudice and superstition, irrational beliefs and false fears. I expected science to be, in Carl Sagan's memorable phrase, "a candle in a demon haunted world." And here, I am not so pleased with the impact of science. Rather than serving as a cleansing force, science has in some instances been seduced by the more ancient lures of politics and publicity. Some of the demons that haunt our world in recent years are invented by scientists. The world has not benefited from permitting these demons to escape free.


Absolutely. Science and politics haveindeed become intermingled. Of course scientists are human and therefore are not free of biases. But I think scientists can do a lot better in trying to be objective. One reason why I'm interested in science and public policy is that I think we need good scientists to get involved in public policy to keep the junk science out of it. Politicians need good scientists to give the best data so they can make the best decisions regarding resource management, healthcare, ethical issues such as cloning, etc. What we don't need is a bunch of partisan hacks that practice bad science pushing their agenda on the rest of the country.


But let's look at how it came to pass.

Cast your minds back to 1960. John F. Kennedy is president, commercial jet airplanes are just appearing, the biggest university mainframes have 12K of memory. And in Green Bank, West Virginia at the new National Radio Astronomy Observatory, a young astrophysicist named Frank Drake runs a two week project called Ozma, to search for extraterrestrial signals. A signal is received, to great excitement. It turns out to be false, but the excitement remains. In 1960, Drake organizes the first SETI conference, and came up with the now-famous Drake equation:

N=N*fp ne fl fi fc fL

Where N is the number of stars in the Milky Way galaxy; fp is the fraction with planets; ne is the number of planets per star capable of supporting life; fl is the fraction of planets where life evolves; fi is the fraction where intelligent life evolves; and fc is the fraction that communicates; and fL is the fraction of the planet's life during which the communicating civilizations live.

This serious-looking equation gave SETI an serious footing as a legitimate intellectual inquiry. The problem, of course, is that none of the terms can be known, and most cannot even be estimated. The only way to work the equation is to fill in with guesses. And guesses-just so we're clear-are merely expressions of prejudice. Nor can there be "informed guesses." If you need to state how many planets with life choose to communicate, there is simply no way to make an informed guess. It's simply prejudice.

As a result, the Drake equation can have any value from "billions and billions" to zero. An expression that can mean anything means nothing. Speaking precisely, the Drake equation is literally meaningless, and has nothing to do with science. I take the hard view that science involves the creation of testable hypotheses. The Drake equation cannot be tested and therefore SETI is not science. SETI is unquestionably a religion. Faith is defined as the firm belief in something for which there is no proof. The belief that the Koran is the word of God is a matter of faith. The belief that God created the universe in seven days is a matter of faith. The belief that there are other life forms in the universe is a matter of faith. There is not a single shred of evidence for any other life forms, and in forty years of searching, none has been discovered. There is absolutely no evidentiary reason to maintain this belief. SETI is a religion.

...Back in the sixties, SETI had its critics, although not among astrophysicists and astronomers. The biologists and paleontologists were harshest. George Gaylord Simpson of Harvard sneered that SETI was a "study without a subject," and it remains so to the present day.


I agree completely. The paucity of testable hypotheses is one reason why I view evolutionary biology as a weak science. It's impossible to reproduce the conditions on the prebiotic earth, so one must guess and speculate, and make a lot of assumptions.

...The fact that the Drake equation was not greeted with screams of outrage-similar to the screams of outrage that greet each Creationist new claim, for example-meant that now there was a crack in the door, a loosening of the definition of what constituted legitimate scientific procedure. And soon enough, pernicious garbage began to squeeze through the cracks.

Now let's jump ahead a decade to the 1970s, and Nuclear Winter.

In 1975, the National Academy of Sciences reported on "Long-Term Worldwide Effects of Multiple Nuclear Weapons Detonations" but the report estimated the effect of dust from nuclear blasts to be relatively minor. In 1979, the Office of Technology Assessment issued a report on "The Effects of Nuclear War" and stated that nuclear war could perhaps produce irreversible adverse consequences on the environment. However, because the scientific processes involved were poorly understood, the report stated it was not possible to estimate the probable magnitude of such damage.

Three years later, in 1982, the Swedish Academy of Sciences commissioned a report entitled "The Atmosphere after a Nuclear War: Twilight at Noon," which attempted to quantify the effect of smoke from burning forests and cities. The authors speculated that there would be so much smoke that a large cloud over the northern hemisphere would reduce incoming sunlight below the level required for photosynthesis, and that this would last for weeks or even longer.

The following year, five scientists including Richard Turco and Carl Sagan published a paper in Science called "Nuclear Winter: Global Consequences of Multiple Nuclear Explosions." This was the so-called TTAPS report, which attempted to quantify more rigorously the atmospheric effects, with the added credibility to be gained from an actual computer model of climate.

At the heart of the TTAPS undertaking was another equation, never specifically expressed, but one that could be paraphrased as follows:

Ds = Wn Ws Wh Tf Tb Pt Pr Pe etc

(The amount of tropospheric dust=# warheads x size warheads x warhead detonation height x flammability of targets x Target burn duration x Particles entering the Troposphere x Particle reflectivity x Particle enduranceand so on.)

The similarity to the Drake equation is striking. As with the Drake equation, none of the variables can be determined. None at all. The TTAPS study addressed this problem in part by mapping out different wartime scenarios and assigning numbers to some of the variables, but even so, the remaining variables were-and are-simply unknowable. Nobody knows how much smoke will be generated when cities burn, creating particles of what kind, and for how long. No one knows the effect of local weather conditions on the amount of particles that will be injected into the troposphere. No one knows how long the particles will remain in the troposphere. And so on.

And remember, this is only four years after the OTA study concluded that the underlying scientific processes were so poorly known that no estimates could be reliably made. Nevertheless, the TTAPS study not only made those estimates, but concluded they were catastrophic.

According to Sagan and his coworkers, even a limited 5,000 megaton nuclear exchange would cause a global temperature drop of more than 35 degrees Centigrade, and this change would last for three months. The greatest volcanic eruptions that we know of changed world temperatures somewhere between .5 and 2 degrees Centigrade. Ice ages changed global temperatures by 10 degrees. Here we have an estimated change three times greater than any ice age. One might expect it to be the subject of some dispute.

But Sagan and his coworkers were prepared, for nuclear winter was from the outset the subject of a well-orchestrated media campaign. The first announcement of nuclear winter appeared in an article by Sagan in the Sunday supplement, Parade. The very next day, a highly-publicized, high-profile conference on the long-term consequences of nuclear war was held in Washington, chaired by Carl Sagan and Paul Ehrlich, the most famous and media-savvy scientists of their generation. Sagan appeared on the Johnny Carson show 40 times. Ehrlich was on 25 times. Following the conference, there were press conferences, meetings with congressmen, and so on. The formal papers in Science came months later.

This is not the way science is done, it is the way products are sold(emphasis added).

...At the conference in Washington, during the question period, Ehrlich was reminded that after Hiroshima and Nagasaki, scientists were quoted as saying nothing would grow there for 75 years, but in fact melons were growing the next year. So, he was asked, how accurate were these findings now?

Ehrlich answered by saying "I think they are extremely robust. Scientists may have made statements like that, although I cannot imagine what their basis would have been, even with the state of science at that time, but scientists are always making absurd statements, individually, in various places. What we are doing here, however, is presenting a consensus of a very large group of scientists(emphasis added)"

I want to pause here and talk about this notion of consensus, and the rise of what has been called consensus science. I regard consensus science as an extremely pernicious development that ought to be stopped cold in its tracks. Historically, the claim of consensus has been the first refuge of scoundrels; it is a way to avoid debate by claiming that the matter is already settled. Whenever you hear the consensus of scientists agrees on something or other, reach for your wallet(emphasis added), because you're being had.

Let's be clear: the work of science has nothing whatever to do with consensus. Consensus is the business of politics. Science, on the contrary, requires only one investigator who happens to be right, which means that he or she has results that are verifiable by reference to the real world. In science consensus is irrelevant. What is relevant is reproducible results(emphasis added). The greatest scientists in history are great precisely because they broke with the consensus.

There is no such thing as consensus science. If it's consensus, it isn't science. If it's science, it isn't consensus. Period.


This is one reason why I am extremely skeptical of the "salesmen scientists." These are the guys (or gals) that you see commonly on T.V., in magazines, in Barnes & Noble. They often water everything down, leave out important details, and promote their pet theories as the gospel truth. Examples include Carl Sagan, Richard Dawkins, and Steven Jay Gould. I bet you've all heard of these guys. That's because they are good at self-promotion. My mentor is also a gifted salesman.

...Finally, I would remind you to notice where the claim of consensus is invoked. Consensus is invoked only in situations where the science is not solid enough (emphasis added). Nobody says the consensus of scientists agrees that E=mc2. Nobody says the consensus is that the sun is 93 million miles away. It would never occur to anyone to speak that way.

But back to our main subject.

What I have been suggesting to you is that nuclear winter was a meaningless formula, tricked out with bad science, for policy ends. It was political from the beginning, promoted in a well-orchestrated media campaign that had to be planned weeks or months in advance.

Further evidence of the political nature of the whole project can be found in the response to criticism. Although Richard Feynman was characteristically blunt, saying, "I really don't think these guys know what they're talking about," other prominent scientists were noticeably reticent. Freeman Dyson was quoted as saying "It's an absolutely atrocious piece of science but who wants to be accused of being in favor of nuclear war?" And Victor Weisskopf said, "The science is terrible but---perhaps the psychology is good." The nuclear winter team followed up the publication of such comments with letters to the editors denying that these statements were ever made, though the scientists since then have subsequently confirmed their views.

At the time, there was a concerted desire on the part of lots of people to avoid nuclear war. If nuclear winter looked awful, why investigate too closely? Who wanted to disagree? Only people like Edward Teller, the "father of the H bomb."

Teller said, "While it is generally recognized that details are still uncertain and deserve much more study, Dr. Sagan nevertheless has taken the position that the whole scenario is so robust that there can be little doubt about its main conclusions." Yet for most people, the fact that nuclear winter was a scenario riddled with uncertainties did not seem to be relevant.

I say it is hugely relevant. Once you abandon strict adherence to what science tells us, once you start arranging the truth in a press conference, then anything is possible. In one context, maybe you will get some mobilization against nuclear war. But in another context, you get Lysenkoism. In another, you get Nazi euthanasia. The danger is always there, if you subvert science to political ends.

That is why it is so important for the future of science that the line between what science can say with certainty, and what it cannot, be drawn clearly-and defended.

What happened to Nuclear Winter? As the media glare faded, its robust scenario appeared less persuasive; John Maddox, editor of Nature, repeatedly criticized its claims; within a year, Stephen Schneider, one of the leading figures in the climate model, began to speak of "nuclear autumn." It just didn't have the same ring.

A final media embarrassment came in 1991, when Carl Sagan predicted on Nightline that Kuwaiti oil fires would produce a nuclear winter effect, causing a "year without a summer," and endangering crops around the world. Sagan stressed this outcome was so likely that "it should affect the war plans." None of it happened.


Snicker.

What, then, can we say were the lessons of Nuclear Winter? I believe the lesson was that with a catchy name, a strong policy position and an aggressive media campaign, nobody will dare to criticize the science, and in short order, a terminally weak thesis will be established as fact. After that, any criticism becomes beside the point. The war is already over without a shot being fired. That was the lesson, and we had a textbook application soon afterward, with second hand smoke.


Crichton goes on to talk about how poor science in the case of 2nd hand smoke has affected public policy via the EPA, but I'll spare you the details since you are probably aware of them since this is fairly recent. In brief, no studies support the "fact" that 2nd-hand smoke is a severe health hazard.

As the twentieth century drew to a close, the connection between hard scientific fact and public policy became increasingly elastic. In part this was possible because of the complacency of the scientific profession; in part because of the lack of good science education among the public; in part, because of the rise of specialized advocacy groups which have been enormously effective in getting publicity and shaping policy; and in great part because of the decline of the media as an independent assessor of fact. The deterioration of the American media is dire loss for our country. When distinguished institutions like the New York Times can no longer differentiate between factual content and editorial opinion, but rather mix both freely on their front page, then who will hold anyone to a higher standard?
.

I love it. Someone outside the blogosphere takes the Old Gray Lady to task.

And so, in this elastic anything-goes world where science-or non-science-is the hand maiden of questionable public policy, we arrive at last at global warming. It is not my purpose here to rehash the details of this most magnificent of the demons haunting the world. I would just remind you of the now-familiar pattern by which these things are established. Evidentiary uncertainties are glossed over in the unseemly rush for an overarching policy, and for grants to support the policy by delivering findings that are desired by the patron. Next, the isolation of those scientists who won't get with the program, and the characterization of those scientists as outsiders and "skeptics" in quotation marks-suspect individuals with suspect motives, industry flunkies, reactionaries, or simply anti-environmental nutcases. In short order, debate ends, even though prominent scientists are uncomfortable about how things are being done.

When did "skeptic" become a dirty word in science? When did a skeptic require quotation marks around it?

To an outsider, the most significant innovation in the global warming controversy is the overt reliance that is being placed on models. Back in the days of nuclear winter, computer models were invoked to add weight to a conclusion: "These results are derived with the help of a computer model." But now large-scale computer models are seen as generating data in themselves. No longer are models judged by how well they reproduce data from the real world-increasingly, models provide the data. As if they were themselves a reality. And indeed they are, when we are projecting forward. There can be no observational data about the year 2100. There are only model runs.


Biophysical papers that rely mainly on computer models are generally not given much credence. You won't see them in the top science journals usually, but I have noticed that some papers in these journals seem to be there because they are "sexy" subjects and not necessarily examples of good science.

...Nobody believes a weather prediction twelve hours ahead. Now we're asked to believe a prediction that goes out 100 years into the future? And make financial investments based on that prediction? Has everybody lost their minds?

Stepping back, I have to say the arrogance of the modelmakers is breathtaking. There have been, in every century, scientists who say they know it all. Since climate may be a chaotic system-no one is sure-these predictions are inherently doubtful, to be polite. But more to the point, even if the models get the science spot-on, they can never get the sociology. To predict anything about the world a hundred years from now is simply absurd.

Look: If I was selling stock in a company that I told you would be profitable in 2100, would you buy it? Or would you think the idea was so crazy that it must be a scam?

...I remind you that in the lifetime of most scientists now living, we have already had an example of dire predictions set aside by new technology. I refer to the green revolution. In 1960, Paul Ehrlich said, "The battle to feed humanity is over. In the 1970s undergod will undergoe famines-hundreds of millions of people are going to starve to death." Ten years later, he predicted four billion people would die during the 1980s, including 65 million Americans. The mass starvation that was predicted never occurred, and it now seems it isn't ever going to happen. Nor is the population explosion going to reach the numbers predicted even ten years ago. In 1990, climate modelers anticipated a world population of 11 billion by 2100. Today, some people think the correct number will be 7 billion and falling. But nobody knows for sure.

But it is impossible to ignore how closely the history of global warming fits on the previous template for nuclear winter. Just as the earliest studies of nuclear winter stated that the uncertainties were probabilities that could never be known, so, too the first pronouncements on global warming argued strong limits on what could be determined with certainty about climate change. The 1995 IPCC draft report said, "Any claims of positive detection of significant climate change are likely to remain controversial until uncertainties in the total natural variability of the climate system are reduced." It also said, "No study to date has positively attributed all or part of observed climate changes to anthropogenic causes." Those statements were removed, and in their place appeared: "The balance of evidence suggests a discernable human influence on climate."

What is clear, however, is that on this issue, science and policy have become inextricably mixed to the point where it will be difficult, if not impossible, to separate them out. It is possible for an outside observer to ask serious questions about the conduct of investigations into global warming, such as whether we are taking appropriate steps to improve the quality of our observational data records, whether we are systematically obtaining the information that will clarify existing uncertainties, whether we have any organized disinterested mechanism to direct research in this contentious area.

The answer to all these questions is no. We don't.


Crichton then proposes an interesting solution to the problem of mixing policy and science. I don't know if it would work, but it's a start.

Just as we have established a tradition of double-blinded research to determine drug efficacy, we must institute double-blinded research in other policy areas as well. Certainly the increased use of computer models, such as GCMs, cries out for the separation of those who make the models from those who verify them. The fact is that the present the present structure of science is entrepeneurial, with individual investigative teams vying for funding from organizations which all too often have a clear stake in the outcome of the research-or appear to, which may be just as bad. This is not healthy for science.

Sooner or later, we must form an independent research institute in this country. It must be funded by industry, by government, and by private philanthropy, both individuals and trusts. The money must be pooled, so that investigators do not know who is paying them. The institute must fund more than one team to do research in a particular area, and the verification of results will be a foregone requirement: teams will know their results will be checked by other groups. In many cases, those who decide how to gather the data will not gather it, and those who gather the data will not analyze it. If we were to address the land temperature records with such rigor, we would be well on our way to an understanding of exactly how much faith we can place in global warming, and therefore what seriousness we must address this.


Crichton then shows how political science has become by discussing the witchhunt of a statistician, Bjorn Lomburg, who published a book questioning the validity of the studies used to support global warming.

The scientific community responded in a way that can only be described as disgraceful. In professional literature, it was complained he had no standing because he was not an earth scientist. His publisher, Cambridge University Press, was attacked with cries that the editor should be fired, and that all right-thinking scientists should shun the press. The past president of the AAAS wondered aloud how Cambridge could have ever "published a book that so clearly could never have passed peer review." )But of course the manuscript did pass peer review by three earth scientists on both sides of the Atlantic, and all recommended publication.) But what are scientists doing attacking a press? Is this the new McCarthyism-coming from scientists?

Worst of all was the behavior of the Scientific American, which seemed intent on proving the post-modernist point that it was all about power, not facts. The Scientific American attacked Lomborg for eleven pages, yet only came up with nine factual errors despite their assertion that the book was "rife with careless mistakes." It was a poor display featuring vicious ad hominem attacks, including comparing him to a Holocust denier. The issue was captioned: "Science defends itself against the Skeptical Environmentalist." Really. Science has to defend itself? Is this what we have come to?

When Lomborg asked for space to rebut his critics, he was given only a page and a half. When he said it wasn't enough, he put the critics' essays on his web page and answered them in detail. Scientific American threatened copyright infringement and made him take the pages down.

Further attacks since have made it clear what is going on. Lomborg is charged with heresy. That's why none of his critics needs to substantiate their attacks in any detail. That's why the facts don't matter. That's why they can attack him in the most vicious personal terms. He's a heretic.

Of course, any scientist can be charged as Galileo was charged. I just never thought I'd see the Scientific American in the role of mother church (emphasis added).


Ouch and double ouch.

Is this what science has become? I hope not. But it is what it will become, unless there is a concerted effort by leading scientists to aggresively separate science from policy. The late Philip Handler, former president of the National Academy of Sciences, said that "Scientists best serve public policy by living within the ethics of science, not those of politics. If the scientific community will not unfrock the charlatans, the public will not discern the difference-science and the nation will suffer." Personally, I don't worry about the nation. But I do worry about science.


This is also why religion should be left out of science and vice versa. They are two different things. Nothing irritates me more than arrogant atheist scientists that rip on others' religious beliefs or creationists that refuse to acknowledge the merits of any scientific theory other than their own. One can be a good scientist and a good Christian or Hindu or Jew or whatever.

No comments: