[asa] Peer Review, Publication in Top Journals, Scientific Consensus, and So Forth

From: Janice Matchett <janmatch@earthlink.net>
Date: Wed May 16 2007 - 12:05:26 EDT

Kind of interesting, huh? ~ Janice

Peer Review, Publication in Top Journals, Scientific Consensus, and So Forth
May 7, 2007 Robert Higgs

In following the discussion of global warming and related issues in
the press and the blogosphere, I have been struck repeatedly by the
assumption or expression of certain beliefs that strike me as highly
problematical. Many writers who are not scientists themselves are
trading on the prestige of science and the authority of scientists.
Reference to "peer-reviewed research" and to an alleged "scientific
consensus" are regarded as veritable knock-out blows by many
commentators. Yet many of those who make such references appear to me
to be more or less ignorant of how science as a form of
knowledge-seeking and scientists as individual professionals operate,
especially nowadays, when national governments most notably the U.S.
government play such an overwhelming role in financing scientific
research and hence in determining which scientists rise to the top
and which fall by the wayside.

I do not pretend to have expertise in climatology or any of the
related physical sciences, so nothing I might say about strictly
climatological or related physical-scientific matters deserves any
weight. However, I have thirty-nine years of professional experience
twenty-six as a university professor, including fifteen at a major
research university, and then thirteen as a researcher, writer, and
editor in close contact with scientists of various sorts, including
some in the biological and physical sciences and many in the social
sciences and demography. I have served as a peer reviewer for more
than thirty professional journals and as a reviewer of research
proposals for the National Science Foundation, the National
Institutes of Health, and a number of large private foundations. I
was the principal investigator of a major NSF-funded research project
in the field of demography. So, I think I know something about how
the system works.

It does not work as outsiders seem to think.

Peer review, on which lay people place great weight, varies from
being an important control, where the editors and the referees are
competent and responsible, to being a complete farce, where they are
not. As a rule, not surprisingly, the process operates somewhere in
the middle, being more than a joke but less than the nearly flawless
system of Olympian scrutiny that outsiders imagine it to be. Any
journal editor who desires, for whatever reason, to reject a
submission can easily do so by choosing referees he knows full well
will knock it down; likewise, he can easily obtain favorable referee
reports. As I have always counseled young people whose work was
rejected, seemingly on improper or insufficient grounds, the system
is a crap shoot. Personal vendettas, ideological conflicts,
professional jealousies, methodological disagreements, sheer
self-promotion, and a great deal of plain incompetence and
irresponsibility are no strangers to the scientific world; indeed,
that world is rife with these all-too-human attributes. In no event
can peer review ensure that research is correct in its procedures or
its conclusions. The history of every science is a chronicle of one
mistake after another. In some sciences these mistakes are largely
weeded out in the course of time; in others they persist for extended
periods; and in some sciences, such as economics, actual scientific
retrogression may continue for generations under the misguided (but
self-serving) belief that it is really progress.

At any given time, consensus may exist about all sorts of matters in
a particular science. In retrospect, however, that consensus is often
seen to have been mistaken. As recently as the mid-1970s, for
example, a scientific consensus existed among climatologists and
scientists in related fields that the earth was about to enter a new
ice age. Drastic proposals were made, such as exploding hydrogen
bombs over the polar icecaps (to melt them) or damming the Bering
Strait (to prevent cold Arctic water from entering the Pacific
Ocean), to avert this impending disaster. Well-reputed scientists,
not just uninformed wackos, made such proposals. How quickly we forget.

Researchers who employ unorthodox methods or theoretical frameworks
have great difficulty under modern conditions in getting their
findings published in the "best" journals or, at times, in any
scientific journal. Scientific innovators or creative eccentrics
always strike the great mass of practitioners as nut cases until
their findings become impossible to deny, which often occurs only
after one generation's professional ring-masters have died off.
Science is an odd undertaking: everybody strives to make the next
breakthrough, yet when someone does, he is often greeted as if he
were carrying the ebola virus. Too many people have too much invested
in the reigning ideas; for those people an acknowledgment of their
own idea's bankruptcy is tantamount to an admission that they have
wasted their lives. Often, perhaps to avoid cognitive dissonance,
they never admit that their ideas were wrong. Most important, as a
rule, in science as elsewhere, to get along, you must go along.

Research worlds, in their upper reaches, are pretty small. Leading
researchers know all the major players and what everybody else is
doing. They attend the same conferences, belong to the same
societies, send their grad students to be postdocs in the other
people's labs, review one another's work for the NSF, NIH, or other
government funding organizations, and so forth. If you do not belong
to this tight fraternity, it will prove very, very difficult for you
to gain a hearing for your work, to publish in a "top" journal, to
acquire a government grant, to receive an invitation to participate
in a scientific-conference panel discussion, or to place your grad
students in decent positions. The whole setup is tremendously
incestuous; the interconnections are numerous, tight, and close.

In this context, a bright young person needs to display cleverness in
applying the prevailing orthodoxy, but it behooves him not to rock
the boat by challenging anything fundamental or dear to the hearts of
those who constitute the review committees for the NSF, NIH, and
other funding organizations. Modern biological and physical science
is, overwhelmingly, government-funded science. If your work, for
whatever reason, does not appeal to the relevant funding agency's
bureaucrats and academic review committees, you can forget about
getting any money to carry out your proposal. Recall the human
frailties I mentioned previously; they apply just as much in the
funding context as in the publication context. Indeed, these two
contexts are themselves tightly linked: if you don't get funding,
you'll never produce publishable work, and if you don't land good
publications, you won't continue to receive funding.

When your research implies a "need" for drastic government action to
avert a looming disaster or to allay some dire existing problem,
government bureaucrats and legislators (can you say "earmarks"?) are
more likely to approve it. If the managers at the NSF, NIH, and other
government funding agencies gave great amounts of money to scientists
whose research implies that no disaster looms or no dire problem now
exists or even that although a problem exists, no currently feasible
government policy can do anything to solve it without creating
greater problems in the process, members of Congress would be much
less inclined to throw money at the agency, with all the consequences
that an appropriations cutback implies for bureaucratic thriving. No
one has to explain all these things to the parties involved; they are
not idiots, and they understand how the wheels are greased in their
tight little worlds.

Finally, we need to develop a much keener sense of what a scientist
is qualified to talk about and what he is not qualified to talk
about. Climatologists, for example, are qualified to talk about the
science of climatology (though subject to all the intrusions upon
pure science I have already mentioned). They are not qualified to
say, however, that "we must act now" by imposing government
"solutions" of some imagined sort. They are not professionally
knowledgeable about what degree of risk is better or worse for people
to take; only the individuals who bear the risk can make that
decision, because it's a matter of personal preference, not a matter
of science. Climatologists know nothing about cost/benefit
cosiderations; indeed, most mainstream economists themselves are
fundamentally misguided about such matters (adopting, for example,
procedures and assumptions about the aggregation of individual
valuations that lack a sound scientific basis). Climate scientists
are the best qualified people to talk about climate science, but they
have no qualifications to talk about public policy, law, or
individual values, rates of time preference, and degrees of risk
aversion. In talking about desirable government action, they give the
impression that they are either fools or charlatans, but they keep
talking worst of all, talking to doomsday-seeking journalists nevertheless.

In this connection, we might well bear in mind that the United
Nations (and its committees and the bureaus it oversees) is no more a
scientifc organization than the U.S. Congress (and its committees and
the bureaus it oversees). When decisions and pronouncements come
forth from these political organizations, it makes sense to treat
them as essentially political in origin and purpose. Politicians
aren't dumb, either vicious, yes, but not dumb. One thing they know
above everything else is how to stampede masses of people into
approving or accepting ill-advised government actions that cost the
people dearly in both their standard of living and their liberties in
the long run.

Robert Higgs is Senior Fellow in Political Economy for The
Independent Institute and Editor of the Institute's quarterly journal
The Independent Review. He received his Ph.D. in economics from Johns
Hopkins University, and he has taught at the University of
Washington, Lafayette College, Seattle University, and the University
of Economics, Prague. He has been a visiting scholar at Oxford
University and Stanford University, and a fellow for the Hoover
Institution and the National Science Foundation. He is the author of
many books, including Depression, War, and Cold War.

To unsubscribe, send a message to majordomo@calvin.edu with
"unsubscribe asa" (no quotes) as the body of the message.
Received on Wed May 16 12:11:28 2007

This archive was generated by hypermail 2.1.8 : Wed May 16 2007 - 12:11:28 EDT