Monday, August 11, 2014

The decline of science: why scientists are publishing too many papers

We are seeing scientists badly failing in convincing decision makers of the urgent need of doing something against the impending disaster caused by global warming. But that's just a symptom of the decline of scientific research, desperately seeking for funds, but oppressed by bureaucracy and by a general disinterest on the part of the public; to say nothing of the rampant phenomenon of pseudoscience. In this text, I argue that one of the causes of the decline of science is the emphasis in publishing (the "publish or perish" rule). I argue that scientific papers have become a form of currency, suffering all the problems which plague the modern financial markets. Both the financial world and the scientific world have developed "emergent" properties which optimize throughput but not necessarily benefits. In short, we are publishing too much. (image above from this page)

The scientific world seems to be swamped by a true tsunami of papers of all kinds, full of sound and fury and signifying nothing. A situation which looks more and more similar to that of the general cacophony of the World Wide Web, swamped by poor quality information drowning the good information (if any). This starts to be a serious problem and some have explicitly asked that scientists should publish a smaller number of papers, but of higher quality (as argued, for instance, by Timo Hannay). 

But why do we find ourselves in this situation? What has caused science to become a paper mill? Here, I argue that it is the result of the basic properties of complex systems. These systems generate emergent properties which are often similar in fields which appear very different at first sight. In particular, scientific publishing turns out to be very similar to the world's financial system, with all the associated problems of uncontrolled growth and waste of resources. Let me explain my point.

From the beginning of one's career, scientists are pressured to publish, publish, and publish. That is known as the "Publish or Perish" rule which is implemented by means of the "peer review" process in which colleagues of the authors have the authority of accepting or rejecting the submitted paper, or request modifications. It looks simple, but it is much more complex than this, with several variants on the theme of "peer review", different prestige of scientific journals, different methods of diffusion (e.g. open access or paid subscriptions) and more.

One of the problems with the system is that the peer review system can usually filter out the really bad papers, but can hardly do the same for papers which are simply mediocre. The limitations of peer review have generated the arcane (and ineffectual) methods of post-publication evaluation which sometimes go under the name of "scientometry" or "scientometrics" (not to be confused with Scientology!!).

For a non-scientist, the urge to publish and the methods of publications in science are hard to understand, but the matter will appear perfectly clear if we compare it to something we are all familiar with: ordinary, monetary currency. Let me examine the many parallels in a non-exhaustive list.

1. Currency. The way we intend monetary currency nowadays is something that has no intrinsic value: it is in the form of sheets of paper or bits in computers. But by having these bits or pieces of paper you gain prestige and luxury items, and you climb up in the social ladder. The situation is exactly the same for scientific papers. In themselves, papers may have little or no value, but the more papers a scientist has published, the higher is his/her prestige and the more he/she can climb up the scientific ladder to higher and more prestigious positions. Papers can also bring luxury items in the form of expensive research equipment (microscopes, particle accelerators, scanners, etc.).

1. Emitting currency. Today, central banks are the entities authorized to emit monetary currency, and they have the authority of stamping a validation mark on an otherwise worthless piece of paper which then becomes 'money'. In science, validation of a paper is the privilege of scientific publishers. But who gave to scientific publishers this authority? It is an interesting question, just as impossible to answer as asking who gave the banks the same kind of authority with ordinary currency.

2. Spending your currency. Ordinary currency has no value in itself, but it can be exchanged with all sorts of items in the market. Scientific papers are not so easy to redeem, but can be transformed into ordinary currency by using them as tokens necessary to obtain a salary, career advancements, honorariums, and more.

3. Inflation. Currency is well known to undergo inflation; it loses part of its value with time. Scientific papers are subjected to the same phenomenon. Older papers are less valuable than new ones and in order to maintain your "wealth", as a scientist you must fight inflation. If your papers get old and no new ones are published, then they will be worth nothing.

4. Interest on currency. Ordinary currency can be deposited in banks in order to acquire an interest in the form of more currency. For scientific papers, the same role is played by funding agencies which transform scientific papers into research grants, which scientists will use to produce more papers. It is a classic example of a reinforcing feedback. 

5. Assaying. The real value of ordinary currency can be ascertained by procedures which may involve chemical assaying of precious metals. With paper currency, there are ways to determine whether they have been printed by authorized agencies. With scientific papers, their validity is verified by "referees;" scientists who will decide whether the data and the interpretation reported are correct.

 6. Counterfeiting. Ordinary currency can be counterfeited in various ways, for instance in the form of worthless metals instead of precious ones, in the form of paper bills printed by unauthorized agencies, and in the form of legitimate - but worthless - currency emitted by the central bank of small and unknown countries. In scientific publishing, counterfeiting is performed by small "predatory" publishers which do not perform the same validity check as the established publishers and may simply publish anything in exchange for a (standard monetary) fee paid by the authors.

7. Bad money replaces the good. This is a well known phenomenon in all economies, with money being debased by reducing the content of precious metals or printing too much of it. In science, we are seeing the same phenomenon with the proliferation of scientific publishers - often shady businesses trying to make a buck from scientists eager to see their paper published but not succeeding with the traditional journals. The result is an inflation of bad papers which tend to swamp the flux of good ones.

8. Ponzi schemes and multi-level marketing.  A Ponzi scheme is a pyramidal structure in which the lower layers pay the higher ones for the privilege of being inside. A multi-level marketing scheme is similar, but you pay for the privilege of being able to sell a product. There is no reason why such schemes cannot exist also in science. Some recently started journals have taken up a pyramidal structure which looks suspiciously like a multi-layer marketing scheme. In this case, scientists are drawn into the scheme with the allure of being defined as "editors." As a result, they work for free for the publisher!

As you see, the similarities are so many and so evident that we can say that the paper publishing system in modern science is a form of currency which exists and prospers within the system which has created it. It is so entrenched and so natural that most scientists seem to show little or no interest in its origins. Yet, the peer review system seems to have been unknown a century ago (see, e.g. this note by Michael Nielsen). For instance, only one of the about 300 papers published by Albert Einsten went through peer review. The scientific publication system we know today seems to have become the rule only in the second half of the 20th century. It is impressive that this system emerged all by itself without anyone planning it. It is an "emergent phenomenon", one of the characteristics of complex systems which tend to evolve in such a way to maximize the dissipation of potential energy (see, e.g. Kaila and Annila).

We could say that the world's financial system has evolved in order to maximize the destruction of the Earth's natural resources; favoring their consumption at speeds much larger than the Earth's capability to reform them - obviously not a benefit for humankind. We could argue that the world's scientific publication system has evolved in order to maximize the production of a large number of mediocre and useless paper. Again, this is not a benefit to science. Scientists are publishing too much!

Can these systems be changed? There is much talk on the subject of reforming the world's financial system, just as there is much talk about reforming the world's scientific publication system. In both cases, however, reform seems to be very difficult, if not impossible. In science, the well intentioned effort to open up to the public the results of scientific research by the "Open Access" system seems to have backfired, generating a wave of "predatory publishers" favoring an even faster dissipation of scientific potentials by greatly increasing the number of mediocre or bad papers. The financial system seems to be even more impervious to all kind of changes.

In the end, it seems that most systems of this kind can be reformed only by rebuilding them after they have crashed. That's not surprising: after all, you should know that if you fight thermodynamics, thermodynamics always wins.


  1. What you raise is certainly a problem. And I think your analogy with currencies is very good. How to fix the problem? Or does the whole system (of producing and publishing scientific papers and all the organizations, institutions and incentives involved in it) have to crash first? Perhaps such “emergent properties” are un-manageable. They will emerge, they will grow, they will diversify, they will crest and then eventually they may crash.

    But if “management” of the problem and the complex system that creates it is even to be attempted it seems to me that one key aspect that would need to be addressed has to do with the value of different scientific papers. Or perhaps the “scientometrists” are now already working on this?

    So that apart from the papers which are worthless or fraudulent which (presumably) peer reviews will sift out what criteria (if any generic such can be formulated either ex ante or ex post) can distinguish a mediocre paper, from a pretty good paper, from a good paper to an excellent paper or a truly landmark paper?

    For instance let’s take the 300 papers published by Einstein which you mention. I think nobody would dispute that Einstein was a top level scientist. (though perhaps some may have disputed it early on in his career). Were all of his 300 papers “landmark papers” such as those on special relativity and a few years later on general relativity? Or were some papers “more landmark than others” and were there some among his three hundred papers that both he and the world might easily have dispensed with? (perhaps there were NONE among the papers Einstein published but surely there must be plenty among the papers published by other scientists, and even TOP scientists).

    In fact I too as a clearly top level e-mail writer can probably write and send at least 50 e mails per day to my friends, acquaintances, ex colleagues and etc. Perhaps one or two (if I am lucky) may have something interesting, incisive or illuminating to say about some current events or situations or other. All the others may be useless or peripheral chit chat or be filled with utter banalities.

    And let us now assume that I were provided strong incentives (money, prestige, recognition, status, "authoritativeness" ) (or in my own case simply the incentive of being left alone) to write 100 e mails per day instead of only 50. Would that increase the number of let us say “landmark” e mails that I might write my friends that said something so momentous and important that they would never forget it, or my e mails?

    Or would I still come up with only a couple of interesting e mails per day at the most in any case? In other words maybe the orange only has so much juice in it no matter how hard one tries to squeeze it or make it squeeze itself? And if my friends told me to kindly stop writing them ANY e- mails unless I had something significant or important to say perhaps I still would write the same two “landmark” e mails per day but could dispense entirely with writing the other 48? Thereby sparing both myself and my friends a monumental nuisance?

    But before knowing how to handle the problem of my friends receiving just TOO MANY e mails from me, (and not knowing what to do with them and in particular if they also receive fifty from each of their other friends) they (and me too) would have to know which e mails are good and interesting and useful and which are a total waste of time.

    So that might be a first step to try to work on. But if and only if we think the problem (and the complex and multi- component system that produces it) can be managed. If it can’t be managed then I suppose we will just have to wait until the whole system crashes, in the same way we are already doing with currencies and the financial system.

  2. This has already been resolved to some extent in mathematics and physics. The point of a scientific paper is to publicize new research, and the "value add" of editing and peer review seems quite limited -- I'm sure any scientist asked off-the-record could name half a dozen fatally flawed papers in his own field alone. So, they simply push their work to the arXiv. It has been a rocky decade, but the arXiv seems to be working its way towards a replacement for traditional publication, and the community decides what discoveries are notable.

    The problem with this for social sciences and humanities is that it is unclear what research really is notable without the gatekeepers at the big journals. Anthropology had a huge scandal some years ago when the AAA actually came out against open access.

    Ugo, I believe part of the problem you are thinking about is the decreasing marginal return on scientific investment, which is a much bigger issue.

    1. Yes, everywhere there is a problem of decreasing marginal returns, but I think it shows much more in technology than in science.



Ugo Bardi is a member of the Club of Rome and the author of "Extracted: how the quest for mineral resources is plundering the Planet" (Chelsea Green 2014)