Scientific misconceptions, publication stress and criticism of molecular ecology as a research field

Posted by Erik Svensson

At Juha Merilä's research group blog, EGRU-blog, one often finds very interesting and provocative posts, that stimulates self-reflection and critical thinking. Here is one such post, which raises some critical questions about the field of molecular ecology and the lack of rigour among some of the scientists defining themselves as belonging to this novel field.

This short post actually refers to a recent Invited Review, which is likely to upset some molecular ecologists, as it is very provocative and questions much of the research practices in this very young and technologically-oriented discipline. I do not necessarily endorse everything in this article, and some points that are discussed are beyond my expertise and research interests. As for myself, I do not get very upset or feel very threatened by the message, because I am not a molecular ecologist (and will never become one), even though we have used molecular techniques in our research lab for several years now, and published several papers in the journal Molecular Ecology as well (e. g. this, this and this).

But using molecular techniques, like we have done in these studies, and even endorsing them, is not the same thing as being a molecular ecologist, in my opinion. It is not even enough to publish in the journal Molecular Ecology, I think. I, for myself,  would never define myself as molecular ecologist. Rather, I define myself as an old-fashioned evolutionary biologist interested in the ecological aspects of evolutionary change. Or sometimes I simply define myself an evolutionary ecologist, who is prepared to use observations, field and lab experiments, quantitative genetics and molecular techniques, depending on what is needed and what question that is being adressed.

In contrast, molecular ecology as a field, as I perceive it, is a primarily a discipline defined by techniques and the use of molecular markers, rather than being defined by research questions. And that is why I have never been very interested in this field, as I tend to be more interested in conceptual problems in ecology and evolution, while not being hostile towards new techniques, when they help to solve these classical problems (which is not always the case, however). Molecular Ecology partly grew out from behavioural ecology during the eighties and nineties, as new molecular methods for determining paternity in birds and other animals were developed (first DNA-fingerprinting and later microsattelites). Later, the field came to include many other research questions being adressed by the use of molecular markers, such as phylogeography and molecular population genetic structure etc.

The current review is - interestingly - published in Molecular Ecology - which I think is to the benefit of this outlet as it shows some self-criticism of the same field that the journal is built upon. Hopefully, this article will help to promote self-reflection and critical thinking, both among molecular ecologists (the main target), but also other biologists using molecular techniques. The paper is Open Acess and can be downloaded here. 

 Here are some excerpts, and quite critical and provocative quotations from the paper (Abstract and full reference given below this post):

"Many misconceptions in the various subdisciplines of molecular ecology arise as a consequence of the huge amount of data that can be relatively easily and rapidly generated and analysed. There are many more automated DNA sequencers than classes in population genetic theory, and as self-educated molecular ecologists contribute in professional service, we sometimes see misconceptions perpetuated by journal authors, reviewers and editors."

And:

"At the end of this review, many readers will still believe that if they can properly format data for mega (Tamura et al. 2011) or arlequin(Excoffier et al. 2005), they do not need population genetic theory, they can pick it up along the way, or all the information they need is in the manual. Considering the high error rate (49.9%) in publications of a simple calculation of a population genetic parameter revealed bySchenekar & Weiss (2011), our answer is this: about half of you are right."


Finally, here is some very harsh criticism against the data publication culture in the field of molecular ecology,  and the tendency to crank out too many papers, with too many authors and ignoring much of the classic work that has already been published and which would be relevant to cite:

"As a result of these technological advances, not just in DNA sequencing, but also in computing power and web-based manuscript review, the trend has been for more data per publication, shorter time to publication and more publications per author in the field of molecular ecology. For example, 181 recently hired tenure-track faculty world-wide had an average of 2.9 years of postdoctoral experience and an average of 11.75 (maximum 45) peer-reviewed publications at the time of hire (Marshall et al. 2009). By comparison, a search of Web of Knowledge (Thomson Reuters formerly ISI) most highly cited authors returned 12 that completed their doctoral dissertations before 1990 and whose CVs are available online. These researchers produced an average of only 4.6 ± 0.2 publications by three years after graduation. Likewise, a dissertation in one of the disciplines of molecular ecology prior to 1990 was typically based on sequences from a single-locus and samples sizes of tens of individuals, whereas today dissertations are routinely expected to include several hundred lengthy sequences for multiple genes.

This increased expectation and rate of publication also results in ever more submissions to journals, which increase rejection rates due to space limitations, and that builds pressure on authors to claim the first, biggest or best study for submissions to high-impact factor journals. Claiming to be the first study to ever show some result is facilitated by an eroding knowledge of the classic literature."

STEPHEN A. KARL1, R. J. TOONEN1, W. S. GRANT2, B. W. BOWEN1

Article first published online: 11 MAY 2012DOI: 10.1111/j.1365-294X.2012.05576.x 

Abstract: The field of molecular ecology has burgeoned into a large discipline spurred on by technical innovations that facilitate the rapid acquisition of large amounts of genotypic data, by the continuing development of theory to interpret results, and by the availability of computer programs to analyse data sets. As the discipline grows, however, misconceptions have become enshrined in the literature and are perpetuated by routine citations to other articles in molecular ecology. These misconceptions hamper a better understanding of the processes that influence genetic variation in natural populations and sometimes lead to erroneous conclusions. Here, we consider eight misconceptions commonly appearing in the literature: (i) some molecular markers are inherently better than other markers; (ii) mtDNA produces higher FST values than nDNA; (iii) estimated population coalescences are real; (iv) more data are always better; (v) one needs to do a Bayesian analysis; (vi) selective sweeps influence mtDNA data; (vii) equilibrium conditions are critical for estimating population parameters; and (viii) having better technology makes us smarter than our predecessors. This is clearly not an exhaustive list and many others can be added. It is, however, sufficient to illustrate why we all need to be more critical of our own understanding of molecular ecology and to be suspicious of self-evident truths.

Rankings of "Open Access"-journals: PLoS ONE rocks!

As several of you might know, I have been complaining about how conservative attitudes some of my colleagues have towards "Open Acess"-publishing in general, and towards PLoS ONE in particular. This is quite frustrating, as me and several of my colleagues in the Editorial Board are doing our best to promote this new "revolutionary" journal, with the hope that it will in the long run change the entire publication landscape - to the benefit of both scientists, readers, taxpayers and the general public.

There are two classical objections against PLoS ONE by ecologists and evolutionary biologists. First, some of them are afraid to publish in PLoS ONE since it is not yet listed in Thomson's databases (ISI), such as "Web of Science". This is the database from which Impact Factors (IF:s) are calculated, and Thomson actually has a monopoly (!) on how to calculate IF:s. Since Thomson has up until now refused to list PLoS ONE in their data-base, some of my scientific colleagues are afraid that work published in that journal will be "forgotten" or not appreciated by the scientific community.

This objection partly shows a lack of knowledge and the misunderstanding about citation data-bases as reflecting some kind of "objective truth". In reality these data-bases are run by commercial companies with their own agendas. ISI is certainly not the only data-base, and it is one of the slowest to list new publications and it does only cover a small minority of all scientific journals. Scopus, for instance, covers more journals and so does probably also Google Scholar and PubMed. Thus, there is luckily severe competition among different data-bases, and hopefully ISI will soon become outcompeted and run out of business by better and faster alternatives with better litterature coverage. ISI is for the scientific world what Microsoft is for the computer world: a mean big company that we should all hate!

The second objection is that PLoS ONE has a publication policy that aims for "technical quality", rather than arbitrary and subjective criteria for acceptance of papers, such as "novelty". In that respect, PLoS ONE differs significantly from all other journals, including Science, Nature, PNAS and PLoS Biology. The philosophy of PLoS ONE is that it is the future scientific readership that should judge whether a paper is "significant" or not, not a few subjective referees or journal editors. The scientific process does not end with the publication of a paper, it starts. This is when a paper is read, discussed and (hopefully) gets cited and thus "accepted" as being important by other scientists.

Some researchers consider this as a weakness of PLoS ONE, and fears that it will become a "dumping ground" for poor quality papers that have not been published elsewhere. I, and many others, on the contrary view this as as strength of PLoS ONE, and I can honestly not say that I think that PLoS ONE has become such a vehicle for bad papers that some feared that it would become. But I am of course biased in my views, since I am involved in the journal, and it is up to others to decide about this.

Given the inherent problems with impact factors and how they are increasingly becoming "corrupt" and the arbitrary parts of traditional publication (biased referees, commercial data-bases, unfair editors etc.), I think we probably all agree that there is a need for newer ranking criteria of journals. These criteria could be based on things like number of downloads of articles, number of citations, more or less informal "ranking lists" by the scientific community, blog coverage, coverage in media etc. None of these rankings are likely to be perfect or reflect the final "truth", but they would provide a nice complement to the traditional measures, such as impact factors of journals.

The blogger "The Open Source Paleontologist" have done some such ranking lists of OA-journals, and you can read about them here, here and here. Although these ranking lists have their limitations and only deal with the paleontological science community, they are nevertheless interesting and revealing. I predict that we will see many more of these lists in the future, and I bet that the traditional "impact factor" hysteria, will soon go away (to the benefit of all science).

Not surprisingly, and pleasingly to me, PLoS ONE does very well in these ranking lists: it is always in the top 15 list of journals, and often among the top 5. Way to go, PLoS ONE!!! I am delighted. And the young scientists among you who reads this should of course not be afraid of publishing in PLoS ONE in the future, it will benefit your careers.