Skip to Main content Skip to Navigation
Journal articles

Prevalence of nonsensical algorithmically generated papers in the scientific literature

Guillaume Cabanac 1 Cyril Labbé 2
1 IRIT-IRIS - Recherche d’Information et Synthèse d’Information
IRIT - Institut de recherche en informatique de Toulouse
Abstract : In 2014 leading publishers withdrew more than 120 nonsensical publications automatically generated with the SCIgen program. Casual observations suggested that similar problematic papers are still published and sold, without follow-up retractions. No systematic screening has been performed and the prevalence of such nonsensical publications in the scientific literature is unknown. Our contribution is 2-fold. First, we designed a detector that combs the scientific literature for grammar-based computer-generated papers. Applied to SCIgen, it has a 83.6% precision. Second, we performed a scientometric study of the 243 detected SCIgen-papers from 19 publishers. We estimate the prevalence of SCIgen-papers to be 75 per million papers in Information and Computing Sciences. Only 19% of the 243 problematic papers were dealt with: formal retraction (12) or silent removal (34). Publishers still serve and sometimes sell the remaining 197 papers without any caveat. We found evidence of citation manipulation via edited SCIgen bibliographies. This work reveals metric gaming up to the point of absurdity: fraudsters publish nonsensical algorithmically generated papers featuring genuine references. It stresses the need to screen papers for nonsense before peer-review and chase citation manipulation in published papers. Overall, this is yet another illustration of the harmful effects of the pressure to publish or perish.
Document type :
Journal articles
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03242216
Contributor : Guillaume Cabanac <>
Submitted on : Sunday, May 30, 2021 - 5:29:38 PM
Last modification on : Monday, July 12, 2021 - 12:36:17 PM

File

CabanacAndLabbé2021.pdf
Publisher files allowed on an open archive

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Guillaume Cabanac, Cyril Labbé. Prevalence of nonsensical algorithmically generated papers in the scientific literature. Journal of the Association for Information Science and Technology, ASIS&T/Wiley, 2021, pp.1-16. ⟨10.1002/asi.24495⟩. ⟨hal-03242216⟩

Share

Metrics

Record views

55

Files downloads

40