Massive Research Audit Discovers Thousands of Bogus References in Biomedical Studies
Almost 3,000 biomedical research papers have been found to contain made-up references in a comprehensive audit of 2.5 million scholarly articles. These findings mark the first comprehensive effort to quantify the extent of bogus citations in the biomedical field.
These dubious citations were detected through a sophisticated automated system that scanned publicly available biomedical articles. The results of the audit suggest that the issue of erroneous references is escalating at a rapid rate in the field of biomedicine, with the number of publications featuring made-up citations increasing twelvefold within a span of a few years.
The Scope of the Problem
Despite these alarming figures, some believe the situation could be far worse. A prominent AI researcher involved in the study believes the findings are just the tip of the iceberg and likely represent a modest estimate of the true scale of the problem.
Others in the field of scientometrics, the study of measuring and analyzing science, technology, and innovation, concur with this view, deeming the study a vital first step in addressing a significant issue. A separate analysis conducted earlier this year suggested that approximately 1.6% of publications from a particular year featured at least one citation corresponding to a non-existent publication.
Spotting the Fabricated Citations
In order to detect the fraudulent references, the research team developed a system that examined over 125 million citations found in 2.5 million papers. They primarily focused their analysis on 97 million citations that had valid unique identifiers assigned by publishers and preprint repositories or an ID given by a scholarly database.
Using advanced language models, the researchers flagged any inconsistencies between the title of each citation and the title of the document it supposedly referred to. The team also searched for these references across multiple scholarly databases. If the title of a citation couldn't be found in any of these databases, the researchers deemed it a bogus reference.
The Impact of Fabricated Citations
The audit identified over 2,500 papers containing one or two fake references, with an additional 246 papers containing three or more. The origin of these fraudulent citations, whether they were the result of human error or computer generation, remains unclear. However, the increasing prevalence of the issue suggests the possibility of artificial intelligence involvement.
In a manual examination of 500 flagged references, independent reviewers confirmed that seven out of ten citations were indeed fabricated. However, it's believed that the actual number of papers featuring bogus citations might be even higher.
One reason for the potential underestimation is the unreliability of certain scholarly databases in verifying references. Some bogus references have been found on these databases, but they don't lead back to legitimate publications, further complicating the issue.
Overall, this study shines a light on a growing issue in the field of biomedicine and underscores the importance of thorough citation verification in maintaining the integrity of academic research.
Almost 3,000 biomedical research papers have been found to contain made-up references in a comprehensive audit of 2.5 million scholarly articles. These findings mark the first comprehensive effort to quantify the extent of bogus citations in the biomedical field.
These dubious citations were detected through a sophisticated automated system that scanned publicly available biomedical articles. The results of the audit suggest that the issue of erroneous references is escalating at a rapid rate in the field of biomedicine, with the number of publications featuring made-up citations increasing twelvefold within a span of a few years.
The Scope of the Problem
Despite these alarming figures, some believe the situation could be far worse. A prominent AI researcher involved in the study believes the findings are just the tip of the iceberg and likely represent a modest estimate of the true scale of the problem.
Others in the field of scientometrics, the study of measuring and analyzing science, technology, and innovation, concur with this view, deeming the study a vital first step in addressing a significant issue. A separate analysis conducted earlier this year suggested that approximately 1.6% of publications from a particular year featured at least one citation corresponding to a non-existent publication.
Spotting the Fabricated Citations
In order to detect the fraudulent references, the research team developed a system that examined over 125 million citations found in 2.5 million papers. They primarily focused their analysis on 97 million citations that had valid unique identifiers assigned by publishers and preprint repositories or an ID given by a scholarly database.
Using advanced language models, the researchers flagged any inconsistencies between the title of each citation and the title of the document it supposedly referred to. The team also searched for these references across multiple scholarly databases. If the title of a citation couldn't be found in any of these databases, the researchers deemed it a bogus reference.
The Impact of Fabricated Citations
The audit identified over 2,500 papers containing one or two fake references, with an additional 246 papers containing three or more. The origin of these fraudulent citations, whether they were the result of human error or computer generation, remains unclear. However, the increasing prevalence of the issue suggests the possibility of artificial intelligence involvement.
In a manual examination of 500 flagged references, independent reviewers confirmed that seven out of ten citations were indeed fabricated. However, it's believed that the actual number of papers featuring bogus citations might be even higher.
One reason for the potential underestimation is the unreliability of certain scholarly databases in verifying references. Some bogus references have been found on these databases, but they don't lead back to legitimate publications, further complicating the issue.
Overall, this study shines a light on a growing issue in the field of biomedicine and underscores the importance of thorough citation verification in maintaining the integrity of academic research.