Cite as: Brown A, Banik R. Peer-review: is it an unnecessary, dispensable process with alternative options or fortifying scientific rigor?. ASRA Pain Medicine News 2024;49. https://doi.org/10.52211/asra020124.015.

Con

Improving imperfect Science: Ways Beyond the Peer Review

by Ratan K. Banik, MD, PhD

Scientific studies are complex, and misinterpretation by reviewers or readers is frequent. At present, options for readers to criticize a scientific article are limited. A reader can write a letter to the editor criticizing an article in the journal that originally published it. However, most journals require that these letters are submitted within 1–3 months of the article’s publication. Letters are not typically sent for peer review. The editor and/or the section editor have sole discretion about whether to publish a letter, and there is no scope of appeal against their decision. Unlike other article types, a rejected letter to the editor cannot be published in other journals.

It is well known that scientists work under immense production pressure to get funding for their work, promotion, or recognition by their peers. The current peer review process, although expected to serve as a gatekeeper to detect flaws of scientific work, is inadequate.¹ In studies where editors intentionally inserted major errors into papers,^1–3 no reviewer was able to identify them entirely. Some reviewers did not spot any, and most reviewers spotted only about 25% of errors. As an editor of multiple journals, I often get reviews that are superficial and partially or completely contradict each other. It is expected that the reviewers, as volunteers, are willing to put sufficient time into a critical review. But in reality, reviewers are often overworked and overwhelmed with professional responsibilities, and have no time for volunteer work that will not even acknowledge their time.⁴ This problem is applicable to other journals where I submit my own work. Here is an example of two reviewers commenting on my paper that was submitted to one of the top journals in anesthesiology.

It is expected that the reviewers, as volunteers, are willing to put sufficient time into a critical review.

“Reviewer 1: This is a well-written, well-reasoned and very timely.... I completely agree with the conclusion of the authors that …... I have a few minor suggestions for improvement.

Reviewer 2: I found the article a bit unfocused in terms of the sequencing of the argument and the review of the literature as it seemed to bounce around a bit. I was left unclear as to the exact suggestion of the authors.”

It can, therefore, be argued that the peer review of a scientific article should not end with the journal's review process or be limited only to the letter of editor forum of the original journal within a specified time. Rather it should continue indefinitely in all related journals, scientific meetings, blogs, social media, and podcasts. I suggest that journals have an option for their readers to leave constructive criticism in the online version of the selected article.These communications will need to be bi-directional; therefore, they should contain responses from article authors prior to their publication. In the near future, a reader should have the ability to leave a review of a scientific article in a similar fashion they leave review for an Amazon product.¹ A large number of scientific studies published today are not reproducible due to the lack of the quality and integrity of the research.^5,6 It is expected that ongoing peer reviews and criticism of the scientific work will result in continual improvement of the process.

Pro

Fortifying Scientific Rigor: The Crucial Role of Peer Review in Advancing Knowledge

by Ashlyn V. Brown, MD, MBA, MS

Peer review in scientific journals constitutes a pivotal cornerstone in the realm of academic research, playing a crucial role in the advancement and validation of scientific knowledge. While it faces criticisms for its limitations and imperfections, the peer review process stands as an indispensable mechanism for enhancing the quality, credibility, and integrity of scholarly work.

Critics often highlight instances where major errors in submitted papers were overlooked during the peer review process. However, despite such challenges, peer review significantly contributes to improving the overall quality of published research. At its core, peer review serves as a stringent quality control system, evaluating the validity and reliability of research submissions before their publication.⁷ This rigorous evaluation process involves subjecting manuscripts to scrutiny by experts in the field, who assess various aspects, including the study's methodology, data analysis, conclusions, and adherence to ethical standards.⁸ While it might not be foolproof, peer review acts as a crucial checkpoint, detecting potential flaws, biases, or inaccuracies in submitted manuscripts, thereby mitigating the dissemination of erroneous or misleading information.⁹

Furthermore, the peer review system fosters accountability and transparency within the scientific community. Authors are accountable for the accuracy and reliability of their research findings, knowing that their work undergoes meticulous evaluation by peers. This fosters a culture of responsible conduct in research, reducing the likelihood of publishing inaccurate or flawed data.¹

Peer-reviewed publications hold substantial weight in academic and professional circles, acting as benchmarks for evaluating scholarly contributions. Such publications provide researchers with a credible platform for presenting their work, enhancing visibility and recognition within their respective fields.¹⁰ This recognition not only promotes intellectual growth but also motivates researchers to strive for excellence in their investigations.

Peer review system fosters accountability and transparency within the scientific community.

The essence of peer review lies not only in its ability to identify and rectify potential errors but also in its role as a catalyst for continuous improvement in scientific inquiry. It fosters ongoing dialogue and critical evaluation, encouraging researchers to refine their work, address limitations, and explore new avenues for investigation.^7,8,11Additionally, the constructive feedback received through peer review aids authors in presenting their research more effectively, improving the clarity and coherence of their manuscripts.⁸

While acknowledging the limitations of the traditional peer review model, there are ongoing discussions on enhancing this process. Encouraging ongoing peer reviews across various platforms, such as scientific meetings, blogs, social media, and podcasts, could foster broader discussions and engagement. Allowing readers to leave constructive criticism in the online version of articles, with responses from authors, could facilitate an ongoing dialogue.⁹ An opposing perspective might argue that extending the peer review process indefinitely across multiple platforms beyond the journal's review process could potentially hinder scientific progress. While encouraging ongoing discussions and criticism of scientific work is valuable, extending the review process to various channels like blogs, social media, and podcasts might lead to unregulated, unfiltered, and non-expert commentary. This proliferation of feedback, especially outside the domain of expertise, could introduce confusion and misinformation, diluting the credibility of scientific findings. 

Furthermore, making reviews similar to consumer product reviews on platforms like Amazon might oversimplify complex scientific evaluations and undermine the rigorous evaluation needed for scholarly research. The focus on perpetual reviews across multiple platforms may also distract researchers from conducting new investigations, leading to excessive emphasis on revisiting and revising older studies rather than advancing new knowledge. Additionally, while the aim to enhance reproducibility and quality of research is commendable, an unstructured, perpetual review system may not necessarily guarantee these improvements and could potentially overwhelm authors and reviewers, hampering the pace of scientific advancement.¹

In conclusion, while peer review has its imperfections, it remains an indispensable cornerstone in ensuring the integrity and advancement of scientific knowledge. Its role in maintaining standards of excellence, fostering critical discourse, and improving the quality of published research is undeniable. Continued efforts to refine and evolve the peer review process could further enhance its efficacy in navigating the complex landscape of scientific inquiry.

The authors thank Paul Casella for editing the con portion of this article.

References

1. Smith R. Peer review: a flawed process at the heart of science and journals. J R Soc Med 2006;99(4):178-82. https://doi.org/10.1177/014107680609900414
2. Schroter S, Black N, Evans S, et al. What errors do peer reviewers detect, and does training improve their ability to detect them? J R Soc Med 2008;101(10):507-14. https://doi.org/10.1258/jrsm.2008.080062
3. Godlee F, Gale CR, Martyn CN. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA 1998;280(3):237. https://doi.org/10.1001/jama.280.3.237
4. Bol L, Hacker DJ. Publishing in high quality journals: perspectives from overworked and unpaid reviewers. J Comput High Educ 2014;26(1):39-53. https://doi.org/10.1007/s12528-013-9073-7
5. Fanelli D. Is science really facing a reproducibility crisis, and do we need it to? Proc Natl Acad Sci2018;115(11):2628-31. https://doi.org/10.1073/pnas.1708272114
6. Ioannidis JPA. Why most published research findings are false. PLoS Med 2005;2(8):e124.https://doi.org/10.1371/journal.pmed.0020124
7. Solomon DJ. The role of peer review for scholarly journals in the information age. J Electron Publ2007;10(1). https://doi.org/10.3998/3336451.0010.107
8. Jefferson T, Rudin M, Brodney Folse S, et al. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database Syst Rev 2007;2007(2):MR000016. https://doi.org/10.1002/14651858.MR000016.pub3
9. Tennant JP, Ross-Hellauer T. The limitations to our understanding of peer review. Res Integr Peer Rev2020;5(1):6. https://doi.org/10.1186/s41073-020-00092-1
10. Meneghini R, Packer AL. Is there science beyond English? Initiatives to increase the quality and visibility of non‐English publications might help to break down language barriers in scientific communication. EMBO Rep2007;8(2):112-6. https://doi.org/10.1038/sj.embor.7400906
11. Wicherts JM. Peer review quality and transparency of the peer-review process in open access and subscription journals. Derrick GE, ed. PLOS ONE 2016;11(1):e0147913. https://doi.org/10.1371/journal.pone.0147913