Most discussions about artificial intelligence in research evaluation focus on whether AI can match or outperform human peer reviewers. This framing, however, misses out on an actual opportunity we have. The more pressing issue is not whether machines can mimic the existing peer-review process, but whether that process itself is adequate for the future of science. As research increasingly generates massive, multi-faceted datasets from 'omics' approaches, the critical question is: can the human-centric model of self-governed scientific oversight effectively manage this new level of complexity?

The primary goal of scientific evaluation has been to determine whether a research proposal’s underlying hypothesis is supported by sound preliminary data and whether the planned experiments are suitable to test this hypothesis. As I argued elsewhere, however, this approach appears to be falling short in light of a reproducibility crisis that even prompted our President to issue an Executive Order earlier this year (Restoring Gold Standard Science).

The reproducibility crisis threatening the credibility of modern science is rooted in multiple contributing factors. On one hand, high-throughput technologies are generating datasets so vast and complex that scientists struggle to extract meaning from them. Reviewers' individual egos, personal biases, time constraints, and institutional politics also don't help. On the other hand, the rise of AI-generated content poses a new threat: a flood of plausible-sounding but scientifically flawed proposals and manuscripts that overwhelms the governing systems. Concerned about the impact of high-volume submissions on the integrity of the review process, the NIH issued a notice in July 2025 limiting the number of research proposals an investigator can submit to six per year (Fairness and Originality in NIH Research Applications).

AI in its current iteration is notorious for producing statements without any merit to truth. A recent example being the FDA’s very own Elsa, which has received mixed reviews. The misinformation, aka AI hallucination, stems from algorithms that seek user engagement and satisfaction, rather than a malicious intent to spread disinformation. Some people refer to this as botshit, making up bullshit, as opposed to lying. Considering the massive rise of AI-generated scientific content, one might rightfully wonder whether AI-empowered peer review processes can effectively combat the rise of AI-generated scientific fraud and misinformation. Or are we entering a perpetual arms race with the AIs used by governing agencies and bad actors?

Rather than seeing AI as an adversary, a better approach is to fight fire with fire. My argument is simple: I am advocating for the incorporation of AI into the review process as an independent third peer reviewer, evaluating research alongside two human reviewers. Investigators would be required to address all raised points in their response, regardless of whether they were raised by humans or the AI. The final judgment call on whether a study has merit would then be made by the review panel for research proposals or the editor for scientific articles.

So instead of relying solely on human reviewers, an extended review process leveraging AI could simply be described as an additional pair of eyes looking at and commenting on a study. Yet, this would be an extremely powerful set of eyes, with a focal range extending far beyond human capabilities. Only AI can manage the complexity of modern data and at the same time detect fabricated content. And while AI might still have its own flaws and biases, they are substantially different from the ones to which humans succumb. This is how we can leverage AI reviewers to fight AI-generated meritless science, building a new gold standard for science.

#artificialintelligence #science #reproducibilitycrisis