What is RCT-Reviewer?

RCT-Reviewer is a free, open-source ML tool that performs automated Risk of Bias (RoB) assessment of Randomized Controlled Trials (RCTs). It is a modernized, standalone version of RobotReviewer, trained on 12,808 clinical trial PDFs, and is designed to assist systematic reviewers, Cochrane contributors, and meta-analysis researchers.

Who should use RCT-Reviewer?

RCT-Reviewer is built for systematic reviewers, Cochrane contributors, meta-analysis researchers, evidence-synthesis practitioners, clinical trial methodologists, biomedical researchers, medical informatics researchers, and PhD students or postdocs conducting clinical research.

How accurate is RCT-Reviewer at assessing Risk of Bias?

RCT-Reviewer achieves 71.0% accuracy on Risk of Bias judgments, performing within 8% of human expert consensus (which stands at 78.3%). It also demonstrates 87% precision and 90% recall for identifying the exact text snippets that support each bias judgment.

How does RCT-Reviewer work as a third-party tiebreaker?

Standard systematic-review guidelines require two independent human reviewers. When those two reviewers disagree, RCT-Reviewer provides an instant, objective, data-driven third opinion to help resolve the tie. Real-world feasibility studies show that human reviewers accept the tool's judgments at the same rate they accept their human peers' judgments (Risk Ratio 1.02).

What input does RCT-Reviewer accept?

RCT-Reviewer accepts clinical trial PDFs. Upload a trial PDF and the tool returns automated Risk of Bias judgments along with the supporting evidence snippets highlighted directly in the document.

Is RCT-Reviewer free to use?

Yes. RCT-Reviewer is completely free and open source under the GNU GPL v3.0 license. No signup or account is required. You can use it online or run the source code locally from GitHub.

How is RCT-Reviewer related to RobotReviewer?

RCT-Reviewer is a modernized, standalone derivative work of RobotReviewer, originally developed by Marshall, Kuiper, and Wallace. It builds upon RobotReviewer's original machine-learning models trained on 12,808 randomized controlled trials and is released under the same GNU GPL v3.0 license.

How do I cite RCT-Reviewer in my research?

RCT-Reviewer provides a built-in citation generator on its homepage supporting APA, Vancouver, MLA, Chicago, Harvard, BibTeX, and RIS formats. The recommended citation references the Zenodo DOI: https://doi.org/10.5281/zenodo.20618338. You should also cite the original RobotReviewer paper.

Where can I find the RCT-Reviewer source code and model?

The RCT-Reviewer source code is available on GitHub at https://github.com/aurumz-rgb/RCT-Reviewer, the trained model is hosted on Hugging Face at https://huggingface.co/Aurumz/RCT-Reviewer, and the archived release with DOI is on Zenodo at https://doi.org/10.5281/zenodo.20618338.

Can RCT-Reviewer replace human Risk of Bias assessment?

No. RCT-Reviewer is intended as a third-party reference and tiebreaker for systematic reviews. It is designed to complement human reviewers, not replace them. Standard guidelines still require two independent human reviewers, and RCT-Reviewer can be used when those reviewers disagree.

RCT-Reviewer: Automated Risk of Bias Assessment Tool for RCTs

Why Use RCT-Reviewer?

RCT-Reviewer is designed as a third-party tiebreaker reference for systematic reviews. Standard guidelines require two independent human reviewers; when they disagree, this tool provides an instant, objective, and data-driven third opinion to resolve ties.

Near-Human Accuracy

The system achieves 71.0% accuracy for Risk of Bias judgments, performing within <8% of human expert consensus (which stands at 78.3%) [1].
Highly Precise Extraction

In a randomized Cochrane user trial, the models demonstrated 87% Precision and 90% Recall for identifying the exact text snippets supporting the bias judgment [2].
Validated Acceptance

Real-world feasibility studies show that human reviewers accept the tool's judgments at a rate equal to that of their human peers (Risk Ratio 1.02) [3].
Rigorous Methodology

Developed by Marshall, Kuiper, and Wallace, the models were trained on 12,808 clinical trial PDFs using "distant supervision" to ensure high-quality classification without prohibitive manual labeling costs [1,4].

References

Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. Journal of the American Medical Informatics Association. 2016;23(1):193-201. doi
Soboczenski F, et al. Machine learning to help researchers evaluate biases in clinical trials: a prospective, randomized user study. BMC Medical Informatics and Decision Making. 2019;19(1):96. doi
Nussbaumer-Streit B, et al. Automating risk of bias assessment in systematic reviews: a real-time mixed methods comparison of human researchers to a machine learning system. BMC Medical Research Methodology. 2022;22:160. doi
Marshall I, Kuiper J, Wallace B. Automating Risk of Bias Assessment for Clinical Trials. IEEE Journal of Biomedical and Health Informatics. 2015;19(4):1406-1412. doi

Citation

If you use this software in your research, please cite both RCT-Reviewer and the original RobotReviewer paper. Select a reference and format below to copy or download your citation.

Select Reference:

Select Format:

Why Use RCT-Reviewer?

Near-Human Accuracy

Highly Precise Extraction

Validated Acceptance

Rigorous Methodology

References

Citation