The Motivation Behind the Revival

The primary motivation for reviving and modernizing RobotReviewer stemmed from observing a persistent demand among researchers. Many senior evidence synthesis researchers and methodologists, in particular, expressed a lack of trust in modern Large Language Models (LLMs). They were concerned about LLMs' tendency to hallucinate and their general-purpose training, which lacks the specialized rigor required for evidence synthesis with tasks such as risk of bias.

These senior researchers continued to place their trust in a traditional machine learning model based software i.e. RobotReviewer, specifically trained to identify risk of bias and PIO (Population, Intervention, Outcome) domains with extracted evidence, trained using 12,808 RCTs across various collaborations.

I personally couldn't let such a valuable software be abandoned. I decided to revive it with a clear goal: to modernize the pipeline, to allow modern researchers to use it, and to explore the dynamic between LLMs, traditional ML models, and the seasoned intuition of an experienced researcher, looking past the current LLM hype.

Architecture & Models

This project preserves the machine learning core of the original RobotReviewer while modernizing the infrastructure.

The ML Pipeline

1. PDF Parsing: Uses PyMuPDF (fitz) to extract text and spaCy to segment sentences. No Java/GROBID dependency required.
2. Feature Extraction: Uses HashingVectorizer (scikit-learn) to convert text into high-dimensional sparse matrices.
3. Classification:
   • MiniClassifier: A lightweight Linear SVM wrapper that loads pre-trained .npz weights.
   • SVM-Only Pipeline: CNN models have been removed due to TensorFlow compatibility issues on Python 3.11–3.12 and are not required for accurate predictions.

Model Redistribution Note

This repository contains joblib-converted model artifacts originally developed in RobotReviewer. The old pickle model files were compressed and redistributed into .joblib format using the convert_models.py script and respective directories to ensure better compatibility and smaller file sizes for modern Python environments.

Hugging Face Repository

The users can access the RCT-Reviewer Hugging Face Repository at:
huggingface.co/Aurumz/RCT-Reviewer

PROJECT ARCHITECTURE
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
│
├──  Entry point detected
│
├──  File Structure
├──  .dockerignore
├──  .gitattributes
├──  .gitignore
├──  .zenodo.json
├──  INFO.md
├──  LFS push:pull guide.md
├──  LICENSE.txt
├──  README.md
├──  convert_models.py
│
├──  data/
│   ├──  bias/
│   │   ├──  bias_doc_level.npz
│   │   ├──  bias_sent_level.npz
│   │   ├──  bias_ab.npz
│   │   ├──  bias_prob_clf.pck
│   │   ├──  domain_clf.pck
│   │   ├──  overall_clf.pck
│   │   ├──  drugbank.pck
│   │   └──  humans/
│   │       ├──  AC.hdf5 / AC.json / AC.pickle
│   │       ├──  BOA.hdf5 / BOA.json / BOA.pickle
│   │       ├──  BPP.hdf5 / BPP.json / BPP.pickle
│   │       └──  RSG.hdf5 / RSG.json / RSG.pickle
│   ├──  pico/
│   │   ├──  I_idf.npz / I_model.npz
│   │   ├──  O_idf.npz / O_model.npz
│   │   ├──  P_idf.npz / P_model.npz
│   ├──  rct/
│   │   ├──  rct_svm_weights.npz
│   │   └── ... (CNN artifacts retained but unused)
│   └──  vocab/
│       └──  embeddings.200d.trimmed.npz
│
├──  rct_reviewer/
│   ├──  __init__.py
│   ├──  app.py (Joblib Mode)
│   ├──  app1.py (Pickle Mode)
│   ├──  app2.py (HF Hub Mode)
│   ├──  config.py
│   ├──  processors/ (bias_robot, pico_robot, rct_robot)
│   └──  ui/ (streamlit components)
│
└──  requirements.txt

🔄 Differences from Original RobotReviewer

Note on SVM vs CNN

This project uses a Linear SVM-only pipeline instead of the original SVM + CNN ensemble. CNN models were removed because they depend on TensorFlow/Keras .h5 files which break on Python 3.11–3.12. The SVM model already contains the full predictive signal with negligible accuracy loss (~0–2%), is faster, and ensures reproducibility across all systems.

Online Deployment: The default hosted version at rct-reviewer.streamlit.app uses app2.py (Hugging Face Hub). The repository for the online deployment is RCT-Reviewer/RCT-Reviewer-Online.

How to run RCT-Reviewer Offline?

Prerequisites

Before you begin, ensure you have the following installed:

1. Python 3.12: This project is optimized for the latest Python version.
2. Git: For cloning the repository.
3. Git LFS (Large File Storage): Required if you plan to run the offline modes (app.py or app1.py) to download pre-trained ML model weights.

⚠️ Note: This project strictly requires Python 3.12. It has been tested and verified to work perfectly on this version.

🛠️ Installation & Usage

This repository contains the main model files along with the code required to run RCT-Reviewer. There are three ways to run this application locally:

1. app.py: Uses .joblib and .npz files (Modernized local storage).
2. app1.py: Uses .pickle, .pck, and .npz files (Legacy local storage).
3. app2.py: Connects to the Hugging Face Hub repository Aurumz/RCT-Reviewer (No large local files needed).

[Recommended to run app2.py over other .py to run latest version.]

General Setup (Required for all modes)

Clone the repository and set up the virtual environment:

git clone https://github.com/aurumz-rgb/RCT-Reviewer.git
cd RCT-Reviewer

# Create virtual environment
python3.12 -m venv .venv

# Activate it
# Linux / macOS:
source .venv/bin/activate
# Windows:
.venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Download NLP Model (Required for all modes)
python -m spacy download en_core_web_sm

Running Specific Modes

Choose one of the following methods to run the app.
[Recommended to run app2.py over other .py to run latest version.]

git lfs install
git lfs pull

python -m streamlit run rct_reviewer/app.py

git lfs install
git lfs pull

python -m streamlit run rct_reviewer/app1.py

python -m streamlit run rct_reviewer/app2.py

🚨 Troubleshooting

Q: I get FileNotFoundError: data/pico/P_model.npz
A: You likely skipped the Git LFS step. Run git lfs pull to download the actual model weights (only required for app.py and app1.py).

Q: I get ModuleNotFoundError: No module named 'rct_reviewer'
A: Ensure you are running the command from the root directory, or set your PYTHONPATH:

export PYTHONPATH=$PYTHONPATH:$(pwd)

Q: Can't find model 'en_core_web_sm'
A: You forgot to download the spaCy model. Run:

python -m spacy download en_core_web_sm

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for details on how to suggest additions or changes.

Acknowledgements

RCT-Reviewer is a modernized version of the original RobotReviewer. I extend my sincere gratitude to the original authors: Iain J. Marshall, Joël Kuiper, Edward Banner, and Byron C. Wallace for their foundational work in biomedical NLP and for releasing the project as open-source.

I would also like to thank all contributors and collaborators involved in the RobotReviewer ecosystem, including the Cochrane Crowd and the research teams at UPenn, Northeastern, and UCL, whose efforts in data collection and model development made this tool possible.

Additionally, I would like to acknowledge the use of RikaiCode (Code Repository Context Generator) and GLM-4.7, which were invaluable in analyzing and understanding the complex logic of the original RobotReviewer codebase, as well as assisting in the development and modernization of RobotReviewer.