Brutal AI Peer Reviewer

What it does

Brutal AI Peer Reviewer allows users to upload a research paper (PDF) and instantly receive a structured critique. The system analyzes the paper and generates: A concise summary Strengths of the work Critical weaknesses and flaws Missing experiments or analysis Overstated claims Reproducibility concerns Final decision (Accept / Reject style) This helps users quickly understand the quality and limitations of their research.

How we built it

We built the system using Python and a lightweight AI pipeline. PDF text is extracted using PyPDF2, then passed into a transformer-based language model (HuggingFace FLAN-T5) to generate structured peer-review feedback. For the interface, we used Gradio to create a simple and interactive web UI that allows users to upload papers and view results instantly. The system is optimized to process only key sections of the paper to ensure fast inference and responsiveness in a hackathon setting.

Challenges we ran into

One of the biggest challenges was model size and performance. Large models like Mistral-7B were too heavy to run reliably in a hackathon environment, causing long load times and crashes. We had to pivot to a smaller, efficient model while maintaining reasonable output quality. Another challenge was extracting meaningful text from PDFs, as formatting inconsistencies sometimes led to incomplete or noisy inputs.

Accomplishments that we're proud of

We successfully built a fully working end-to-end system that takes a research paper and generates a structured AI critique in seconds. We also managed to balance performance and usability by switching to a lightweight model without breaking the core functionality. Most importantly, we created a tool that mimics real-world academic review behavior, which can be extremely valuable for students and researchers.

What we learned

We learned how to adapt quickly under constraints, especially when dealing with large AI models and limited compute resources. We also gained hands-on experience in integrating NLP pipelines with user interfaces and handling real-world data like PDFs. Additionally, we understood the importance of trade-offs between model quality, speed, and reliability in production systems.

What's next for Brutal AI Peer Reviewer

In the future, we plan to integrate more powerful models like Claude for higher-quality reviews, add section-wise analysis (abstract, methodology, results), and include scoring metrics similar to real conference reviews. We also aim to build a browser extension and integrate with platforms like Overleaf to provide real-time feedback duringBuilt with

Built With

• gradio
• huggingface-transformers
• python