Inspiration Marginalia started with a problem I kept running into as a newer researcher. I wanted to learn from AI research papers, but I often struggled to follow complex ideas and understand how all the pieces connected. The reading experience made that challenge even harder. A paragraph on one page might discuss a figure located several pages later. A citation might point to a reference at the very end of the paper. By the time I found the figure or reference and returned to the paragraph, I had lost the thread of the argument. I could have asked AI to summarize every paper, but that did not feel like the right solution. It felt like a shortcut that could limit my learning. Experienced researchers built their knowledge by reading papers closely, and I wanted to develop that ability too. Instead of creating a tool that reads papers for people, we wanted to create one that helps people become better readers.
What it does Marginalia is a research paper reader that keeps important context close to the passage you are reading. When the text mentions a figure, table, or citation, Marginalia connects that mention to its original source in the paper. A reader can view a graph while following the authors’ explanation instead of scrolling back and forth between pages. Figures can remain visible as the reader continues through the paper, making it easier to understand how the authors use the same evidence across different sections. Marginalia does not try to replace the paper with a summary. Its purpose is to help readers explore the original work while keeping the evidence and reasoning connected.
How we built it We used Codex throughout the development process, often across several CLI sessions and branches at the same time. Different sessions focused on areas such as research investigation, PDF extraction, evidence grounding, visual learning, and interface quality. When a user uploads a paper or enters an arXiv ID, Marginalia processes the PDF and extracts its figures, tables, captions, sections, and references. The reader then identifies where those items are mentioned and connects each mention to the correct material. The result is a reading experience where the paper remains at the center. The system helps surface useful context, but the reader still engages directly with the authors’ words and evidence. Challenges we ran into One of our biggest challenges was coordinating all the work happening at once. At some points, we had seven or eight Codex sessions working across different branches. Each branch contained useful progress, but merging everything safely was difficult. We had to make sure changes did not conflict and that no important work disappeared during the process. Managing the agents was also harder than we expected. When many sessions are active, it becomes difficult to remember what each one changed, which decisions it made, and whether its work still matches the direction of the overall product. PDF extraction presented another major challenge. Research papers use many different layouts, and a system can easily crop the wrong area, miss a figure, or connect a passage to the wrong item. We decided to prioritize precision over recall. If Marginalia is uncertain, it is better to show nothing than to confidently show the wrong evidence.
Accomplishments that we're proud of We are proud that Marginalia now supports the complete reading experience we originally imagined. A user can enter an arXiv ID, open the paper, select a figure mention, and see the relevant figure without losing their place. We also built a multi-agent architecture with specialized agents for research investigation, visual learning, evidence grounding, and interface quality. Instead of relying on one general-purpose AI layer, each agent has a focused responsibility. One example is our UI Guardian Agent. It checks generated graphs, diagrams, interactive experiences, motion, accessibility, and layout. At the same time, it is prevented from changing the scientific meaning of the material. Most importantly, we created an evidence-first architecture. Claims, passages, figures, tables, experiments, and citations remain connected to their original sources. Marginalia also distinguishes between relationships that come directly from the paper, relationships inferred or generated by AI, and relationships created by the user. It does not present every AI-generated connection as a fact.
What we learned Building Marginalia taught us that using more AI agents does not automatically make development easier. Agents can help a small team move quickly, but parallel work creates a new coordination problem. Responsibilities need to be clear, branches need to stay focused, and every merge needs to be reviewed carefully. We also learned that trust is essential when building for researchers. A polished interface does not matter if it connects a reader to the wrong figure or presents an inference as though it came from the paper. Every design decision needs to protect the relationship between an idea and its original evidence. Most of all, we learned that AI does not have to replace difficult work to be useful. It can remove unnecessary friction while leaving the meaningful learning process in the hands of the user.
What's next for Marginalia The next chapter for Marginalia is about learning from real users. One member of our team is part of Algoverse, an AI research program. Through that community, we hope to meet students who are beginning their research journeys and may be facing the same challenges that inspired Marginalia. We want to watch how they read with the product, understand where they still become confused, and learn which features actually help them develop as researchers. Their feedback will shape what we build next. Our goal is simple: help people read research papers more deeply, without taking the reading away from them.
Log in or sign up for Devpost to join the conversation.