NemoDoc

Your personal notetaking intelligence platform with agentic workflows

Comment

Inspiration

How can students really leverage AI for research and notetaking. Researching complex topics often means switching between documents, websites, and AI tools while manually verifying information.

I built NemoDoc to make AI-powered research more transparent and automated workflow.

What it does

NemoDoc is a personal notebook for annotating and studying. You can ask questions about a passage or across a notebook, follow citations back to source pages, and create study guides, flashcards, quizzes, and mind maps. The research agent can work with sources you approve, and its activity traces show how the work was done.

Its key capabilities include:

  • Multi-agent research: Specialized agents collaborate to plan research, gather information, review findings, and generate reports.
  • Document intelligence: Retrieve relevant information from documents and produce answers grounded in citations.
  • AI model integration: Support for NVIDIA Nemotron and other language models through configurable providers.
  • Tavily web search: Research agents use Tavily to find and extract public web sources for their research.
  • Controlled tool access: Enforce user-approved permissions for external sources and tools.
  • Persistent knowledge: Organize research through notebooks, saved notes, and reusable instructions.
  • Automated workflows: Schedule research tasks and deliver results through supported notification channels(email,whatsapp).
  • Transparent execution: Track agent decisions, tool calls, and execution history for debugging and auditing.

How we built it

Built the frontend with React, TypeScript, and Vite, with an Express backend for AI and agent features. Notebooks and source files are stored in the browser. For AI tasks, NemoDoc sends the relevant text or page images to the configured provider. The agent workflow uses the OpenAI Agents SDK for TypeScript, and NVIDIA Nemotron is the default model.

Technology stack:

  • Frontend: React, TypeScript
  • Backend: Node.js, Express
  • Agent orchestration: OpenAI Agents SDK
  • AI models: NVIDIA Nemotron and configurable LLM providers
  • Retrieval: Document processing, RAG, and external research integrations
  • Validation: Zod for structured outputs and runtime validation
  • Database: PostgreSQL
  • Infrastructure: Docker
  • Testing: Unit tests, API tests, and Playwright end-to-end tests

We designed a multi-agent workflow in which a planning agent breaks down research requests, parallel research agents collect evidence, and a reviewing agent synthesizes findings into structured, citation-backed reports.

We also implemented server-side permission checks, execution limits, persistent traces, and scheduling mechanisms to make agent workflows more controllable and reliable.

Challenges we ran into

One of our biggest challenges was making autonomous AI agents reliable rather than simply capable of generating responses.

Agent orchestration: Coordinating multiple agents required managing tool execution, parallel research, intermediate outputs, and cancellation.

Citation reliability: Ensuring that generated citations referenced permitted, retrieved sources was challenging. Valid citation formats alone do not guarantee that the cited evidence supports a claim.

Security: Preventing unauthorized tool access and keeping agents within approved source boundaries required backend-enforced permissions.

Reliability: Handling interrupted workflows, duplicate tool calls, and persistent execution state introduced additional engineering complexity.

Model integration: Supporting different language-model providers required careful handling of structured outputs and inconsistent model behavior.

Accomplishments that we're proud of

  • Built a working full-stack AI research application rather than a standalone chatbot prototype.
  • Implemented collaborative multi-agent research with planning, parallel execution, and review.
  • Integrated NVIDIA Nemotron into agent-powered workflows.
  • Added server-enforced permissions, audit trails, and persistent execution history.
  • Built automated tests covering unauthorized access, invalid citations, scheduling, and agent execution failures.
  • Implemented scheduled workflows and reusable notebook-based research.

What we learned

Useful AI study tools need more than fluent answers. They need clear evidence, dependable links to the original material, and ways for people to check and edit the result. I gained practical experience with agent orchestration, tool calling, retrieval pipelines, structured-output validation, and full-stack application development. I also learned that agent reliability must be designed and tested explicitly. An agent producing a plausible response does not necessarily mean its evidence is correct or that its execution was safe.

What's next for NemoDoc

Our next goal is to make NemoDoc a more powerful and dependable research workspace by improving accounts and sharing with friends, adding group study features and strengthening OCR for scanned materials.

Planned improvements include:

  • Semantic citation verification: Check whether cited passages actually support generated claims.
  • Agent evaluation framework: Benchmark factual accuracy, citation quality, tool usage, and instruction adherence.
  • Advanced document understanding: Improve retrieval across complex PDFs, tables, and multimodal documents.
  • Enterprise integrations: Connect research workflows with databases and external business systems through permission-controlled tools.

Our long-term vision is to turn NemoDoc into a trustworthy AI research assistant that can automate complex workflows, and explain how it reached its conclusions.**

Share this project:

Updates

Submission history