Inspiration

As students constantly juggling dense lecture slides, multi-page PDFs, and complex STEM diagrams, we faced a recurring dilemma: typing notes is fast, but writing by hand is scientifically proven to boost memory retention and conceptual understanding. Standard AI summary tools only produce generic text paragraphs—leaving out the tactile, visual, and active-recall benefits of handwritten study. We wanted to solve this by building a feature bridge for digital note-taking apps like Notability.

What it does

Study Ink is an AI-powered local handwriting and diagram studio that bridges typed efficiency with handwritten retention. It automatically extracts text and slide geometry from uploaded PDFs, images, or typed notes, converting them into continuous, full-width handwritten study sheets. Beyond basic text, it reconstructs complex STEM diagrams (such as chemical bonds and graph algorithms) into editable vector drawings and automatically generates active-recall worksheets with fill-in-the-blank blanks to test your knowledge.

How we built it

Backend Processing: Built with a Python HTTP backend using PyMuPDF (fitz) and pytesseract to parse structured text and perform local multi-stage OCR on image-heavy slides.
Multimodal Vision: Integrated Claude Vision API to parse complex spatial slide geometry and technical diagrams into structured JSON coordinates.
Digital Ink Engine: Utilized custom glyph and polyline trajectory mapping (handwriting.py) to convert extracted text and vector shapes into realistic handwritten sheets.
Frontend UI: Built a responsive studio with raw HTML5 Canvas, CSS Grid, and asynchronous JavaScript fetch API for seamless full-width document scrolling.

Challenges we ran into

Complex Slide Extraction: Standard OCR frequently broke on multi-column slide layouts and diagram labels. We built a hybrid fallback engine that measures text density and dynamically switches between native text parsing and high-DPI OCR.
Continuous Document Layouts: Transitioning from rigid page-based rendering to fluid, full-width continuous scrolling without line clipping or aspect-ratio distortion required precise CSS styling and math updates in our backend line wrapper.

Accomplishments that we're proud of

Successfully turning static, complex slide diagrams into editable, handwritten vector drawings.
Developing a robust backend extraction pipeline that runs safely off the frontend UI to prevent clutter and crashes.
Creating an active-recall pass that automatically hides around 25% or less of key concepts to generate instant practice worksheets.
Personalized Ink Profiling, which allows users to upload a single sample photo of their handwriting to auto-generate a 1:1 custom stroke profile.

What we learned

Spatial Diagram Parsing: How to enforce strict JSON schema outputs from vision models to extract usable (x, y) coordinate bounding boxes for complex spatial diagrams.
Handwriting OCR & Verification: How to utilize computer vision to read raw, handwritten student ink input and intelligently evaluate its conceptual correctness against the active-recall solution key.
Dynamic Image Payload Streaming: Best practices for streaming dynamic image payloads via Base64 and blobs between a lightweight Python backend and frontend canvas layers.

What's next for Study Ink

Direct Notability Plugin: Packaging the system as a native iPadOS extension/plugin for Notability and GoodNotes.
Interactive Apple Pencil Support: Enabling real-time handwriting recognition on Apple Pencil input for instant grading on active-recall worksheets.