aipen.ink

AI turns any text or math into real handwriting in fly.

Comment

Inspiration

I learn best through handwritten notes. Nowadays, I spend every day learning from AI. I wanted a way to save those AI generated insights in my own handwriting, to build a personal, beautifully written library that I can annotate and eventually pass down to my family, friends and students.

Right now, AI is accelerating other industries at breakneck speed. Video, image, and text generation rely on complex, autonomous workflows. But in education, adoption is painfully slow. It is well known that handwriting learning material has more retention power then digital AI output.

So for this hackathon, our team decided to build a bridge. We created an engine that takes my real handwriting and writes whatever I can imagine at super speed using Gemini. The challenge is huge, but the opportunity to change how we interact with educational AI is even greater.

What it does

In short: you give aipen.ink words or prompts. It gives them back written by hand as a paginated, infinite zoom vector PDF. Every character is drawn differently, equations keep their real structure, and graphs are drawn rather than pasted in as images. The output is a print ready vector PDF, so it stays crisp at any size.

For example, ask for Euler's identity and you get \( e^{i\pi} + 1 = 0 \) in the exact same hand as the prose around it. Ask for the quadratic formula, and the fraction bar, the radical, and the \( \pm \) are all drawn while preserving perfect mathematical structure:

$$x = \frac{-b \pm \sqrt{b^2 - 4ac}}{2a}$$

It runs in three places: a web app, a Chrome extension, and inside ChatGPT through an MCP server (currently waiting for official approval in the ChatGPT store).

How we built it

I should say this early: I do not write code. I describe what I want, read what comes back, notice when it is wrong, and argue with it literally day and night until it isn't. Two years ago that was a useless skill.

Attempt 1: LaTeX with a custom font. LaTeX already lays out beautiful maths, so give it a handwriting font. Straight, plain and simple. LaTeX does not draw an equation from one font. It assembles it from a maths font with specific metrics for radicals, integrals, braces and stretchy delimiters. My font had letters. It did not have a square root sign that knows how to grow. The result was handwritten letters with crisp digital operators in the same equation. Worse than either alone.

Attempt 2: the browser. KaTeX picked up about twenty of my glyphs and ignored the rest. MathJax's HTML output kept its own font for anything stretchy, because big brackets and fraction bars are not really characters. They are positioned boxes that happen to look like characters.

Attempt 3: MathJax SVG output. This is where it worked. SVG output does not give you text. It gives you shapes, and the coordinates where each shape goes. MathJax had already solved the hard problem, which is where every glyph belongs so the equation is correct and readable.

So I stopped fighting the layout and kept it, then replaced every glyph's path data with the matching glyph from my handwriting font, at the position MathJax had already computed. Then, for complex math operators, we use a free GitHub centerline extractor called Flo-mat and perfect-freehand.js to make it look written by a human. With Paged.js as our engine, it finally produces an industry standard vector PDF in under 5 seconds.

Then the humanisation: correct handwriting still looks fake if every a is identical. The font ships 20 natural variations of each character, drawn separately rather than generated, and every glyph picks one using a seeded PRNG mixing its position, its codepoint and a per render seed, then gets up to about 1.5° of rotation and a slight vertical nudge. Because the seed is deterministic, pressing Humanize with a new seed gives you the same words in a different variation, pulled from an R2 bucket of our private 20 variation natural fonts.

Now for the final magic recipe. We hooked the Gemini API under the hood to understand our app from the ground up and teach it. We made it read every line of code and engineered the curated, tailored, simplified prompt for the API call so it can understand our needs in seconds. With several tries, Gemini started giving the data we need with my handwriting font. Goal reached!

It runs on Google Cloud Run, with Firestore and Firebase Auth, Gemini for generation, GCS and Cloudflare R2 for assets, and Playwright driven headless Chromium for rendering.

Right now it supports:

  • \frac, \sqrt, integrals, and matrices, all rendered in the same hand as the prose.
  • Hand drawn graphs and charts, generated from a description.
  • Over 25 European languages with correct accents.
  • Your own handwriting, where you upload it once and write everything in your hand.
  • Print ready vector PDFs.

Challenges we ran into

Making 20 variations of each character that support math is not a task someone can imagine. Plus, since there is a lack of material on how to render math in handwriting on the web, the project was a dead end from the start. But with frontier intelligence harvested from Gemini and Claude, and our persistent motivation to be useful somehow, we successfully managed to hack MathJax, and made the first app on the web that generates math in natural handwriting authentically.

Focusing on the technical side a lot, we forgot our marketing side, so there was a distribution problem too.

A traffic spike. 28.5 million Cloudflare R2 operations in three weeks costing us $10. Found and resolved; the following month was 1,810 operations and $0.00.

Accomplishments that we're proud of

  • The maths actually works. That is the thing nobody else does, and the team is super proud of it. Student, creators, teachers, small business, and disable people can now generate tailored handwriting notes and assets at speed of light!
  • 90 users acquired during the period, 47 Chrome extension installs, 222 unique MCP clients across 1,466 requests at an 85% success rate.
  • 66% of extension installs are on ChromeOS, which is school hardware.
  • ~12,000 reads of the free handwritten notes, with $0 spent on marketing.
  • Gemini cost $1.58 for three months of production traffic. Total spend for the period was $149.44.
  • An agent inside ChatGPT chooses blue ink and grid paper that nobody asked for, then renders a PDF right inside the ChatGPT app. The same happens with Claude in developer mode.

What we learned

The hard part was making the real handwriting and converting it to digital format, with accurate unicode assignment using the PUA area so our system is the standard as per MathJax.

The output has no text layer and it makes the format a poor fit for anything needing machine readable text.

Distribution is harder than engineering, which is something we still need to figure out.

What's next for aipen.ink

  • Voice dictation. Speak, and it appears in your own hand, which will be most useful for the accessibility case that started this.
  • Better graphs. They are about 80% right. Axis labels collide, and curves that should look confident look hesitant. The remaining 20% fails in ways obvious to a human and hard to specify to a machine.
  • Per subject templates like lab reports, problem sets, and lecture notes, each with the layout conventions that the subject expects, instead of one page style for everything.
  • Expanding free notes reach. We already have twelve thousand people reading our PDF notes for free!

Built With

Share this project:

Updates