Slate

Example note page with 3d visual of a cone
GIF
GIF

Inspiration

Earlier this week, we were eating lunch with a friend of ours who is a math major. He told us that he's always wanted a tool that creates math visuals alongside OneNote-like multi-modal notes. We instantly knew what we'd be working on at Calhacks.

What it does

Slate allows users to create rich math visuals from voice prompts. These visuals exist within notes pages that can store handwritten and typed notes as well. Our multimodal solution solves problems faced by students, educators, and thinkers alike. It achieves this by relieving the user of the burden of drawing complex visuals like Parametric Equations, Scatterplots, ANOVA models, 3D trigonometric functions, Bar Charts, and Line Graphs.

screenshot displaying a regression plot on a note page

Click a button -> Describe the visual (can be broad like "Show me a line graph displaying unemployment data", or specific like "Graph the function z equals sin of x times y") -> your visual appears in your notes

gif showing a 3d trigonometric function z=cos(xy)

How we built it

We built Slate from the ground up, relying less on pre-built libraries than we initially intended. Our stack consisted of a Vite.js frontend, Express.js backend, and a MongoDB database. We've deployed to Vercel and MongoDB atlas, so Slatedraw.com is available to everyone everywhere.

Challenges we ran into

It turns out that attempting to create a OneNote clone on the web with added functionality is kinda difficult. What posed a particularly difficult challenge was deciding how to design the audio -> visual pipeline. We ultimately chose to enforce output from gpt-4o into JSON from a selection of pre-determined formats that could then be rendered on the client side using libraries like Three.js as well as a few bespoke libraries for statistical plots/charts.

We also wanted our app to be fully functional and usable by an uninitiated person. Unfortunately, this required a ton of extra work like figuring out how to save the state of note pages (handwritten, typed, graphs and all!) into objects we could send and receive between the client, server, and database. Eventually we had some breakthroughs, especially on the speed side of things (those visuals are beefy!).

gif showing a user flow