Magic Slate

Inspiration

It all started with a trip down memory lane, back to the days of MS Paint. Remember the sheer joy of just... doodling? No fancy tools, no pressure, just pure, unadulterated digital scribbling. That simplicity, that freedom, always stuck with me.

Then, there was the pragmatic side: Apple Calculator's math notes feature. It was incredibly useful, instantly solving equations you'd just jotted down. I found myself using it constantly for quick calculations, appreciating how it bridged the gap between handwritten input and digital solution. A thought began to bubble: what if we could marry these two seemingly disparate worlds? What if the spontaneous creativity of MS Paint could meet the intelligent processing of math notes?

What it does

Magic Slate is my answer to that "what if." It's a versatile digital canvas where your ideas come to life. You can doodle freely, just like in MS Paint. But the magic happens when our AI, powered by Gemini, steps in. It can take your simple sketch—say, a rough outline of a tree—and transform it into a vibrant, detailed illustration. The cool part is, you can even customize the AI generation by choosing different styles like watercolor, pop-art, or even a whimsical fantasy look. Beyond art, it also intelligently recognizes and solves handwritten mathematical equations, just like Apple's math notes, making it a powerful tool for both creative expression and quick problem-solving.

How we built it

The core idea was to create a seamless blend of free-form drawing and intelligent AI processing. We built the front-end with a focus on a simple UI – making sure the drawing experience was fluid and intuitive. The real powerhouse behind Magic Slate is Gemini. We integrated Gemini's advanced AI capabilities to handle both the image generation from doodles and the optical character recognition (OCR) for solving equations. Its ability to understand context and generate diverse outputs was crucial for the customizable artwork styles. For the equation solver, we trained the system to accurately interpret various mathematical notations, from basic arithmetic to more complex expressions.

Challenges we ran into

One of the primary challenges was balancing the "free-form" nature of drawing with the structured input required for AI. Getting Gemini to consistently interpret diverse drawing styles for enhancement, while also offering meaningful style customization, required significant iteration and fine-tuning of prompts and models. Another hurdle was ensuring the handwritten equation recognition was robust across different handwriting styles and complexities. There's a fine line between over-correction and maintaining the user's original input intent. Performance optimization was also key; we wanted the AI transformations to feel instantaneous, not like waiting for a complex process.

Accomplishments that we're proud of

I'm incredibly proud of the intuitive and simple UI we managed to achieve, especially considering the powerful AI working behind the scenes. Seeing a rough sketch instantly transform into a polished piece of art, or a handwritten equation solved in real-time, is truly satisfying. The integration of Gemini has been a major accomplishment, making the "magic" of AI accessible and fun. The ability to offer customizable AI generation for artwork also stands out, giving users creative control beyond just a simple enhancement. It feels like we’ve genuinely combined the best of nostalgic simplicity with cutting-edge technology.

What we learned

This project reinforced the power of combining seemingly disparate ideas to create something truly innovative. I learned a lot about prompt engineering for generative AI and the nuances of making AI-powered features feel intuitive rather than overwhelming. The importance of user feedback during development became incredibly clear, as it helped us refine both the drawing experience and the AI's interpretations. Most importantly, I learned that a strong vision, even if it starts with a simple "what if," can lead to building something genuinely magical and useful.

What's next for Magic Slate

For Magic Slate, the future is exciting! We plan to expand the library of customizable AI generation styles, offering even more artistic options. I'm also looking into incorporating more interactive elements for the math notes, perhaps allowing users to manipulate variables or visualize solutions. Integration with cloud storage for seamless saving and sharing of creations is also on the roadmap. Ultimately, I want Magic Slate to evolve into an even more powerful, yet still beautifully simple, tool for both creativity and productivity, continuing to push the boundaries of what's possible when human imagination meets intelligent AI.

Built With

• genkit
• nextjs
• typescript