Magic Chalk

Tired of writing tangled and twisted equations on computer? Well, here is our solution, Magic Chalk, our virtual whiteboard solver!

Comment 1

Inspiration 🌟

We saw those cool YouTube math tutors writing on transparent walls and thought, "Why not make it simpler?" So, we ditched the fancy gear and decided to use what we've got – our fingers. Perfect for teachers or tutors who want an easy, accessible, and tech-savvy way to teach.

What It Does 🖐️🎨

Fire up the camera, and our AI is on it, tracking your hand movements. It's all about the fingers. One finger out? Choose your tool (just wait for the circle to complete). This way, you can move around without altering the canvas. Whip out two fingers, and you're all set to draw or erase. We've even got a 'clear canvas' button for a fresh start. Wrote something down? Our custom-trained AI kicks in to solve those equations. And for those important notes during lectures, just hit the bookmark 📚 to save a snapshot and the solve button to display a solution to the equation.

How We Built It 🔧🐍

The frontend was entirely built using streamlit. We used MediaPipe, OpenCV, and numpy to build the drawing system that tracks hand gestures. We implemented the WolframAlpha API to solve the equations. And tensorflow and scikit learn to train our ML model which uses a Sequential model with a dataset found on kaggle.

Challenges We Faced 🚧

  1. Ditching Latex: We initially wanted to use Latex conversion, but it was a roller coaster. The API wasn't reliable and almost all options were costly, so we switched gears. We focused on creating our own datasets for numbers and basic operations. Simple math, but way more accurate 🎯.
  2. Camera Overkill: Our first camera choice was too powerful which resulted in a lot of fixing time. So, we switched to a basic webcam, and suddenly, everything clicked 📷.
  3. Learning Curve with Streamlit: Usually, we're Flask and React folks, but Streamlit caught our eye. It was a bit of a puzzle at first (especially the sidebar thing), but we managed 💻.

What We're Proud Of 🌈

Our web app's design is clean, user-friendly, and fairly fluid. We ran it by our friends, and they got it right away. From the toolbar to the theme, we crafted every bit ourselves.

What We Learned 🤓

Streamlit, MediaPipe, and reinforce our knowledge of ML and OpenCV

What's Next for Magic Chalk 🚀

We built this with virtual math teaching in mind. Next stop? Integrating Magic Chalk into online platforms like Zoom, Google Hangout, Teams, and maybe for lecture recordings. We are also planning to make our AI recognize a larger variety of mathematical symbols such as integrals, sums, etc.

Built With

Share this project:

Updates