MaMo

An application dedicated to rewarding kids for doing math at home with cool characters (MaMo, short for math monsters) that can be used in games of the app.

Comment

Inspiration

We wanted to find a way to make learning a better experience, by creating this reward system for correctly solving problems.

What it does

Our working page takes as input an image of a student's working which is then compared to the question being presented and is converted to LaTeX format with MathPix for better request processing. The request is processed with the gpt-4o model, where a step by step assessment of what went wrong is displayed and then depending on whether the submission is correct or incorrect, a MaMo can be generated within a gacha system for the user to keep.

How we built it

We started by integrating the GPT api call with Gradio (used to host and display our front end and model output), and then we used stable diffusion from AWS Bedrock to randomly generate the MaMo's, and the MaMo is only awarded to the user if they get the question correct.

Challenges we ran into

Converting data types was a bit confusing at first (Darin) when Gradio automatically preprocesses image inputs into numpy arrays, which had to be converted into the proper base64 format for GPT and MathPix to process, and the prompt we used did not always display the correct LaTeX output. We also ended up with GPT hallucinating at the start and then correcting itself afterwards, and to fix that, we thought of putting in another request to process the previous output into a better format.

Creating the MaMo's with AWS Bedrock was also a bit confusing at first when done through google collab because of the predefined processing required (Duc)

Accomplishments that we're proud of

We are proud that we got the model to work within a day, and that the images can be processed pretty well.

What we learned

We learned that designing and using a precise prompt setup is really important.

What's next for MaMo

Next up is to create a proper and working full-stack application for this, with user login ability, and then allowing the ability for users to use their MaMo's in games. We thought of possibly being able to convert the MaMo image into a 2d pixelated, programmable character (with different frames) using a separate model that we would have to create, and then using that character in various games with similar style to Mario party.

Built With

Share this project:

Updates