Compass — A Screenless AI Tutor

Inspiration

AI and new technologies are transforming education. They give students access to more information, personalized learning experiences, and increasingly powerful tools to help them understand difficult concepts.

But there is a paradox: we are using the same screens for learning, entertainment, and work.

As a result, studying often means being surrounded by notifications, social media, and other digital distractions. For many students, staying focused in front of a screen is becoming increasingly difficult.

We believe AI can enable a different approach.

With the latest multimodal AI models, we can build learning experiences that don't require a screen at all. What if students could learn through natural conversation, while keeping their attention on the physical world around them?

That's the idea behind Compass.

What it does

Compass is a screenless AI tutor designed to help students learn, stay focused, and build confidence, without the distractions of a screen.

Using conversational AI and vision, Compass can understand what a student is working on and provide real-time guidance, explanations, and encouragement.

But Compass isn't designed to simply give students the answer.

Instead, it acts as a tutor: it asks questions, provides hints, adapts its explanations, and guides students toward the solution themselves. The goal is to develop understanding and independence, rather than just completing an exercise.

Compass also includes a second AI agent dedicated to understanding the student's learning journey.

By analyzing progress, performance, and learning patterns over time, this agent can identify areas for improvement and provide personalized recommendations.

Together, these two agents create an adaptive, conversational learning environment that allows students to learn at their own pace — while staying away from the screen.

How we built it

We started by defining the product and user experience in Notion, focusing first on what a screenless learning experience should actually feel like.

For our proof of concept, we built the tutoring agent with FastAPI, based on Google's Gemini Live API sample.

We then developed a second agent dedicated to learning analytics and dashboarding, powered by Gemini 3.5 Flash-Lite, to analyze student progress and generate personalized insights.

Finally, we built the frontend with Next.js and connected the different components into a complete working prototype.

The entire project was developed and deployed during the hackathon, giving us the opportunity to explore Google Cloud and several technologies that were completely new to us.

Challenges we ran into

One of our biggest challenges was finding the right personality for the tutoring agent.

A good tutor shouldn't be too passive, but it shouldn't take control away from the student either. We spent a lot of time debugging and refining the agent's behavior to make sure it would guide students without giving away the answers.

We also had to learn a completely new development and deployment environment during the hackathon.

It was our first experience working with Google Cloud, and we were pleasantly surprised by how accessible and straightforward it was to go from a prototype to a deployed application.

Accomplishments we're proud of

The most meaningful accomplishment isn't just that we built a working prototype.

We already used Compass to help a middle-school student prepare for an exam.

Seeing the system move from an idea to something that could actually support a student in a real learning situation was a major milestone for us.

What we learned

This project pushed us far outside our comfort zone.

We learned how to:

Build and deploy applications on Google Cloud
Develop a modern frontend with Next.js
Integrate real-time multimodal AI into an application
Design and refine an AI tutor's personality and behavior
Turn an idea into a working product under a tight hackathon deadline

Most importantly, we learned that building with AI is not only about choosing the right model, it's about designing the right interaction between the model and the user.

What's next for Compass

If we have the opportunity to continue developing Compass, we want to make it much more versatile.

Beyond math

Our current prototype focuses primarily on mathematics. Next, we want to expand Compass to support a wider range of subjects, from science to languages and humanities.

A physical writing companion

We also want to introduce a robotic writing assistant that could physically interact with the student's workspace.

Using its camera feed, the robot could understand where objects are on the desk and communicate visually with the student — pointing, writing, or interacting with physical materials.

The ultimate goal is to push the concept even further:

A truly multimodal AI tutor that can teach, see, talk, and interact with the physical world, without putting another screen between the student and their learning.