Inspiration

We were inspired by a problem we often face when trying to learn something new. To work through a difficult math, physics, chemistry, or foreign language problem, we might need ChatGPT beside us and a separate whiteboard to show our work. We began imagining what it would be like if the whiteboard itself could listen, see the work through a camera, and respond using a marker. However, we did not want Smart Slate to provide instant answers, because real learning takes time, effort, and mistakes. The student should still do ninety-nine percent of the work. When they genuinely feel stuck, they can ask Smart Slate for a small hint that guides them toward the next step without solving the problem for them. The amount of assistance available, including whether a complete explanation can eventually be shown, could be controlled through a setting.

What it does

Smart Slate brings a conversational AI study partner directly to a physical whiteboard. A camera mounted above the board watches the student’s work, while real-time voice allows the student to talk naturally with the system. When the student becomes confused and asks for help, Smart Slate decides whether a spoken explanation or a visual hint would be more useful. Its XZ marker arm can then draw a line, diagram, equation, or small clue directly on the whiteboard without completing the problem for the student. A student could use Smart Slate to work through many practice problems while preparing for a test. The problems might come from a teacher, worksheet, or textbook, but Smart Slate can also create additional practice questions to solidify understanding for a student. Over time, it can recognize which types of problems the student finds difficult and provide more practice in those areas. It is essentially an AI tutor with a voice, a camera, a whiteboard, and a marker—while still leaving the actual learning and problem-solving to the student.

Business Aspect

For engineering and finance teams, Smart Slate could be scaled into a much larger, boardroom-sized whiteboard that acts as a decision-making copilot for the entire room. Engineering teams could use it to organize design options, clean up diagrams, add measurements, perform calculations,and compare concepts side by side. Finance and economics teams could use it to display budgets, cost-benefit analyses, forecasts, and base, upside, and downside scenarios that update as assumptions change. Because the board would be large enough for everyone in the meeting to see and contribute, it would create one shared view of the decision instead of separating information across individual laptops. Afterward, Smart Slate could preserve the alternatives, calculations, assumptions, risks, final decision, and assigned action items so the team’s reasoning is not lost when the physical board is erased.

Challenges we ran into

One of our biggest team challenges was the six-hour time difference between Wisconsin and the United Kingdom. We could not speak whenever an important question came up, and our calls often had to end early because it was already late for part of our team. We usually waited until the next call instead of resolving questions through messages or shared sketches, which slowed our progress significantly. This also contributed to a major misunderstanding within our team: we had two different ideas of Smart Slate’s design. One idea placed the marker on the XZ mechanism, while the other placed the camera on the mechanism and treated Smart Slate mainly as a voice assistant. We did not discover this difference until late in the project, putting our team into a serious time crunch. Once we agreed on a fixed overhead camera and an XZ mechanism that moves the marker, we still had to understand how the motion system would work and translate it into a practical CAD design. Learning the mechanism, developing the CAD, and working with the limited time remaining made it difficult for us to create the complete prototype we originally imagined. Our most realistic physical demonstration is now a moving XZ table carrying a marker. Although it would not yet include every Smart Slate feature, it would prove the central mechanical idea and give our team a foundation for adding the camera, voice system, and AI guidance later.

How we built it

We developed Smart Slate’s core software as a modular, cross-language system connecting visual input, artificial intelligence, and spoken output. Our architecture combines C# and Python so that each language handles the part of the system for which it is best suited. The C# application serves as the primary control layer, managing the program’s inputs and outputs while coordinating communication with the Python-based AI pipeline. When Smart Slate captures an image of the user’s work, the C# layer passes it into the Python modules for processing. Python resizes the image to reduce the amount of data that must be analyzed, prepares it for the model, and sends it to Qwen3. After the model interprets the visual input and generates a response, the Python pipeline converts the returned text into an audio file. Both the written response and generated audio are then transferred back to C#, where they can be displayed and played for the user. By separating the interface and control logic from the AI and media-processing tasks, we created a flexible foundation in which individual components can be improved or replaced without rebuilding the entire program. This prototype demonstrates Smart Slate’s essential digital interaction: receiving visual information from the whiteboard, processing it through an AI model, and returning guidance in both written and spoken forms. The XZ marker mechanism is the next component to be connected to this software pipeline.