DentAI Skills

Inspiration

Dental students need repeated practice to develop indirect vision and fine motor control. However, access to simulation laboratories and specialized dental phantoms can be limited by cost, schedules, and location.

We asked a simple question: could the smartphone that students already own become an accessible psychomotor training station?

What it does

DentAI Skills displays a virtual route over a horizontally mirrored live camera feed. The student follows the route using a safe blunt stylus with a bright green marker while looking only at the phone screen.

The browser detects the green reference point locally and measures:

• Precision
• Stability
• Route completion
• Time
• Overall performance

The route exists only on the phone, so no printed template or specialized hardware is required. Camera images remain on the device and are never uploaded or stored.

The public demonstration uses deterministic local feedback, preventing anonymous visitors from generating paid API requests.

How we built it

DentAI Skills is a mobile-first React application built with browser MediaDevices and Canvas APIs.

The application mirrors the camera image, renders a virtual route, and uses lightweight color segmentation to identify the green marker. A deterministic scoring layer compares the detected point with the route and calculates performance metrics.

GPT-5.6 contributed to the educational experience, feedback design, product reasoning, and structured-feedback architecture. The repository contains an optional server-side GPT-5.6 feedback pathway, disabled by default in the public deployment to prevent unauthorized API spending.

Codex was used to review and improve the implementation, verify the public demo, strengthen the safety controls around the optional API pathway, correct production metadata, validate the documentation, and run the complete build and testing process.

The human builder retained all educational, safety, scope, and product decisions.

Challenges we ran into

The first prototype used a physical route printed on paper and required a fixed camera position. Real smartphone tests showed that this setup was unnecessarily complicated and route scoring became unreliable when the paper moved.

We simplified the experience by moving the route completely into the smartphone. The camera feed, virtual route, and detected marker now share the same coordinate system.

Another challenge was preventing the public demonstration from generating uncontrolled API costs. We solved this by using local feedback by default and requiring two separate server-side settings before the optional OpenAI pathway can be activated.

Accomplishments that we're proud of

• Created a functional mobile psychomotor training experience using everyday objects.
• Implemented real-time green marker detection directly in the browser.
• Kept camera processing local to the device.
• Removed the need for printed routes or specialized hardware.
• Added measurable precision, stability, completion, and time metrics.
• Built a public demo that cannot generate paid API traffic.
• Completed a successful production build, automated test, lint check, and real-device demonstration.

What we learned

The physical experience must remain simpler than the technology behind it. Real-world testing changed the product more effectively than additional planning.

We also learned that deterministic measurement and generative educational feedback should remain separate. This makes the system easier to test, safer to deploy, and more transparent for learners.

What's next

The next steps are to add several difficulty levels, longitudinal progress tracking, expert-designed rubrics, and formal reliability testing.

We also plan to conduct an educational pilot with dental students and explore additional psychomotor exercises involving mirror control and instrument positioning.

Built With

• codex
• computer-vision
• gpt-5.6
• html5
• mediadevices-api
• openai
• react
• typescript