About the project

Inspiration

Skincare decisions today are largely driven by guesswork. Consumers invest heavily in products without measurable insight, while dermatological consultations remain costly, slow, and subjective.

Lumière AI was inspired by the need to replace blind skincare routines with objective dermal intelligence while preserving the premium feel of a clinical consultation. The vision was to create a unified environment where voice interaction, biometric vision, and structured knowledge converge into a frictionless diagnostic workflow.

What it does

Lumière AI is a voice-first dermal diagnostics platform delivering real-time biometric analysis and personalized skincare regimens.

Key capabilities include:

Hands-free diagnostic workflows powered by real-time voice interaction
Computer vision extraction of 14+ dermal biomarkers including hydration, texture, melanin distribution, and vascular tone
Cyber-terminal visualization translating biometric signals into clinical insights
Retrieval of dermatological research tailored to the user’s biometric profile
Automated regimen mapping using structured product intelligence
Generation of secure clinical diagnostic reports

A generated report demonstrates validated biomarker scoring alongside therapeutic recommendations, showcasing the complete diagnostic pipeline implemented within the platform.

How we built it

Lumière AI is implemented as a multi-modal architecture integrating voice intelligence, biometric vision, and structured clinical knowledge.

Technical architecture highlights:

Next.js App Router enabling responsive real-time UI orchestration
Deepgram streaming voice agent providing natural command-driven diagnostics
Perfect Corp AI extracting high-dimensional dermal biomarker vectors
You.com retrieval engine delivering contextual dermatological insights via RAG workflows
Sanity CMS serving as the structured backbone mapping biomarkers to regimen protocols
Progress KendoReact rendering enterprise-grade biometric visualizations
Foxit and jsPDF enabling secure clinical document generation
CoCreate WebRTC capture supporting short temporal diagnostic video context

A custom event-driven orchestration layer synchronizes voice triggers, webcam capture, analysis pipelines, and visualization streams without blocking the UI thread.

Challenges we ran into

The primary challenge was synchronizing multiple real-time AI modalities within a unified interface. Voice commands, video capture, analysis pipelines, and visualization layers required precise state coordination to maintain a seamless user experience.

Additional challenges included:

Managing asynchronous biometric streaming without latency artifacts
Designing trust-inducing clinical visualizations rather than generic dashboards
Mapping probabilistic biometric signals to deterministic regimen recommendations
Preserving privacy while generating client-side diagnostic reports

Accomplishments that we're proud of

Achieved sub-second workflow transitions from voice command to biometric visualization
Delivered a high-fidelity clinical UI exceeding typical hackathon MVP expectations
Implemented real-time research retrieval directly linked to biometric outputs
Generated professional diagnostic reports resembling telehealth documentation
Demonstrated cohesive multi-modal AI convergence within a single user workflow

What we learned

The project reinforced that multi-modal AI effectiveness depends heavily on orchestration and state management rather than model sophistication alone. Structured content significantly improves interpretability when combined with specialized AI APIs.

We also learned that clinical design language plays a critical role in building trust, making visual semantics as important as analytical accuracy.