Inspiration
User testing today creates transcripts, not clarity.
Teams sit through recordings, extract notes manually, and still miss the most important signal: where real friction actually occurred.
We noticed three consistent problems:
1. Emotional frustration gets lost in text summaries
2. Visual context is separated from verbal feedback
3. Recommendations lack competitive grounding
MogUX was built to turn passive recordings into an autonomous qualitative analyst.
What it does
MogUX transforms user testing videos into an evidence-backed, self-improving UX playbook.
It operates through a three-layer intelligence loop:
- Detect Friction (Modulate)
We analyze the user’s voice for measurable emotional signals such as frustration, hesitation, and stress spikes. These serve as objective indicators that something in the UI failed.
- Correlate Context (Reka)
When a frustration spike is detected, MogUX analyzes screenshots from that moment to determine:
What the user was trying to do
What UI elements were visible
What likely caused confusion
This produces timestamped friction logs grounded in visual evidence.
- Benchmark & Recommend (Yutori)
For each friction event, MogUX autonomously researches:
-> Industry best practices
-> Competitor UX patterns
-> Proven layout or interaction improvements
It then generates structured, benchmark-backed recommendations.
Over time, MogUX merges repeated friction points into an evolving UX playbook, increasing the priority of recurring issues and refining insights across sessions.
How we built it
MogUX is built as a modular agentic pipeline:
Modulate API → Detects emotional signals from audio
Reka Multimodal Model → Analyzes screenshots + transcript context
Yutori Research & Browsing APIs → Performs competitive benchmarking
LLM Orchestration Layer → Structures friction insights into actionable bullets
Delta Playbook Store → Performs incremental updates to maintain a living knowledge base
Architecture Flow
User Testing Video → Emotion Spike Detection (Modulate) → Visual Context Analysis (Reka) → Benchmark Research (Yutori) → Incremental Playbook Update
We intentionally structured the output as bullet-level friction logs instead of long summaries to preserve qualitative detail.
Challenges we ran into
Multimodal Alignment
Correlating emotional timestamps with screen state required designing time-window batching rather than relying on exact millisecond precision.
Avoiding Generic Summaries
Large language models tend to over-generalize UX feedback. We solved this by enforcing structured, localized friction bullets tied to evidence.
Scope Control
Automatically modifying UI code would introduce instability. We instead focused on high-confidence qualitative recommendations for human designers.
Designing Real “Self-Improvement”
We avoided fake learning claims by implementing incremental delta updates that merge recurring friction into a prioritized playbook.
Accomplishments that we're proud of
Successfully chained Modulate → Reka → Yutori into a coherent autonomous loop
Built multimodal friction detection grounded in emotional and visual signals
Created an evolving UX playbook that strengthens as more sessions are analyzed
Designed a structured output format that designers can immediately act on Most importantly, we demonstrated that user testing can move from passive observation to active intelligence.
What we learned
Emotional signals are a powerful and underutilized UX metric
Multimodal reasoning dramatically improves qualitative insight quality
Autonomous research agents increase credibility of recommendations
Self-improving systems do not require retraining — they require structured memory and incremental updates
What's next for MogUX
Real-time live session monitoring instead of post-session analysis
Cross-session clustering to detect systemic product weaknesses
Team dashboards with severity scoring and prioritization
Direct integration with product management tools (Linear, Jira)
A feedback loop where designers validate recommendations and refine the model’s prioritization
Our long-term vision is a world where user testing sessions automatically strengthen product intelligence without adding manual overhead.
Log in or sign up for Devpost to join the conversation.