MindScoper

Inspiration

Therapists often struggle to track complex patterns across sessions. MindScoper automates the extraction of psychological entities, behavioral patterns, and relationships from therapy conversations — presenting them as an interactive knowledge graph.

What it does

Records or accepts uploaded therapy session audio Transcribes speech using OpenAI Whisper Uses Google Gemini 2.5 to extract symptoms, emotions, triggers, behaviors, and outcomes Maps relationships between entities as an interactive D3.js knowledge graph Computes a 1–10 wellness score with clinical severity levels Detects temporal patterns (episodic, trait-based, reactive) Suggests possible condition hints and follow-up recommendations How we built it Next.js 16 with App Router and TypeScript OpenAI Whisper API for accurate speech-to-text Google Gemini Gemini for structured clinical JSON extraction via prompt engineering D3.js force-directed graph for entity/relationship visualization TailwindCSS v4 for styling localStorage for session persistence Challenges we ran into Whisper API rejects files without proper extensions — had to reconstruct File objects with MIME-to-extension mapping Gemini sometimes wraps JSON in markdown fences — built a resilient JSON extraction parser Balancing prompt specificity to get consistent structured output from Gemini What we learned Prompt engineering for structured clinical data extraction D3.js force-directed graph with React integration Handling browser MediaRecorder audio formats across the AI pipeline Professional medical report using the data ready to export

How we built it

Next.js 16 with App Router and TypeScript OpenAI Whisper API for accurate speech-to-text latest Google Gemini model for structured clinical JSON extraction via prompt engineering D3.js force-directed graph for entity/relationship visualization TailwindCSS v4 for styling localStorage for session persistence

Challenges we ran into

Whisper API rejects files without proper extensions — had to reconstruct File objects with MIME-to-extension mapping Gemini sometimes wraps JSON in markdown fences — built a resilient JSON extraction parser Balancing prompt specificity to get consistent structured output from Gemini

What we learned

Prompt engineering for structured clinical data extraction D3.js force-directed graph with React integration Handling browser MediaRecorder audio formats across the AI pipeline

This is for the open track for the hackathon

Built With

• css
• d3.js
• gemini
• google
• lucide
• next.js
• openaiwhisper
• react
• tailwind
• typescript