inPocus.com

• 

• 

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for inPocus.com

inPocus.com - AI-Powered POCUS Training Platform 💡 Inspiration During our POCUS workshops across Poland (Poznań, Kraków, and even Sicily), we witnessed a critical gap in medical education. Medical students, paramedics, and doctors struggled with the steep learning curve of ultrasound technology. Traditional training requires expensive equipment, constant expert supervision, and hundreds of hours of practice. We were inspired by a simple observation: while students quickly grasped the theory, the practical application remained challenging due to:

Limited hands-on time with expensive equipment Difficulty in maintaining consistent probe pressure and angle Lack of immediate feedback on positioning accuracy The physical strain of holding probes steady during learning

When OpenAI released the GPT-OSS models, we saw an opportunity to democratize POCUS education by combining open-source AI with affordable robotics to create an intelligent training assistant. 🎯 What it does inPocus.com is an educational platform that transforms POCUS training through AI-powered robotic assistance and voice control. Our system acts as an intelligent teaching assistant that helps medical professionals learn ultrasound techniques more effectively. Core Features: 🤖 Robotic Training Assistant

SO-100 arm holds and positions the ultrasound probe with precision Maintains consistent pressure and angle for optimal imaging Allows learners to focus on image interpretation rather than manual technique

🎤 Voice-Guided Learning

Natural language commands: "Show me cardiac four-chamber view" Real-time feedback: "Rotate probe 15 degrees clockwise for better visualization" Hands-free operation allows note-taking and observation

📚 Workshop Integration

Multi-station setup for group training sessions Supports various POCUS devices (GE, Butterfly, Echo Nous, Chinese models) Comparative learning across different ultrasound systems Perfect for the workshop format we've tested in Kraków and Poznań

🎓 Educational Protocols

Pre-programmed scanning sequences for standard examinations VExUS protocol training (heart, lungs, abdomen, pelvis) Step-by-step guided tutorials with voice narration Mistake detection and correction suggestions

💾 Learning Analytics

Records scanning sessions for review Tracks progress across different protocols Identifies areas needing improvement Generates personalized practice recommendations

🔧 How we built it Technical Architecture:

AI Brain - GPT-OSS 20B Model

Fine-tuned on medical terminology and POCUS protocols Runs locally using LLaMA Studio for privacy Processes voice commands and generates robot instructions Provides educational feedback in natural language

MCP (Model Context Protocol) Integration

Bridges the gap between AI understanding and robot execution Maintains session state for continuous learning Handles safety constraints and movement boundaries Enables real-time adjustment based on ultrasound image feedback

Robotic Platform

SO-100 arm with custom mounting for POCUS devices 6 degrees of freedom for complete positioning control

python # Core components

• Voice Recognition: Whisper API for medical terminology
• Command Processing: GPT-OSS for intent understanding
• Initilize and sleep robot using MCP server calls
• Motion Planning: Custom kinematics library
  
• Workshop Management: Django backend for multi-station control

Workshop Infrastructure

RTX 3090 ti / Raspberry pi / Mac m1 units for distributed control Local network for multi-station synchronization Web dashboard for instructor oversight Cloud backup for session recordings (optional)

Development Timeline:

Robot-ultrasound mechanical integration GPT-OSS training on POCUS protocols MCP protocol implementation Voice control and safety systems Workshop management system

🚧 Challenges we ran into

Medical Vocabulary Recognition Solution: Created custom dictionary of POCUS and SO-ARM 100 MCP Tools calls. Multi-Device Compatibility Challenge: Different ultrasound devices have varying APIs and interfaces

Gel Management System (showcase - MCP server in practice)

Challenge: Traditional gel application was messy in workshop settings Solution: Developed spray-on gel that evaporates cleanly after training sessions

Built With

• gpt20b