MAITRI

Inspiration

Space exploration pushes the boundaries of human endurance. During long-duration missions—often lasting up to 18 months—astronauts face intense confinement, sensory monotony, circadian disruption, and deep isolation. Conventional Earth-based support systems rely on ground communication, but deep-space travel introduces severe signal latency and unpredictable communication blackouts, making immediate psychological intervention impossible. We realized that current onboard monitoring systems are largely fragmented and reactive, relying on manual symptom self-reporting rather than continuous, non-intrusive assessment. This inspired us to build MAITRI—an autonomous, edge-deployed AI companion designed to safeguard the psychological and physical well-being of astronauts in real time, completely independent of ground mission control.

What it does

MAITRI acts as a 24/7 empathetic companion and proactive wellness monitor directly inside the spacecraft habitat: Multimodal Wellness Tracking: Continuously monitors facial micro-expressions (via MediaPipe/OpenCV), vocal acoustics (prosody, pitch, MFCCs), and text sentiment during everyday interactions. Multimodal Fusion Engine: Merges audio, visual, and textual signals through a unified classification pipeline to detect early signs of stress, fatigue, anxiety, and burnout. Autonomous Micro-Interventions: Provides low-cognitive-load conversational support, guided breathing sessions, and mindfulness cues dynamically tailored to the crew member's emotional state. Emergency Escalation Protocol: Operates on a prioritized triaging model—normal states trigger supportive dialogue, moderate stress activates guided relaxation, and critical distress logs high-priority flags for ground synchronization when communication windows open. Astronaut & Ground Dashboards: Provides crew members with daily wellness timelines and heatmaps, offering actionable self-regulation insights while safeguarding user privacy.

How we built it

We designed MAITRI around an offline-first, low-power edge architecture optimized for deep-space hardware constraints: Frontend: Built with Next.js and Tailwind CSS to deliver a sleek, dark-mode cockpit UI with minimal cognitive strain for crew members. Backend & Real-Time Comms: Developed using Express.js, WebSockets, and an encrypted SQLite database for secure local logging and low-latency interaction loops. Vision & Audio Pipelines: Leveraged MediaPipe and OpenCV for facial landmark tracking and Action Unit analysis, combined with spectral subtraction for spacecraft ambient noise filtering and speech-to-text processing. AI & NLP: Deployed DistilBERT and transformer-based fusion layers via Hugging Face to contextualize emotional nuance and manage dialogue turns. Edge Optimization: Converted models using ONNX Runtime and TensorRT quantization to enable efficient execution on edge compute units (e.g., NVIDIA Jetson / ARM architectures) without heavy power draw.

Challenges we ran into

Spacecraft Acoustic Noise: Life-support systems, fans, and avionics generate continuous ambient hum. We had to implement spectral subtraction and specialized audio preprocessing before feeding signals into our acoustic feature extractors. Edge Compute Constraints: Running multi-stream video analysis, audio processing, and transformer inference simultaneously posed a heavy compute load. We addressed this by model pruning, INT8 quantization, and utilizing ONNX/TensorRT runtimes. Data Scarcity in Microgravity Environments: Sourcing real astronaut psychological datasets is inherently difficult. We utilized research datasets like IEMOCAP, RAVDESS, and WESAD, applying transfer learning and space-analog stress simulations to tune our emotion classification boundaries. Avoiding Alert Fatigue: Astronauts reject systems that misinterpret normal workload strain as emergencies. We introduced confidence calibration scoring to ensure the system only intervenes or flags ground control when multiple modalities concurrently indicate critical fatigue or distress.

Accomplishments that we're proud of

Working Functional Prototype: Built and deployed a working end-to-end prototype with real-time UI, live emotion tracking, and interactive conversational sessions. True Multimodal Fusion: Successfully synchronized and integrated facial micro-expression tracking, speech acoustic analysis, and NLP sentiment detection into a cohesive emotion classification pipeline. Edge-Optimized Offline Architecture: Optimized inference pipelines using ONNX and TensorRT quantization, proving that complex multimodal AI can run locally on constrained edge hardware during deep-space communication blackouts. Robust Noise Resilience: Implemented preprocessing filters that isolate vocal biometrics from continuous background spacecraft noise and equipment hum. Non-Intrusive Astronaut Care: Designed a tiered intervention and confidence-scoring system that prevents false alarms while proactively delivering micro-interventions to safeguard mental resilience.

What we learned

Building MAITRI taught us how to bridge affective computing with mission-critical systems engineering. We learned the critical value of multimodal fusion—how voice tone and facial kinematics validate each other when one stream is noisy or occluded. Crucially, we discovered that designing AI for isolated environments requires restraint: an effective space companion must be non-intrusive, privacy-preserving, and built to empower human autonomy rather than replace it.

What's next for MAITRI

Integrating non-invasive wearable telemetry (HRV, galvanic skin response, sleep stages) for deeper physiological correlation.
Expanding federated, on-device incremental learning so MAITRI adapts to individual astronaut baseline behaviors over months in deep space.
Testing under extended mission simulations in Earth-based analog space habitats.

Built With

• bert
• distilbert
• express.js
• hugging-face
• javascript
• mediapipe
• next.js
• onnx
• opencv
• python
• sqlite
• tailwind-css
• tensorrt
• vercel
• websockets