Nebo | Devpost

The Badge
The Landing Page
The Chat Inference featuring Nebo

Inspiration

The inspiration for Nebo came from a very specific, shared human experience: the "3 AM Crisis."

We realized that mental health struggles rarely stick to business hours. Professional therapy is often inaccessible or expensive, and leaning on friends or family late at night can feel incredibly burdensome. We saw a massive gap between "I'm just feeling a bit down" and "I need emergency help."

We wanted to build a bridge—a companion that is always awake, never judges, and doesn't just text back, but actually speaks and interacts with you. We wanted to move away from cold, static chatbots and create a warm, grounding 3D presence in your browser.

What it does

Nebo is an emotionally intelligent AI mental health companion accessible via the web. It transforms the standard chatbot experience into an immersive, audio-visual interaction.

Empathetic Conversation: It acts as a psychological support partner, using grounding techniques to help users navigate anxiety or stress.
3D Interactive Presence: Unlike flat text-bots, Nebo is represented by an interactive 3D Avatar (GLB) that provides a visual focal point for the user, making the interaction feel more "real."
Dynamic Voice Synthesis: Nebo speaks. It analyzes the conversation's context and generates hyper-realistic audio using ElevenLabs, adjusting its tone based on the user's emotional state.
Mood Tracking: It silently analyzes the sentiment of inputs to track emotional trends over time, storing them securely in the cloud.

How we built it

We built Nebo using a robust modern web stack that combines high-fidelity 3D rendering with powerful cloud AI.

Frontend (React & 3D): We built the user interface using React. To create the immersive experience, we integrated a GLB 3D model directly into the browser, allowing for a reactive visual companion rather than just a text box.
The Brain (Vertex AI): We leveraged the Vertex Gemini API on Google Cloud. We chose Vertex AI for its enterprise-grade scalability and its ability to handle complex system prompts that enforce our "Therapist Persona" safety guardrails with low latency.
The Voice (ElevenLabs): We integrated the ElevenLabs API for text-to-speech, mapping the AI's sentiment output to specific voice stability settings to sound more human.
Backend & Data (Python & Firebase): We used a Python backend to orchestrate the logic between the AI and the frontend. Firebase Cloud was used for real-time database management, securely storing user chat logs and authentication data.

To determine emotional tone, we calculate sentiment polarity $P$ where $P \in [-1, 1]$:

$$ P = \frac{\sum (w_i \cdot s_i)}{\sqrt{(\sum (w_i \cdot s_i))^2 + \alpha}} $$

If $P < -0.3$, the system triggers a "Gentle/Supportive" voice model and adjusts the GLB model's behavior.

Challenges we ran into

3D Model Optimization: Loading a detailed GLB model into a React application without killing performance was difficult. We had to optimize the mesh compression and use efficient rendering hooks to ensure the avatar loaded instantly.
Latency vs. Immersion: Chaining Vertex Gemini $\rightarrow$ ElevenLabs $\rightarrow$ React Frontend introduced potential delays. Waiting for the audio to generate broke the illusion of conversation. We solved this by implementing streaming responses in Python, allowing the audio buffer to start playing before the full file was generated.
Context Management: Keeping the conversation coherent over long sessions was tricky. We had to implement a sliding window context in Python to feed relevant history into the Gemini API without exceeding token limits.

Accomplishments that we're proud of

The "Presence" Factor: Seeing the 3D model on screen while hearing a sigh or a soft spoken response created a genuine sense of connection that text alone never achieved.
Seamless Cloud Integration: Successfully connecting Google's Vertex AI, Firebase, and ElevenLabs into a single, cohesive Python/React pipeline.
Safety Implementation: Fine-tuning the Vertex Gemini safety attributes to ensure Nebo provides help without crossing the line into giving medical advice.

What we learned

Visuals Matter: A 3D avatar (GLB) changes the psychological relationship the user has with the AI. It becomes a "being" rather than a "tool."
Cloud Architecture: We gained deep experience in orchestrating multiple cloud APIs (Google Cloud & Firebase) to work in sync.
Latency Engineering: We learned that in conversational AI, speed is just as important as intelligence.

What's next for Nebo

Real-time Lip Syncing: We plan to animate the GLB model's mouth to synchronize perfectly with the ElevenLabs audio stream.
Voice Input (STT): Implementing Whisper AI so users can speak to Nebo directly instead of typing.
RAG Memory: Adding a vector database to give Nebo long-term memory of past conversations.