Sigh-cus

Meet Gerald—the dramatic desk plant that turns posture and hydration reminders into personality.
A compact companion with quick access to Gerald’s commentary, voice controls, and water breaks.

Inspiration

Long desk sessions make it easy to forget simple things: sitting comfortably, drinking water, blinking, and taking a break. Another serious reminder can feel like one more notification to ignore. Sigh-cus gives those reminders a personality: Gerald, a dramatic desk plant whose disappointment is hard to miss.

What it does

Gerald reacts to your habits through animated expressions, sarcastic comments, and an optional spoken voice. First, open “Manage My Disappointment” to choose which habits Gerald watches and adjust the reminder timings. Then start the camera and calibrate your upright posture. Gerald keeps you company while you work. He watches for signs of slouching, prolonged staring, leaning too close, frowning, yawning, a hand near the mouth, downward gaze associated with phone use, and leaving the desk. These are landmark-based cues, not medical assessments or direct phone recognition. Hydration reminders encourage water breaks, with a manual “I just drank water” button when automatic detection misses a sip. Gerald’s leaves droop and his pot squints, rolls its eyes, or shakes in disappointment. Once he confirms a habit has stopped, he responds with a backhanded compliment. A mood trail shows how the session unfolds, and quiet mode pauses the commentary. A small corner icon opens Gerald’s latest message and quick controls without a separate floating window. Voice is off by default; when enabled, ElevenLabs gives Gerald a dry British delivery, with browser speech as a fallback. Camera-free demo controls are available on local and Lovable preview hosts.

How we built it

The interface uses React, TypeScript, TanStack Start, Vite, and Tailwind CSS. MediaPipe processes face and hand landmarks in the browser. A calibrated rule engine turns those readings into habit events, with timing thresholds and cooldowns to prevent repeated reminders. Gemini, accessed through a server-side AI gateway, writes contextual comments and provides selected image-based posture and drink-vessel checks. Those checks use small snapshots rather than streaming video to the model. ElevenLabs generates speech through server functions, keeping API credentials out of client code. An optional Backboard.io integration records and recalls habit events for later context. The plant illustration uses SVG and CSS animations, including reduced-motion support.

Challenges we ran into

Brief blinks can happen between slower timer ticks, so blink observations need to be retained in timestamped readings. Phone-related posture overlaps with slouching, which makes the signals difficult to separate. Sip detection also needs timing and cooldown rules to avoid counting ordinary movement as drinking. Hand tracking can briefly lose fingertips, so we added short latches to stabilize hand-at-mouth and hand-below-face signals. We also worked through voice configuration and graceful fallbacks, balanced expressive animations with readable controls, and kept simulated actions available for development without exposing them on public hosts.

Accomplishments that we’re proud of

We brought sensing, personality, animation, and voice together into a working desk companion. Gerald responds when a habit changes, rather than only issuing reminders. The interface is responsive, the corner widget stays compact, and all 26 automated tests pass for the project’s rules and behavior.

What we learned

Camera signals need calibration, temporal context, and conservative interpretation. Small interaction details—clear voice controls, adjustable triggers, timely feedback, and quiet mode—matter as much as the detection logic. A memorable character can make a simple reminder more engaging.

What’s next for Sigh-cus

We want to validate every habit with more real-camera sessions across different lighting and camera positions, reduce false positives, and improve accessibility and personalization. We also want to expand the existing optional Backboard habit memory into richer history and personalization. The current interface keeps a limited mood history saved locally.