Kinship

Inspiration

Our team was inspired by a simple but pressing reality: as technology advances, the elderly are often left behind by complex, touch-heavy interfaces. We wanted to build Kinship to prove that the most advanced AI should also be the most accessible. We envisioned a world where a senior doesn’t have to learn a new UI, but can simply speak to their device as they would to a friend, turning natural conversation into organized digital actions.

What it does

Building Kinship was a masterclass in Collaborative Full-Stack AI Orchestration. Through this project, we learned how to: Handle Binary Streams: Managing raw Blob data from the MediaRecorder API and ensuring it reaches the backend without losing its "identity" (MIME types). Model Handshaking: Learning the strict requirements of JSON-mode in LLMs and how a single parameter can make or break a pipeline. Asynchronous UX: Managing the "state" of a conversation so the user feels heard in real-time, even while complex processing happens in the background. Technical Resilience: We learned that the "Inspect" tool is a developer's best friend when APIs return cryptic status codes.

How we built it

We divided the labor to ensure a seamless integration between the "eyes, ears, and brain" of Kinship: Frontend: Built with Next.js 15 (Turbopack) and Tailwind CSS, focusing on high-visibility, senior-friendly design patterns. Backend: A Node.js/Express server acting as the central hub for API routing and file handling. AI Pipeline: * Groq Whisper (whisper-large-v3): For near-instant, high-accuracy speech transcription. Groq Llama 3.1 (llama-3.1-8b-instant): To analyze intent and output structured JSON data. Database: Firebase Firestore to provide a persistent "memory" of reminders and interactions. Speech Synthesis: Web Speech API to provide the warm, audible "voice" of Kinship.

Challenges we ran into

The biggest hurdle was a persistent AxiosError: Network Error followed by a 400 BadRequest from the AI endpoint. For a moment, our frontend was talking, but the AI wasn't listening. The "Inspect" Breakthrough: We spent hours acting as "code detectives." By diving into the Browser Inspect Tool and analyzing the Network Tab and console.error logs, we traced the failure to two specific bugs: Metadata Erasure: The backend was stripping file extensions from the Multer upload, causing the transcription engine to reject the file as an invalid type. Model Sensitivity: We identified that our chosen model, llama-3.1-8b-instant, required a very precise response_format configuration to handle our JSON intent-parsing. Solving these "handshake" issues through the console was our team's proudest moment—it was the moment Kinship finally "woke up" and replied to us.

Accomplishments that we're proud of

The "Inspect" Breakthrough: We are incredibly proud of our persistence in debugging. Facing a persistent AxiosError: Network Error and a cryptic 400 BadRequest from the AI endpoint, we didn't give up. By diving deep into the Browser Inspect Tool and analyzing console.error logs, we traced the failure to a specific model configuration issue with llama-3.1-8b-instant. Seamless Voice-to-Action Pipeline: We successfully built a bridge between raw human speech and structured digital data. Seeing the AI correctly categorize a senior's spoken sentence into a JSON-based "Reminder" or "Answer" was a massive win for us. Near-Zero Latency: By utilizing Groq’s LPU inference engine, we achieved near-instant transcription and response generation, which is critical for making technology feel "natural" to elderly users.

What we learned

Full-Stack AI Orchestration: We mastered the lifecycle of a voice request—from capturing a Blob in the browser to streaming it through a Node.js/Express server using Multer, and finally piping it into specialized LLM models. The Importance of Metadata: We learned the hard way that APIs like Groq are sensitive to file headers and extensions. Ensuring that a file maintains its .webm identity across the stack is just as important as the code that processes it. Strict JSON Schema Compliance: We learned how to handle "Model Handshaking" by configuring response_format and system prompts to ensure the LLM always returns valid, machine-readable JSON. Collaborative Debugging: This project taught us how to divide and conquer technical debt under pressure, moving between the frontend console and backend logs to find a single point of failure.

What's next for Kinship

Medical API Integration: We plan to sync Kinship with pharmacy and healthcare APIs to provide real-time medication alerts and health check-in reminders. Multimodal Vision Support: Using MLLMs (Multimodal Large Language Models), we want to allow seniors to hold up a pill bottle or a letter to the camera and ask Kinship, "What does this say?" or "When do I take this?" Emotional Intelligence Tuning: We aim to further refine the system prompt to detect emotional distress or confusion in a user’s voice, allowing the AI to offer comfort or alert family members if necessary. Hardware Integration: Moving beyond the browser into a dedicated, "always-on" smart speaker device designed specifically for bedside use.

Built With

• axios
• dotenv
• express.js
• groq
• multer
• next.js