VFam

Inspiration

Family technology is fragmented: learning apps, chats, parental controls, and smart assistants rarely work together. At the same time, introducing AI into a home raises an important question: how can families benefit from intelligent assistance without giving up control of their children's data?

Vulture was created to explore a different approach—one where useful AI, family coordination, and parental authority work together, while sensitive voice interactions can remain inside the home.

What it does

Vulture brings family management, children's learning, secure communication, and the FamBuddy voice companion into one connected platform.

Parents can organize family members, assign chores, manage homework, review AI-generated learning material, and follow a child's progress. Children receive a focused experience for practice and everyday tasks. Families also have private chat and audio/video calling.

FamBuddy adds a voice-enabled home companion. It can recognize speech offline, respond with synthesized speech, handle basic requests locally, and use on-device AI on supported hardware. Its defining feature is a two-stage parental approval system: the exact question must be approved before AI processing, and the exact answer must be approved before FamBuddy speaks it. If local AI cannot answer a child's question, the request waits for a parent instead of automatically going to the cloud.

How it works

The platform is divided into three components with clear responsibilities:

• Vulture is the family control plane. It manages accounts, family permissions, learning data, tasks, encrypted-chat relay, call signaling, and remote FamBuddy coordination.
• VFAM is the Tauri-based desktop and mobile application. It provides the parent and child interfaces and performs sensitive networking, storage, and cryptographic operations in native Rust.
• vBuddy is the local FamBuddy gateway and device runtime. It handles voice recognition, local AI, encrypted device state, parental approvals, media, and hardware integration.

VFAM communicates with Vulture through authenticated native APIs. It connects to a home FamBuddy through a pinned TLS certificate. Optional remote FamBuddy access uses encrypted commands, while the local device remains responsible for final approval and playback decisions.

Key features

• Family onboarding, role-based permissions, chores, and shared dashboards
• Homework creation, guardian-reviewed AI drafts, practice, and progress tracking
• Offline speech recognition and text-to-speech on FamBuddy
• Optional on-device AI for children's questions
• Separate approval of every question and answer before playback
• End-to-end encrypted family chat using Messaging Layer Security
• WebRTC audio/video calls with short-lived TURN credentials
• Encrypted local storage, certificate pinning, and operating-system credential stores
• Local-first device safety with mute, revocation, network isolation, and signed updates

How it was built

The backend and device services are written primarily in Rust, using Tokio, Axum, and tonic gRPC. PostgreSQL stores authoritative family data, while SQLCipher protects local application and device databases. Private files use S3-compatible object storage.

VFAM combines Tauri 2, Svelte 5, SvelteKit, TypeScript, and Tailwind CSS. Private chat uses OpenMLS with modern authenticated encryption. Real-time updates use Socket.IO, events use Nisshi's Kafka-compatible API, and calls use WebRTC.

On FamBuddy, Vosk provides offline speech recognition, Piper provides speech synthesis, and an optional pinned llama.cpp model supplies local AI. Linux deployments use ALSA, FFmpeg, systemd, Unix sockets, and nftables for media and process isolation. Docker Compose and Traefik support server deployment.

Challenges

The hardest challenge was making AI useful without making it authoritative. Generated homework needs guardian review, and FamBuddy needs approval of both its input and output. This required explicit state machines and security boundaries rather than relying on UI restrictions alone.

Another challenge was connecting cloud, native-app, and home-device architectures without mixing their responsibilities. Strict service/repository layering and typed protocol contracts keep each component focused and testable.

Accomplishments

The project establishes one coherent family platform across three focused codebases. It combines native family apps, a privacy-conscious backend, and a local voice/device runtime with strong approval boundaries. The design supports offline home interaction, encrypted communication, parental oversight, and future hardware expansion without treating cloud access as a requirement for every action.

What we learned

Privacy is an architecture decision, not a checkbox. Transport encryption, local processing, encrypted storage, parental approval, and clear failure behavior must work together. It is equally important to distinguish implemented capabilities from future platform goals and to avoid presenting AI confidence as educational or safety authority.

What's next

Next steps include deeper physical-device validation, completing native Android/iOS FamBuddy hosts, expanding learning evaluation, and improving remote-relay scalability. The long-term goal remains the same: helpful family AI that works around parental authority and privacy from the beginning.

Built With

Rust, Tauri, Svelte, SvelteKit, TypeScript, Tailwind CSS, Tokio, Axum, tonic gRPC, PostgreSQL, SQLCipher, OpenMLS, Socket.IO, Nisshi, WebRTC, Vosk, Piper, llama.cpp, Docker, and Traefik.

Built With

• axum
• docker
• llama.cpp
• nisshi
• openmls
• piper
• postgresql
• rust
• socket.io
• sqlcipher
• svelte
• sveltekit
• tailwind-css
• tauri
• tokio
• tonic-grpc
• typescript
• vosk
• webrtc