Flowchart - 1
Flowchart - 2

Persona: Your Digital Self in a Plushie

💡 Inspiration

Imagine a world where your personal AI exists not just in your phone or computer, but as a physical entity, your own AI-powered plushie! Persona is exactly that, a fusion of AI assistants like Alexa and Siri, social connectivity, and personal companionship in a tangible form.

🚀 What It Does

Persona is a hardware-based AI assistant that embodies your digital self, allowing you to:

Connect with friends physically, when two Persona plushies touch, their owners become friends.
Do everything Alexa or Google Home can do, but better! Persona integrates deeper with your personal life.
Access friends' schedules (with permission)—instead of asking your friend what they're up to, just ask their Persona!
Manage interactions via a web dashboard, see your connections in real-time and customize your Persona's behavior.

🛠️ How We Built It

1️⃣ Setting Up the Infrastructure

We used two Raspberry Pis, each stuffed inside a plushie.
The Pis communicate via MQTT, with a broker running on a central server using Docker.
A MongoDB Atlas cloud database stores friend connections and permissions.
Each Pi has a speaker, NFC reader/transmitter, and LED for interaction.

2️⃣ Speech-to-Text & AI Processing

Voice input is processed using OpenAI Whisper via the speech_recognition library.
A Retrieval-Augmented Generation (RAG) pipeline determines the intent of user queries.
Persona can recognize multiple intents, such as:
- General queries
- Current location requests
- Distance calculations
- Schedule access requests (powered by Google Calendar API)
Responses are generated and converted to speech using Typecast.ai for a custom voice.

3️⃣ The Web Dashboard

Built with Next.js & TypeScript, the dashboard:
- Displays real-time status updates via WebSockets.
- Allows users to manage their Persona and friend permissions.
- Acts as a visual indicator of what’s happening in the system.

🧩 Challenges We Ran Into

Many-to-Many Communication Complexity
- Unlike simple REST APIs or WebSockets, MQTT’s pub-sub architecture required careful handling of multiple concurrent streams.
Hardware Development Constraints
- Code that works on a laptop doesn’t always work on a Raspberry Pi.
- Development was waterfall-style, as only one person could work on the hardware at a time.
Speech-to-Text on Low-Power Devices
- Heavy models weren’t an option due to the lack of GPUs.
- Instead, we relied on API calls, increasing latency to 5+ seconds per request.
Synchronization Between Devices
- Ensuring smooth operation between the two Pis and the central broker required deep MQTT optimization.

🌍 Future Plans

We see Persona evolving into:

A more compact, portable device - our current plushie is quite large since it's a prototype.
A full-fledged AI companion - blending a pet-like presence with the intelligence of an AI assistant.
Enhanced security & privacy controls - to make sure users can define exactly who gets access to what.
Offline capabilities - to reduce dependency on API calls and improve response time.