Window

Experience another life? Learn their culture? Understand their language? No problem.

Window pairs each person with one pen pal in a city they dream of visiting, then gives both people one opportunity each day to share a photo, caption, and voice note: a small, real window into their lives.

Inspiration

It can be difficult to learn about someone else’s culture from a distance. We can look up information about a country or city, but that rarely creates a personal connection with the people who live there. Language barriers can make that first connection even harder as many people fear being misunderstood.

We created Window to make international connection feel more natural. Window pairs people who want to visit each other's cities and share similar interests. Through photos, messages, and voice notes from ordinary moments, each person gets a more personal glimpse into the other's everyday world. Transcription, translation and cultural context developed using AI help reduce barriers while keeping the sender's real photo and voice at the center of the experience.

What it does

• Pairs intentional pen pals. Window matches people using shared interests and the cities they hope to visit. The experience centers on one active pen pal connection at a time.

• Lets people share daily Windows. Each person can send a photo, a short caption, and a voice note from an ordinary moment in their day.

• Makes language easier to share. Voice notes and captions can be transcribed and translated, but recipients can still listen to the sender's original audio. The app also offers conversation prompts based on the shared Windows to foster communication.

• Adds grounded cultural context. Meta AI can help identify relevant details in a submitted image and explain them in the context of the sender's city and culture.

• Connects physical companion Windows. A physical Window can replay a pen pal's knock rhythm, notify the paired phone of a knock, and change its lighting to reflect the time of day.

• Preserves meaningful moments. A bulletin board lets people save their pen pal's Windows. When a sender chooses to share location, it can also show where a photo was taken.

How we built it

Behind the scenes, Window is a mobile, realtime, and AI-assisted exchange system:

• Capture Layer → The Expo / React Native app lets each person create daily Windows with a photo, short caption, and voice note. Expo Camera, Audio, and Location provide the native mobile capture experience.

• Identity and Match Layer → Supabase Auth creates secure user accounts. Profiles, match preferences, and active pen pal relationships are stored in Supabase Postgres. The product is intentionally designed around one active pen pal connection at a time.

• Privacy Layer → Row Level Security (RLS) protects every exchange. A person can only read their own profile, their active pen pal relationship, and Windows shared with them. Media files and knock events follow those same relationship rules.

• Media and Delivery Layer → Photos and voice notes are uploaded to Supabase Storage. Once a Window is sent, Supabase Realtime delivers it to an open recipient device. If the app is closed, a database webhook reaches the backend, which sends an Expo Push Notification.

• AI Processing Layer → Our FastAPI server, deployed on Render, processes submitted Windows without exposing private AI credentials in the mobile app. OpenAI Moderation checks submitted content before it is shared.

• Language and Cultural Context Layer → Meta Muse Voice Transcribe turns a sender's voice note into text. Meta Muse Spark uses the photo, caption, transcript, and city context to translate the exchange, generate pronunciation-ready word stickers, and explain cultural details grounded in the real moment. ``

• Physical Window Layer → A Raspberry Pi Pico companion device reads a pen pal's shared state and knock events through Supabase. When someone sends a knock rhythm in the app, the paired Pico recreates that rhythm on its physical Window and can reflect the pen pal's local sky.

Together, these layers create one complete flow:

`capture → secure upload → safety and AI processing → realtime delivery → a meaningful received Window`

Challenges we ran into

• Hardware fit and materials. Fitting the electronics, wiring, and display elements into a window-sized physical prototype was difficult. When we did not have correctly sized wood available, we adapted the prototype with cardboard components while preserving the core interaction.

• Hardware-to-software connection. Connecting the Raspberry Pi Pico Window to the app's shared data required careful coordination. There was some variability with the sensors and how they would work with the knocking. To fix the variability we soldered the pins to the pico.

• Deployment configuration. Render and Supabase initially could not communicate because the `SUPABASE_URL` environment variable was malformed. The resulting server configuration error looked similar to an expired login, which made the root cause harder to identify. However we were eventually able to locate and debug this issue.

• Database schema mismatch. When attempting to send messages Windows initially failed immediately after the safety check. The AI server tried to save every processed result into a table intended only for translated text, so Supabase rejected the entire write and the Window never delivered. We fixed this by ensuring the server only writes fields that exist in the appropriate database tables.

Accomplishments that we're proud of

• Creating a complete exchange loop: capture → safety and AI processing → delivery → received Window.
• Making translation and cultural context feel like an addition to a real moment rather than a generic chatbot response.
• Building one cohesive experience across phone screens, backend services, realtime events, and a physical companion Window.
• Designing Window as a daily ritual that prioritizes depth over endless connections.

What we learned

We learned how AI can support social connection by helping people understand one another instead of replacing the relationship. Translation makes a voice note understandable, cultural context makes a photo richer, and safety screening helps make the exchange more trustworthy.

We also learned how to connect a React Native client, a secure Supabase backend, a Python AI service, push notifications, and constrained hardware into one product flow.

What's next for Window

• Finish and validate the Meta AI pipeline end to end with real submitted Windows.
• Improve matching around travel interests, language goals, and shared routines.
• Deploy Window onto its own application to be accessed by anyone.
• Run a small pilot with people in different cities and learn what helps a connection last.

Built With

• expo-router
• expo.io
• fastapi
• github
• meta-muse-spark
• meta-muse-voice-transcribe
• openai
• postgresql
• raspberry-pi
• react-native
• realtime
• render
• row-level-security
• supabase-auth
• supabase-storage
• typescript