Human Human Interaction Lab

• 

  Logo for our app

• 

  Initial high-fidelity sketch of our card

• 

• 

• 

• 

• 

  3D printed case

Inspiration

At an event like Cal Hacks, you meet dozens of people by chance. Time spent with one person means less time to meet others, and someone you would have loved to meet may walk right past. Research points to the same challenge: people at mixers often gravitate toward people they already know (Ingram and Morris, 2007), and we underestimate the potential of conversations with strangers (Epley and Schroeder, 2014; Boothby et al., 2018). Yet new acquaintances and weak ties can open doors to jobs, ideas, and friendships (Granovetter, 1973; Rajkumar et al., Science, 2022).

Our initial community is collegiate hackathon attendees who arrive without an established network and need to find collaborators with complementary skills. Being in the same room does not tell you whom to approach, what you could build together, or whether the interest is mutual. We experienced that uncertainty ourselves while meeting people during the weekend.

So we asked: can we make each encounter more valuable on average? We wanted to give people a concrete reason to say hello and an easier way to stay in touch, without putting another feed between them and the people in front of them.

What it does

Before: Text our agent on iMessage, or fill out a short profile in the app, with who you are, what you can offer, and whom you hope to meet. The agent learns about you conversationally and saves profile details through tool calls.

During: Wear your phone in a 3D-printed lanyard holder. It becomes a live badge with your name in large text and up to three conversation topics. Claude evaluates each pair of attendees in both directions to identify potential value for both people.

As Ultra Wideband (UWB) detects a matched attendee moving closer, both avatars shift toward a shared color. Each badge highlights topics relevant to the other person, giving both participants a reason to start a conversation. When they begin talking, the two avatars hop into each other's badges.

After: The agent asks whether to save the person to your People list. When a conversation transcript is available, Claude generates a summary, topics to revisit, and a follow-up draft to help carry the connection beyond the event.

How we built it

• iOS app: We built the app in SwiftUI on iOS 26, using Nearby Interaction for UWB distance and direction. A procedural avatar drawn in Canvas supports blinking, squash-and-stretch animation, and reactions to touch. A flowing MeshGradient uses our designer's palette, while the cards' corner radii echo the phone's screen.

• Reciprocal matching: Claude evaluates how well each person can help the other and how likely they are to connect personally. We combine the directional need-fit scores using their geometric mean, then incorporate personal affinity. With other factors held equal, scores of 0.9 and 0.1 rank below scores of 0.5 and 0.5. This penalizes one-sided relevance. Claude also produces concise explanations and badge topics. Optional Voyage embeddings prefilter candidates in larger rooms on the Convex backend.

• Claude integration: JSON-schema structured outputs give matching results and conversation summaries predictable formats for the app. The onboarding agent uses tools to save profile details, confirm contact saves, and collect feedback. Conversation summaries use the available transcript alongside participant profiles to suggest concrete follow-ups.

• Convex backend: Convex manages profiles, proximity updates, matches, conversations, saved connections, and file storage. The iOS app subscribes to live badge state and other application data, so a proximity report from one phone can update the other person's display without a manual refresh. Actions handle external AI calls, while scheduled functions and crons coordinate matching, stale-conversation cleanup, notifications, and messaging.

• Cloud messaging: A Convex-hosted agent checks Photon for incoming messages every five seconds, runs Claude's tool-use loop, and processes an outgoing-message queue. It persists conversation history and includes retry handling and message identifiers to reduce duplicate processing. The cloud agent does not depend on a laptop remaining connected.

• SpacetimeDB alternative: We ported the backend to a SpacetimeDB TypeScript module while preserving the function names and JSON response shapes used by the app. The iOS app can switch providers in Settings. Our SpacetimeDB integration uses HTTP routes and transactions, with approximately one-second polling in place of Convex live subscriptions. The two providers maintain separate datasets.

• Design and hardware: We used Figma for storyboards, badge layouts, an interactive prototype, and our research poster. These designs informed the SwiftUI implementation. A 3D-printed lanyard holder turns an existing phone into a visible wearable display, making the software useful to the person approaching its wearer as well as the wearer themselves.

Challenges we ran into

• Keeping two badges in sync: UWB distance estimates fluctuate, so we debounce proximity updates. A conversation already in progress takes priority over a stronger match walking by. The display needs to communicate a stable interaction despite changing sensor readings.

• Making matching reciprocal and explainable: We refined this late in implementation, which meant reworking our Convex data model to score pairs in both directions. We also revised Claude's explanations to connect numerical scores to concrete reasons two people might want to meet.

• Coordinating operations at different speeds: Proximity reports arrive frequently, while AI calls and messaging take longer. We had to coordinate those operations without making the badge wait for every remote decision. Unreliable internet during the hack added another constraint; we worked around it with a makeshift connection-sharing network across several computers.

• Handling messaging constraints: Photon's shared iMessage line required people to initiate a conversation before the agent could message them. We needed to account for that onboarding step, queued messages, retries, and repeated agent ticks.

• Supporting two backend models: Convex subscriptions and SpacetimeDB HTTP polling expose different update mechanisms. Preserving a common API let us reuse client models while handling those differences explicitly.

• Getting the details right on real phones: Dark mode, autofill, phone-number formats, corner radii, and haptics all needed attention. These small details had a large effect on whether the experience felt coherent and usable.

Accomplishments that we're proud of

We connected a working experience on real phones: onboarding, matching, approaching another attendee, starting a conversation, and saving the connection. When a transcript is available, the system also produces a summary and follow-up suggestions.

The physical interaction is central to the project. A phone worn as a badge makes useful context visible before either person opens an app or asks an introductory question. Shared colors and personalized topics translate a recommendation into something both people can recognize in the room.

We also implemented the experience across two backend providers. The same client-facing API connects profiles, proximity, badge state, conversations, and contacts, while the Convex implementation supports the cloud-hosted messaging agent. Bringing these pieces together was a substantial part of the engineering work.

What we learned

Matching is only useful if people understand why they should act on it. A score alone does little in a crowded room; a visible signal and a relevant conversation starter make it actionable.

We learned to design around actions people already take: walking closer, glancing at a badge, and saying hello. Making those actions useful mattered more than adding another feature to the screen.

We also learned that predictable data formats and explicit fallback behavior matter as much as the model's answers. Missing transcripts, delayed messages, changing proximity, and interrupted connections all need understandable outcomes. Intentional design, visual polish, and a responsive architecture work together to make the experience trustworthy.

What's next for Human Human Interaction Lab

Our next step is to evaluate the experience with hackathon attendees who arrive without an established network. We plan to measure useful introductions per attendee, both participants' feedback on a match, the proportion of conversations that lead to saved contacts, and whether those connections lead to a follow-up after the event. These are evaluation goals, not measured outcomes from this prototype.

We want to explore an Apple Watch handshake interaction that lets both people save a contact through a shared physical gesture. The goal is to make exchanging details a memorable part of meeting someone.

We also plan to add agents that arrange introductions with both people's opt-in, maintain connection history across events, and improve behavior during connectivity interruptions. For the backend alternatives, we want to further evaluate the responsiveness and operational tradeoffs of live subscriptions versus polling under realistic event conditions.

Built With

• 3d-printing
• anthropic
• claude
• convex
• figma
• imessage
• ios
• photon
• spacetimedb
• swift
• swiftui
• typescript
• uwb
• voyage-ai