MeetMind

Inspiration

As university students, we sometimes struggle to approach recruiters and industry professionals confidently. We wanted to make those first conversations easier by helping people find relevant professional context and thoughtful questions. We also wanted to explore hardware development, which inspired our longer-term vision of a wearable networking assistant.

What it does

MeetMind’s current code supports a workflow where an ESP32-S3 camera captures a name badge or readable handwritten name and company, then transfers the JPEG to a host computer. The firmware includes USB serial transfer and an alternative Wi-Fi/HTTPS upload mode.

The host application uses Meta’s Muse Spark API to extract a name, organization, job title, and event information when readable. With live web research enabled, the AI pipeline searches public professional information and generates a concise briefing with professional interests, organization context, and conversation starters linked to supporting sources.

A mobile-friendly web interface displays processing progress, sources, identity confidence, and information the system could not verify. Users can correct extracted details, review uncertain matches, try another candidate, cancel processing, and delete results. When evidence is insufficient to identify a person, the application can provide organization-only context or report that useful information is unavailable.

The application also includes a clearly labeled fictional demo mode. Résumé-based personalization exists in an earlier Raspberry Pi prototype script but is not integrated into the current application.

How we built it

We wrote the ESP32-S3 firmware in C using Espressif’s ESP-IDF framework. It handles button-triggered camera capture, device configuration, and image transfer. The firmware also includes upload retries, connection recovery, configurable status LEDs, watchdog handling, and optional deep sleep. For wired communication, a Python bridge using pySerial receives JPEG data over USB serial and submits it to the local application. The alternative wireless path uploads images over HTTPS on Wi-Fi, with device pairing and certificate validation.

We built the backend with Python, FastAPI, and Uvicorn. Pillow validates images, removes metadata, and checks for potential blur, darkness, or glare. Pydantic validates structured data, while HTTPX connects the backend to Meta’s Muse Spark API. The AI pipeline separates field extraction, identity research, briefing generation, and a verification pass, with a repair attempt and fallback behavior when results cannot be adequately supported.

The frontend uses HTML, CSS, and JavaScript, with responsive layouts, dark mode, copy controls, and live updates through Server-Sent Events. Its service worker caches the static interface; live AI research still requires connectivity.

The backend includes authenticated sessions, device-token management, duplicate-upload protection, rate limits, cancellation, a capture pause control, and automatic result expiry. Briefings and research caches are held in memory and expire after 30 minutes. The host processing pipeline does not normally save captured images; the USB bridge provides an explicit optional image-saving flag. Live extraction sends the image to Meta.

Challenges we ran into

We initially planned to use a Raspberry Pi 5 for its computing power and flexibility. However, the Raspberry Pi we borrowed remained unusable despite repeated troubleshooting across devices and software tools. This forced us to pivot to an ESP32 camera board and move AI processing to a host computer.

The change also required us to narrow our scope, work through embedded-device setup, and build communication between the camera firmware and the host application.

Accomplishments that we're proud of

For our first hardware hackathon, we are proud of adapting our design after a major hardware setback and building the software foundation for a camera-assisted networking tool.

We implemented the camera firmware, USB bridge, AI-processing backend, and mobile-friendly interface. The project’s verification record documents 47 passing host tests and a simulated image-upload-to-briefing flow using fictional results. We also tested live extraction and research components, although full live end-to-end completion and physical-camera reliability remain unverified.

What we learned

We learned to verify hardware early with basic flashing and capture tests before building the rest of the system around it. Setting up drivers, libraries, and development tools across teammates’ computers also makes troubleshooting and cross-platform testing easier.

We also learned to distinguish between implemented code, simulated demonstrations, and behavior verified on physical hardware. For AI-generated professional information, source links, uncertainty indicators, and user corrections are important parts of the experience.

What's next for NA

Our immediate priority is to validate the complete physical-camera-to-live-briefing workflow and measure its reliability and response time.

Next, we want to integrate résumé-based personalization into the current application so conversation starters can connect a professional’s experience with the user’s background. We plan to explore running that comparison locally on a working Raspberry Pi or another suitable device, evaluating model quality, memory requirements, and latency before choosing the hardware.

We also want to test wireless operation, add portable power, and design a 3D-printed enclosure or glasses attachment. These changes would move the prototype toward a wearable device. Fully offline operation would require local inference and an alternative to live web research, such as previously downloaded professional context.