Inspiration
Concerts, festivals and hackathons all bring thousands of people but have little insight into workshop occupancy, event attendance, and other kinds of headcounts and movements. What if we could change that with a handful of cheap ESP32s, and track the movements of large groups of people by the wifi signals their devices already produce?
What it does
ESP32 nodes sniff the wifi probes phones already send, hash the MAC, and report signal strength to triangulate the location of devices in a set area. Claude works with a Fetch AI agent to act as a concierge, answering questions like "which areas are most busy between midnight and 4am" or "which hacker room is the quietest right now?" or "which team moves around the venue the most?"
How we built it
SpacetimeDB module (TypeScript), Go bridge for ingest and the API, React + three.js client, ESP-IDF firmware with browser flashing, Kotlin fingerprint app, Claude + uAgents concierge for movement queries, TigerData for travel history, Cartesia for audio queries from user questions. All deployable in one Dockerfile.
Challenges we ran into
Phones randomise MAC addresses per network for the most part, with some exceptions, so it's difficult to track unique users across networks. Additionally, a limited number of ESP32 nodes means a lower accuracy (and a minimum of three is effectively required for direction and accuracy). The Wifi here also isolates devices, but was circumvented by ESP32's ESP-NOW functionality which allows for peer-to-peer communication between the three disconnected ESPs (they only need to be put somewhere with a battery to function!)
Accomplishments that we're proud of
Can run with zero hardware and zero API keys, though we support integrations with several services that enhance the utility of the project.
What's next for Earshot
More than three ESP nodes, better integration for consumers to import custom venues with multiple floors and different styles of terrain, better profiling + consumer friendly casing and install methods that are also appropriate for the outdoors.
Log in or sign up for Devpost to join the conversation.