recapsule

Bury a time capsule at a real place, locked until you return. "recapsule" rebuilds the moment from your photos, texts, and music, and the principles behind how you think and act.

Comment

Inspiration

We take thousands of photos and remember almost none of what was actually happening in them. Most memory apps make this worse: they show you a grid, or an AI that makes up a vibe with nothing behind it. We wanted a memory you can step back into, tied to the real place it happened, that tells you something about yourself and can show exactly where that came from.

So we built recapsule. You bury a time capsule at a real place, and it stays locked until you physically return.

What it does

You seal a capsule at a real location. It stays locked until you come back and stand there again, not on a timer. When you return and open it, recapsule:

  • Reconstructs the moment from your own data: the photo with its EXIF GPS and timestamp, the texts you sent around then, and the music you actually had on (from real Spotify listening history).
  • Writes a storyline of the moment with citation chips, so each line points back at the evidence it came from.
  • Lets you talk to your past self, a version that only knows what it knew at that timestamp.
  • Ends with one forward-looking reflection instead of leaving you in the past.

Across capsules, recapsule surfaces your principles: short, one-line patterns in how you think and act, like "I show up before I feel ready" or "I find my people by making things with them." They are plural, they connect to each other with typed relations (goes with, tension with), and each one is traceable to the specific moments and sources it was drawn from.

How we built it

recapsule is a memory engine and an app that meet at one canonical data type.

The engine (Python). Raw data rises through a fixed ladder:

  1. Ingest: each source becomes a canonical Event. iMessage is read read-only from chat.db; photos, Spotify, YouTube Music, and Claude chats have adapters that emit the same Event. Everything lands in one source-agnostic events table.
  2. Segment: events are grouped into coherent memory units by inactivity gap, with no LLM, and every unit keeps references back to its exact source rows.
  3. Retain: units go into Hindsight (embedded Postgres and pgvector), which builds episodic, semantic, and entity memory.
  4. Mint principles, cluster-first: we recall a set of memories, cluster them by their own embeddings, and the model only ever sees one cluster at a time, so it can only cite memories inside that cluster. A non-LLM step then verifies every citation, requires at least two distinct supporting memories, and runs a novelty check so a principle cannot just restate one memory.
  5. Link: principles connect into a graph with typed, grounded edges.

Confidence is computed from the evidence (how many memories support it, how diverse the sources are, how recent, and any contradictions), not from the model's self-report.

The app (recapsule). A zero-build, vanilla HTML, CSS, and JS mobile web app. A real Berkeley satellite map (Leaflet with Esri imagery) places capsules at their true EXIF coordinates. Reconstruction streams cues from multiple sources, the reveal renders the storyline and the real soundtrack, and the principles screen shows each belief with the moments behind it. Sound is synthesized live with the Web Audio API. Inference routes through OpenRouter (Claude for reasoning and minting, Qwen embeddings for the vector space). Secrets are managed with Doppler. The whole thing also runs locally from one command (recall serve): UI, API, and media on one origin, reachable from a phone over your own network.

Ethics and privacy

A memory app that reads your messages and photos has to earn trust by design.

  • Local-first: chat.db is read read-only, data stays on your device, and the phone talks to your own laptop, not a cloud. Only abstractions leave for inference, and inference is remote in this build with a path to local models.
  • Provenance by construction: a principle cannot be written directly. It only exists if it rose through the pipeline with citations, so the app cannot make a claim about you it cannot ground.
  • Wellbeing: every reflection ends forward-looking rather than pulling you back.
  • Known gap: inviting people back into emotionally heavy moments needs crisis-awareness before it runs on real data. We scoped that as a requirement, not an afterthought.

Challenges we ran into

  • Making "about you" provably grounded. The hard part was refusing to surface an insight that was not backed by evidence. Cluster-first minting plus non-LLM citation verification was our answer to hallucination.
  • Messy real data. Around 99% of iMessages store their text in an encoded attributedBody blob we had to decode; Google Takeout has no clean artist/track split. Keeping one canonical Event made each source a new model, not new architecture.
  • The map. We went through a satellite map that drifted, pins that clipped, and markers that looked like pills, before landing on a static, drawn satellite with capsule pins at true coordinates.

Accomplishments that we're proud of

  • A working pipeline that turns real personal data into principles with traceable evidence.
  • A polished, zero-dependency mobile app on a real satellite map with real GPS, real texts, and real music.
  • An architecture where provenance is a property of the path, not a field stamped on after the fact.
  • Treating privacy and wellbeing as design constraints from the start.

What we learned

The useful version of personal AI is not a better summary, it is a grounded one. Grounding is an architecture problem, not a prompt. The constraint (locked until you physically return) carried more weight than any single feature, and stating clearly what is faked earned more trust than hiding it.

What's next for recapsule

  • Real geofencing to replace the demo's "I'm back" button.
  • The contradiction loop: when your behavior contradicts a stated principle, recapsule asks you about it, and your answer becomes better data that sharpens the next principle. Designed and specced, not yet shipped.
  • Local inference to close the privacy loop end to end.

Built with: python, javascript, typescript, html, css, fastapi, hindsight, postgresql, pgvector, sqlite, leaflet, openrouter, claude, gemini, qwen-embeddings, pydantic, vite, react, web-audio-api, exif, doppler, github-pages, uv

Github links

https://github.com/aaryan-rampal/scaling-octo-fortnight

Built With

Share this project:

Updates