Hereafter

Inspiration

Shoutout to the game BitLife, the movie Everything Everywhere All At Once, and of course Git version control.

In life, especially as young adults, we have many decisions big and small that we reflect on every day. Whether that's choosing higher education and career paths, to even the smallest social dilemmas. Or your meal plan that day, or whether you should study or go to a hackathon.

The irony of version controlling our futures is that we can't roll back from the present moment in real life. So we make do with the FUTURE. We can LIVE THE MULTIVERSE! Backed by personal context, real-time computed present state, probability, published statistics; to help us make our decisions and reflect on our journeys.

What it does

Builds a life out of unstructured input. Social links, personality quiz results, chatbot exports, documents, and any other context you want to give it goes through a single pipeline, and out as dated, sourced, confidence-scored events.
The user inputs a decision they are trying to make, with 2–4 choices / possible realities. The choices are always theirs.
*Researches each reality live a cloud browser reads real pages about each path, quoting every figure and verifying it in code.
Simulates a thousand lives per reality: Different outcomes with different probabiliities shown numerically and visually
Lets your "spirit" walk the futures. Commit a further hypothetical at any future step, branch from more decision nodes, compare paths side by side. Merge into main = the present reality
Asks <= 3 clarifying questions, each chosen by arithmetic because its answer would most change what you should do.

How we built it

Python/FastAPI backend, React + React Three Fiber frontend, Elastic Cloud Serverless for storage and retrieval, Browserbase for reading the web, the OpenAI API, and Codex as a development teammate.

Elasticsearch: Memory, Retrieval, and Evidence

Elastic Cloud Serverless holds three things: each person's life events, a shared pool of statistics researched off the web, and every year of every simulated future.

Search by meaning, using Jina embeddings v5: User-inputted events are stored in a semantic_text field, so Elasticsearch embeds them itself as they're written.
Jina reranker decides when we reuse research instead of re-crawling: Before fetching a statistic from the web, we score the new question against the questions we've already answered. If the score clears a threshold, we reuse the stored figure and skip the crawl.
An Agent Builder agent works out who you are now. Running on Elastic with five ES|QL tools, it searches your life log, sees what it still doesn't know, and searches again. It decides what to look up; we decide what the answers mean.
A scheduled Workflow expires old statistics. Every 24 hours it marks stored figures past their age limit so reuse skips them and the app researches them fresh.

Comparing two futures is one query. We index every simulated year of every path, so significant_terms returns what happens more often in the path you're considering than in the ones you didn't take.

agent picks what to look up → who you are now, conflicts resolved
→ "have we answered this before?"  (the Jina reranker decides, at 0.2)
├─ yes, and not stale → reuse it, no crawl
└─ no                 → read the web → verify the quote → write back to Elastic
→ the guess becomes a sourced figure → re-simulate 1,000 lives
→ re-index them → significant_terms changes what the user reads
→ 24h later the audit Workflow re-checks every figure's age

Browserbase - Live Web Research

Browserbase + Playwright for live web scraping: create Browserbase sessions, connect with Playwright over CDP, and use READABILITY_JS to pull out the useful page content while filtering out things like nav bars, headers, footers, and hidden elements.
Browserbase + Stagehand for dynamic pages: use Stagehand when basic scraping is not enough, such as expanding sections or extracting structured information from pages that require more interaction.
Browserbase for live research across decision paths: search the web for sources related to each path, read them in a Browserbase session, pull out relevant statistics and supporting text, verify the evidence in code, then send it back to Elasticsearch and re-run the simulation.

OpenAI: structured extraction, planning, narration

The model never gets to just say things. Every call goes through responses.parse into a Pydantic schema, and the schema is what actually enforces the product's rules, not a prompt asking nicely. Without that constraint, a model narrating someone's simulated life could invent a statistic or quietly reorder events to make a better story, and you'd have no way to catch it. With a schema, that's not a discouraged behavior, it's an impossible one: the model can't return an event type it wasn't given, and it can't add a probability field to a response schema that doesn't have one.

We use this for three jobs:

Extract. One Extraction type per source. Anything the source doesn't support just stays null instead of getting guessed.
Plan retrieval. PlannedQuery.tool is a Literal over three store methods, so the model can only ever name one of three. The bigger constraint is architectural: llm.py has no store import and no credentials in it at all. The model names a query, our code runs it, and the calls get replayed against the store so everything downstream sees typed LifeEvents. The model literally cannot touch data it wasn't handed.
Narrate. The model writes prose over a simulation log it did not create and cannot edit, it can't add, drop, or reorder events. We test this directly: app/sim is not allowed to import llm.

A fourth call scores each proposed event 1 to 5 on fit, difficulty, accessibility, and evidence strength, human-judgment stuff a model is actually good at rating. The schema for that call has no probability field, on purpose. Turning a 1-to-5 rating into an actual likelihood is a separate, deterministic step (probability.py), so the number you see never came from the model asserting a percentage, it came from code.

Codex: a build partner, not a runtime dependency

Codex determined failure taxonomy before writing any implementation. For every subsystem, what does a half-failure return instead of throwing an error? Answering that question first, with Codex, is what produced the three page tiers, the three retrieval stages (BM25, dense, rerank), and the three planners (elastic, llm, rules), each one a fallback for when the layer above it isn't available.

It also handled the semantic reinde script that migrates us to a new embedding model. This is trickier than it sounds because the semantic text field baked in the inference ID directly into the index mapping, so swapping models isn't a config change, it's a structural one. We had Codex write that migration dry-run-first, so it reports what it would do before it touches a live index.

The same approach carried into testing: a scripted fake plus LocalStore parity tests keep our full suite of 154 tests running offline and deterministic, and HEREAFTER_LLM=off is the default in CI, so the test suite never depends on a live model call to pass.

The useful bug this same approach caught: our first Elastic Agent Builder wiring used index_search, and the agent stuffed the person's id into the query string to compensate for a tool that had no filter and was reading every person in the index. All five tools are now ES|QL with ?person_id as a bound parameter it cannot omit.

Challenges we ran into

Nothing specific enough to matter has a statistic. No study exists about your ex or your first year somewhere. Solution: name the nearest reference class that does get studied, research that, and record how that population differs from this person.
We built a demographic simulator at the start, which was the wrong approach:* Our first engine was eleven Statistics Canada life tables with a full fetch → build → check pipeline and every transformation documented. It was full of niche information that failed to address general day-to-day queriesl, but was the direct reason for our reference-class research pipeline.
The Agent Builder agent leaked across people before we constrained its tools. Fixing it also made it far cheaper: an earlier version took 80 s and 21 LLM calls; the ES|QL version takes 42 s and 4, producing an identical state vector.
One concurrent Browserbase session means research is sequential: ~45 s to a first simulation, ~2 min to finished research. We made the wait legible by streaming the research feed and drawing branches immediately from estimates, firming them as evidence lands.

Accomplishments that we're proud of

Using a cross-encoder as a decision function with Elastic JINA embeddings
Indexing the multiverse: 444,037 docs so that "what's distinctive about this future" is an aggregation against sibling futures rather than a loop we'd have had to define ourselves.
Privacy consideration: pseudonymous indices, Fernet at rest, chat exports never persisted, other people's names replaced before any model call, and data erasure

What we learned

Keep interating on ideas until you find one with soul. Our team came up with so many creative ideas, even implementing some nearly to completion with beautiful UIs, until we landed on this one. Quite literally merged an alternate future with conviction.
Some times you can leverage the information in an the agent's tool calls, not necessarily its prose/judgement. Loggin this out helped us identify and resovlve a privacy bug of writing ids into search text
Aggregations are a modelling language: significant_terms with a background_filter turned out to be precisely the question we'd been approximating in numpy.

What's next for Hereafter

Feed live research into the decision agent, so requirement probabilities come from published base rates rather than model estimates. The interfaces already line up.
Voice notes The ingest pipeline is one path - a new source type is one branch in the router, exactly like the PDF branch - so a transcribed voice memo becomes dated events through the same extraction, confidence handling and index as everything else, with no change to Elasticsearch at all.
Calibrate: Backtest against real outcomes for decisions with known base rates and publish a calibration curve