AFTERSIGHT

AFTERSIGHT: Your eyes see it. Your AI remembers it.

Comment

We live in a world where artificial intelligence can recognize almost anything, yet it struggles to remember the everyday details of our physical lives.

Have you ever stood in a grocery store wondering whether you have eggs at home? Forgotten where you left your keys? Or struggled to remember whether you packed something before leaving? Today's multimodal AI systems can understand photographs and videos. But recognizing an object in a single moment is fundamentally different from remembering it across time. That distinction inspired AFTERSIGHT.

We imagined a future where wearable AI doesn't just see the world but develops a persistent, evolving memory of what it observes.

A future where you can simply ask: "Do I have eggs in my refrigerator?" "Where did I last see my keys?" "What changed since this morning?" And receive answers grounded in actual observations rather than guesses. Our goal was to give AI something beyond perception: memory that understands time, context, and change.

What it does AFTERSIGHT is an AI-powered personal memory system that transforms everyday visual observations into persistent, searchable, and actionable knowledge.

Designed as a memory layer for wearable AI, AFTERSIGHT uses a live camera to simulate the visual perspective of smart glasses.

The system has five core capabilities.

  1. See: Understand the physical world Users can capture real-world scenes through a camera. AFTERSIGHT uses OpenAI's vision capabilities to identify visible objects, their approximate quantities, and environmental context.

For example, a refrigerator observation might identify eggs, milk, and other groceries. Each observation is associated with a timestamp and location context.

  1. Remember: Build persistent memory with HydraDB AFTERSIGHT transforms visual observations into structured memories stored in a self-hosted HydraDB graph database.

Rather than treating each observation as an isolated image, the system records relationships between objects, locations, events, and time. The graph supports the retrieval of relevant historical observations, allowing the AI to reason about previously encountered objects.

  1. Reason: Understand what changed This is the central innovation of AFTERSIGHT. Imagine the following sequence:
  2. 8:00 AM: Six eggs observed in the refrigerator.
  3. 12:00 PM: Two eggs used.
  4. 3:00 PM: Two eggs observed remaining.
  5. 6:00 PM: The last two eggs used.

A conventional retrieval system might find the older observation containing six eggs and mistakenly report that eggs are available.

AFTERSIGHT instead combines temporal observations with a deterministic state engine to calculate the latest known or estimated state.

It also recognizes an important distinction:

Buying six eggs does not mean those eggs are necessarily inside the refrigerator. AFTERSIGHT separates direct observations from inferred states and acknowledges when information is uncertain.

  1. Recall: Ask questions about your world Users can ask natural-language questions such as: "Where did I last see my keys?" "Do I have eggs?" "What changed in my refrigerator?" "What should I buy?" The system retrieves relevant memories from HydraDB and uses AI reasoning to generate answers supported by available evidence.

Responses distinguish between: Observed: Supported by direct visual evidence. Inferred: Derived from a sequence of observations or events. Unknown: Insufficient evidence to establish an answer.

  1. Act: Turn memory into useful actions AFTERSIGHT goes beyond answering questions. When a user asks what they should buy, the system can evaluate relevant memories and recommend adding an item to a shopping list. Actions require explicit approval before execution.

This demonstrates how persistent memory can support agentic workflows without removing human control.

How I built it

AFTERSIGHT combines multimodal AI, persistent graph storage, temporal state reasoning, and an interactive application interface.

          LIVE CAMERA
               |
               v
         OPENAI VISION
               |
               v
    STRUCTURED OBSERVATIONS
               |
               v
       HYRADB GRAPH MEMORY
               |
               v
      TEMPORAL STATE ENGINE
               |
               v
       MEMORY RETRIEVAL
               |
               v
       OPENAI REASONING
               |
         +-----+-----+
         |           |
         v           v
   ANSWER USER    SUGGEST ACTION
                     |
                     v
               USER APPROVAL
                     |
                     v
               EXECUTE ACTION

Challenges we ran into

  1. Self-hosting HydraDB under a hackathon deadline One of our biggest challenges was getting HydraDB running locally. We installed the Rust toolchain and native dependencies, compiled the database from source, configured local storage and authentication, and launched its graph server. We then verified that its HTTP endpoint was responding.
  2. Integrating with HydraDB's supported OpenCypher syntax Our initial graph queries were incompatible with portions of HydraDB's supported OpenCypher subset. We encountered errors involving query expressions, property assignments, and unsupported clauses. To resolve these issues, we examined the open-source repository and adjusted our queries to align with the database's supported syntax. This required changes to how observation nodes and relationships were written and retrieved.
  3. Moving beyond simple image recognition Detecting an object is comparatively straightforward. Understanding how that object's state changes over time is harder. We needed to distinguish direct observations from inferred changes and avoid treating outdated memories as current facts. Our solution was to build a deterministic temporal state engine rather than relying exclusively on an LLM for state calculations.
  4. Building a camera experience without smart glasses We did not have physical smart-glasses hardware available. Instead, we built a browser-based camera interface that demonstrates the same visual-memory architecture using a conventional camera. This also helped establish that AFTERSIGHT's memory layer can be independent of the specific camera hardware.
  5. Balancing functionality and presentation Under a tight deadline, we had to prioritize working memory ingestion, retrieval, reasoning, and actions over additional features. We deliberately deferred continuous camera analysis and more advanced automation to keep the core demonstration reliable. Accomplishments that we're proud of
  6. Built a working end-to-end visual memory pipeline. AFTERSIGHT can process camera observations using OpenAI and persist structured visual memories in HydraDB.
  7. Successfully self-hosted HydraDB. We compiled the open-source Rust database, launched its graph server, and connected it to our application.
  8. Implemented temporal reasoning beyond semantic search. Our state engine distinguishes outdated observations from newer evidence and supports chronological state reconciliation.
  9. Built evidence-grounded memory recall. AFTERSIGHT retrieves historical observations and distinguishes direct evidence from inferences and unknown information.
  10. Added a human-approved agentic workflow. The AI can suggest shopping-list actions based on remembered information while keeping execution under user control.
  11. Created an interactive smart-glasses-inspired experience. The application combines a real camera interface, memory timeline, natural-language recall, activity tracking, and system diagnostics.
  12. Passed all six deterministic state-engine evaluations. Our tests cover changing object quantities, corrected estimates, inventory assumptions, and last-known object locations. Most importantly, we demonstrated that AI memory can be more than a searchable archive. It can maintain an evolving understanding of the physical world. What we learned Memory is not the same as retrieval A system that retrieves relevant information isn't necessarily capable of determining which information is still valid. Temporal reasoning matters because the physical world continuously changes. Structured memory makes AI more useful When visual observations become structured records with timestamps, locations, and relationships, AI agents can do more than describe images. They can answer questions about previous experiences and support informed decisions. Deterministic reasoning and LLMs complement each other Language models are valuable for interpreting questions and communicating answers. However, deterministic calculations provide a more reliable foundation for tracking inventory changes and reconciling event sequences. Combining both approaches produces a more dependable system. Persistent memory should be model-independent By separating visual understanding, graph storage, and reasoning, AFTERSIGHT demonstrates a modular architecture. The memory remains available independently of the model used to interpret it. Trust requires acknowledging uncertainty An observation from several hours ago is not proof of what exists right now. We learned that a useful AI memory system must distinguish evidence, inference, and uncertainty rather than presenting every answer with equal confidence. What's next for AFTERSIGHT Our vision is to evolve AFTERSIGHT into a personal memory infrastructure layer for the next generation of wearable AI.
  13. Opt-in Auto Memory Introduce an optional mode that periodically samples camera frames while enabled. The system would identify meaningful changes and create new memories only when necessary. For example, if a refrigerator still contains six eggs, no new inventory event is required. If the quantity changes to two, the system records the change. Our goal is event-based memory rather than continuous video storage.
  14. Smart-glasses integration Explore integrations with supported wearable-camera platforms. The long-term vision is a hardware-independent memory layer that can work across smart glasses, smartphones, and other camera-equipped devices.
  15. Voice interaction Enable users to ask questions and receive spoken answers. For example: "Hey AFTERSIGHT, where did I leave my keys?" The system would retrieve the relevant observation and respond conversationally.
  16. Richer graph relationships Extend our HydraDB memory model to represent relationships between objects, rooms, activities, and people where appropriate and authorized. This could enable questions such as: "What did I bring with me when I left home?"
  17. Privacy and memory controls Introduce configurable retention policies, memory deletion, sensitive-scene filtering, and stronger user controls over what is remembered.
  18. More agentic capabilities Expand beyond shopping lists into reminders, missing-item alerts, packing assistance, and contextual everyday recommendations. Our ultimate vision is an AI assistant that doesn't simply understand what you show it, but can responsibly remember what matters and help you act on it.

Built With

Share this project:

Updates

Submission history