AyuMind AI

An AI fitness companion with long-term memory that remembers your meals, workouts, and health journey to give personalized insights over time.

Comment 1

AyuMind

An AI fitness companion that actually remembers your health journey — and can show you the evidence behind every answer.

Inspiration

In today's fast-paced world, many individuals living away from home—whether in hostels, shared accommodations, or college dorms—struggle to maintain a healthy lifestyle. Unstructured routines, easy access to junk food, and the absence of home-cooked meals often lead to poor dietary habits and neglected health. Over time, this can become the new normal, and the lack of a structured health management system only compounds the problem.

As I explored the intersection of agentic AI and persistent memory, I noticed a recurring gap: while people increasingly turn to AI chatbots like ChatGPT for health advice, these models treat each conversation as isolated. They have no memory of the user's history, forcing individuals to repeatedly re-explain their context, goals, and past data. This makes long-term health tracking and personalized coaching nearly impossible.

At the same time, existing fitness apps are heavily form-driven. They require manual logging—selecting foods, entering quantities, filling out multiple fields—which quickly becomes tedious. They lack an intelligent layer that understands natural language or the context behind what a user is saying.

This led me to a simple question:

What if an AI fitness companion could actually remember your health journey and understand it naturally?

That became the core idea behind AyuMind—an AI fitness companion that combines natural-language interaction with persistent memory, enabling the AI to understand not just what you ask today, but your entire journey over time.

What it does

AyuMind is an AI fitness companion built around persistent memory. Instead of treating every interaction as an isolated question, it builds a structured, queryable memory of the user's health journey and uses that memory to provide personalized answers over time.

Natural-language health logging

Users can describe their health data naturally instead of filling out forms. AyuMind can understand and log:

  • Meals and nutrition
  • Weight
  • Workouts
  • Sleep
  • Body scans
  • Blood reports
  • Supplements

For meals, the system extracts individual food items and quantities, then runs them through a separate nutrition-validation layer that calculates protein, carbohydrates, fat, and calories.

It also tracks estimated quantities, confidence, excluded items, and unresolved foods instead of silently inventing missing values.

Persistent long-term memory

Health information is stored as structured memory rather than just raw chat history.

AyuMind combines JSONB-based structured data with vector search and SQL retrieval, allowing it to answer questions about a user's history.

It supports retrieval operations such as:

Operation What it answers
Timelines "Show me my last two weeks of meals."
Event lookup "When did I last do legs?"
Counting "How many times did I skip breakfast?"
Recall "What was my weight in June?"
Aggregation "What's my average protein intake?"

An agent that reasons over health history

A LangGraph-based agent decides what the user is asking and selects the appropriate tools.

Tool inputs are validated against typed specifications and the engine's own vocabularies before database queries are constructed.

This lets users ask questions naturally instead of learning a specific command or interface.

Evidence-backed answers

AyuMind doesn't just return an answer and ask the user to trust it.

Each turn can produce a deterministic Evidence Trace, with citation markers connecting the response back to the underlying memory records.

The Glass Box interface lets users inspect:

  • Evidence
  • Retrieval queries
  • Nutrition calculations
  • Reasoning details

Short-term + long-term memory

AyuMind has two distinct forms of memory:

  • Short-term memory — recent messages from the current conversation, so the AI understands context.
  • Long-term memory — the user's accumulated health history across time.

These are deliberately separated. Conversation history can help the conversational agent understand context, but it cannot become a source of new long-term memories.

Insights and patterns

The system can detect longer-term patterns such as level shifts and intervention outcomes, with confidence based on effect, coverage, and specificity.

It can also retract an insight when later evidence contradicts it.

Beyond text

AyuMind also supports:

  • [x] Photo-based meal logging
  • [x] Profile and onboarding data
  • [x] Daily health review
  • [x] Streaming chat
  • [x] Authentication
  • [x] Health timeline
  • [x] Multi-thread conversations

How I built it

AyuMind is a full-stack AI application built with React + TypeScript on the frontend and Python + FastAPI on the backend. The agent orchestration is handled by LangGraph, while CockroachDB Cloud acts as the persistent memory layer.

Layer Technology
Frontend React + TypeScript
Backend Python + FastAPI
Agent orchestration LangGraph
Memory / database CockroachDB Cloud (JSONB + vector search)
Cloud AWS ECS, ECR, Secrets Manager, IAM, Bedrock

Agent architecture

LangGraph orchestrates the complete interaction:

User message
    ↓
Conversation history
    ↓
Planner
    ↓
Ingest / Retrieve / Consolidate
    ↓
Evidence assembly
    ↓
Narration
    ↓
Persist turn + trace

Challenges I ran into

The biggest challenge was not getting an AI model to respond—it was designing a memory system that could remain correct as the application became more complex.

Designing the memory architecture

My architecture evolved significantly during development. As I added structured health memories, retrieval, evidence traces, insights, and conversational context, I had to repeatedly revisit where each piece of information should live and how it should flow through the agent.

One important realization was that an AI with long-term memory also needs short-term conversational memory. I initially focused heavily on persistent memory, but eventually realized that an AI that can remember what a user ate six months ago should also understand what the user said a few seconds ago.

Separating short-term and long-term memory

Adding short-term memory introduced a subtle correctness problem:

Conversation history should provide context to the agent, but it must never become a source of new long-term memories.

For example, suppose a user says on August 16:

"Today I ate 3 eggs."

AyuMind stores the meal as 3 eggs on August 16.

On August 17, the user says:

"Today I ate 100g paneer."

The August 16 conversation is still part of the agent's short-term history. The conversational LLM can see both messages and understand the context.

But the memory engine must receive only the current August 17 message. Otherwise, it could see "today I ate 3 eggs" from the previous conversation and incorrectly create another memory for August 17.

I solved this by separating the two pipelines. Conversation history is provided to the conversational agent, while long-term memory ingestion receives only the current user turn. I also made log_memory() a zero-argument signal, meaning the LLM can signal that the current turn should be remembered, but cannot directly provide memory text to the memory engine.

This is enforced at the backend level rather than relying only on the LLM to behave correctly.

Making memory reliable rather than just persistent

Storing information is easy; making sure the AI stores the right information at the right time is much harder.

This led me to add backend-level safeguards and regression tests specifically for cases where previous conversation exists alongside a new health event. The August 16 → August 17 scenario is now covered by an automated test that verifies the previous meal is not re-logged.

I also had to make relative dates such as "today" use the current turn's clock consistently across the agent and memory extraction pipeline.

Production deployment

The other major challenge was deploying the system reliably to AWS.

Several issues only became visible during production deployment, including production dependencies missing from the Docker image and CockroachDB TLS certificate provisioning. The application could appear healthy while database-backed functionality was unavailable, so I had to distinguish between "the container is running" and "the application is actually working."

I eventually added the CockroachDB Cloud CA certificate to the production image, corrected the production dependency configuration, and verified the application against the real CockroachDB Cloud environment.

These challenges changed my approach from simply building features to building guardrails around an agentic system—making the memory pipeline, retrieval, evidence, and deployment behavior predictable even when the underlying LLM is not.

Accomplishments that I'm proud of

Building AyuMind as a solo developer, I am most proud of turning an idea about AI memory into a working system rather than just a chatbot prototype.

Building persistent memory that actually works

The biggest accomplishment for me is the memory engine itself. AyuMind can store structured health events, retrieve them using both structured and semantic search, build timelines and aggregations, and use evidence from those memories to answer questions about a user's journey over time.

I am particularly proud of the memory safety architecture. While building short-term conversation memory, I found that simply giving the previous conversation to the LLM could cause old messages to be accidentally stored again as new memories. I solved this at the backend level instead of trusting the LLM to always behave correctly.

Building a Glass Box instead of a black box

I am also proud of the evidence system. AyuMind can connect an answer back to the underlying memory records through an Evidence Trace and expose that information through the Glass Box interface.

The goal was not just to make the AI answer, but to make it possible to ask:

"Why did you tell me this?"

and actually inspect the evidence behind the answer.

Taking the project to the cloud

I also managed to take the application from development to a real cloud deployment using AWS ECS, ECR, Secrets Manager, IAM, and Bedrock, with CockroachDB Cloud as the persistent database.

As a developer, learning and using these AWS services in a real application was a major accomplishment. Debugging the production deployment also gave me a much better understanding of how an application behaves outside a local development environment.

What this project gave me personally

Perhaps the biggest accomplishment is that this project changed my confidence in what I can build.

I started this project while exploring agentic AI and persistent memory. After building the agent, memory engine, retrieval system, evidence layer, cloud deployment, and the surrounding infrastructure myself, I now feel confident that I can build agentic AI applications and memory-driven systems, not just conventional web applications.

I also gained hands-on experience with CockroachDB's distributed SQL, JSONB, vector search, indexing, transactions, and PostgreSQL compatibility, while learning how those capabilities can be used to build an actual AI memory system.

For me, AyuMind is as much a working product as it is proof that I can take a new technology, learn it, and turn it into a complete system.

What I learned

This project taught me that building an agentic application is very different from simply calling an LLM API.

An LLM is not the whole system

I learned that the LLM should not be trusted with everything. It is good at understanding natural language and making decisions, but important operations such as memory creation, data validation, retrieval, and database writes need deterministic boundaries around it.

This changed how I think about agentic systems:

The intelligence comes from the model, but reliability comes from the system built around it.

Memory is more than storing data

I initially thought persistent memory mainly meant saving information and retrieving it later. Building AyuMind showed me that the difficult part is deciding what should be remembered, when it happened, how it should be represented, and what evidence supports it.

I learned about bi-temporal data, structured JSONB payloads, vector retrieval, evidence traces, and the importance of separating conversational context from long-term memory.

How CockroachDB can power an AI memory system

Working with CockroachDB Cloud gave me hands-on experience with distributed SQL, JSONB, vector indexes, inverted indexes, transactions, and PostgreSQL compatibility.

More importantly, I learned how these features can work together instead of treating the database as just a place to store rows.

AWS by building a real application

I had previously explored AWS mainly through learning and student programs. With AyuMind, I actually used services such as ECS, ECR, IAM, Secrets Manager, and Bedrock together in a production deployment.

The deployment problems were especially educational because they showed me that an application working locally does not mean it is production-ready. Networking, secrets, TLS certificates, container dependencies, health checks, and cloud configuration all matter.

Confidence in building agentic systems

Most importantly, I learned by actually building.

I started this project while exploring agentic AI and persistent memory. By the end, I had designed and implemented an agent, a memory engine, retrieval and evidence systems, short-term conversation memory, and a cloud deployment.

That experience gave me confidence that I can take unfamiliar AI and cloud technologies, understand them deeply enough to use them, and build complete agentic AI and memory-driven applications with them.

What's next for AyuMind

The immediate next step is to test AyuMind with real users.

I plan to start by giving it to a small group of friends and let them use it as they would naturally use a fitness assistant—logging meals, asking questions about their history, tracking workouts, and interacting with it over time.

This will help me discover problems that automated tests cannot fully capture: whether the natural-language interaction feels intuitive, whether the memory is actually useful, where the agent misunderstands users, and which features people genuinely use.

Short term

  • [ ] Test AyuMind with real users and collect feedback.
  • [ ] Improve memory accuracy and retrieval quality based on real usage.
  • [ ] Improve nutrition estimation and handling of ambiguous food inputs.
  • [ ] Monitor agent behavior and production reliability.
  • [ ] Refine the Glass Box experience based on what users actually want to understand.

Longer term

I want AyuMind to move beyond simply remembering health data and become a personal health intelligence layer—understanding long-term patterns across nutrition, workouts, sleep, body measurements, and other health information.

I also want to explore richer multimodal interactions, better personalization, and more proactive insights while keeping the same principle at the core:

The AI should know why it believes something, remember what actually happened, and be able to show the user the evidence behind its conclusions.

Built With

  • amazon-bedrock
  • amazon-ecr
  • amazon-ecs
  • amazon-titan
  • anthropic-claude
  • aws-secrets-manager
  • boto3
  • cockroachdb
  • docker
  • fastapi
  • github-actions
  • langgraph
  • playwright
  • psycopg
  • pydantic
  • pytest
  • python
  • react
  • react-router
  • tailwindcss
  • tanstack-query
  • typescript
  • uvicorn
  • vite
  • zod
Share this project:

Updates

Submission history