Hilo

Inspiration

Every AI chat starts with amnesia. If you use an assistant across a multi-day project, you pay a re-explanation tax at the start of every single session — pasting context, restating decisions, re-asking questions you already answered.

What it does

Brief Hilo once. When you leave (/quit — or even Ctrl+C), one extra Qwen call distills the entire session into structured memory: facts, preferences, open_threads, stored in memory.json. The next session injects that compact memory and opens by resuming exactly where you left off: "Welcome back — session 4. Where we left off: …". No re-explanation, ever.

How we built it

Python. All inference is Qwen ([MODEL]) on Alibaba Cloud Model Studio via the DashScope international OpenAI-compatible endpoint. agent.py runs the session loop — it is the deployment-proof file; every model call in it goes to Alibaba Cloud. memory.py owns distillation, forgetting, and crash-safe atomic persistence.

Track fit (MemoryAgent). Efficient memory storage and retrieval: one distillation call per session compresses the whole transcript into ≤40 structured items. Timely forgetting of outdated information: resolved threads are dropped at distill time, stale facts are rewritten instead of appended, and a hard 40-item cap prunes oldest-first. Recalling critical memories within limited context windows: sessions inject only the compact memory block plus recent turns — never past transcripts — so recall costs hundreds of tokens, not a context window.

Challenges we ran into

Making memory unbreakable: atomic writes (temp file + rename), corrupt-file quarantine, and failure-safe distillation — if the model call fails at quit time, memory degrades to "unchanged", never to "corrupted". We demo a mid-session crash on video.

Accomplishments that we're proud of

First live contact went green on night one: smoke test answered in 1.1s on qwen-turbo, and the brief→restart→resume demo worked on the very first real API run — our CI re-proves the whole flow on every push to main. The memory engine ships with a 13-check offline test suite: atomic writes, corrupt-file quarantine, hard 40-item cap, resolved-thread forgetting.

What we learned

Build on the international console from minute one (Singapore Model Studio + dashscope-intl) — the Beijing console is a trap for international builders. Model catalogs rotate fast, so our smoke test auto-discovers a working model instead of hardcoding one. And a memory agent is only trustworthy if failure degrades memory to "unchanged", never to "corrupted" — that principle shaped the whole engine.

What's next for Hilo

Multi-project memory namespaces; semantic retrieval over long memories; shared team memory so a whole team never re-explains the project either.

Built With

• alibaba-cloud
• dashcope
• openaisdk
• python
• qwen