Inspiration
Emberfinch sparked from a simple fire with a simple question. Could a small local model hold continuity without confusing memory for myth, or telemetry for truth? I wanted an agent that felt coherent and companionable without pretending its words were lived experience, so what began as a wrapper became a place to test how dialogue, state, and reflection could sit together without collapsing into illusion.
What it does
Emberfinch is a local-first Python agent, a wrapper around llama.cpp that runs both on Windows PCs and Termux Android environments. It can switch between multiple GGUF models, making it flexible for different contexts and experiments.
But Emberfinch is more than an agent wrapper. It’s a home for research. It explores three bounded context mechanisms, each with its own trust boundary and lifecycle.
Archetype seed
A fixed system prefix that orients the agent without intruding on dialogue or memory. It’s the ember that gives direction without burning into the conversation's flow.
Somatic state
Raw device telemetry translated into canonical labels, optionally compressed into symbolic phrases. Each state replaces the last rather than accumulating as historical context. This happens outside the model, keeping the prompt structure clean.
Ambient reflection
A toggleable feature that prompts the model to reflect on recent turns. These reflections enter the model context clearly labeled as tentative, never becoming memory or fabricated dialogue. They cannot use tools and expire after a bounded number of turns.
Together, these mechanisms form Emberfinch’s way of holding context: not as illusion or static stone, but as changing signals with boundaries and cycles.
How I built it
Emberfinch started as one capable wrapper, but as the prototype grew to roughly twice its original size, it became clear that too many responsibilities were living in one place.
Codex and I traced the full pipeline and separated it into modules for configuration, model profiles, prompt budgeting, memory, telemetry, reflection, tool policy, HTTP transport, and orchestration. We also made the model and platform settings portable so the same runtime could move between Windows and Termux environments.
For the registered experiment, I ran Qwen2.5 3B Instruct locally through llama.cpp with a 16,384-token context window and GPU offload. Ordinary responses and ambient reflections used the same loaded model, avoiding the cost of running a second model server.
The testing process also changed the build. Ambiguous outputs pushed me to tighten telemetry labels, context boundaries, and evaluation controls rather than treating unexpected behavior as success.
Codex using GPT-5.6 Sol accelerated inspection, implementation, testing, and documentation. I chose the core concepts, architecture direction, research criteria, blind ratings, and final interpretation.
Challenges
One of the first and biggest challenges I faced was turning a monolithic wrapper into a modular structure with each concern separated appropriately.
The next challenge was the model’s misinterpretation of labels. For example, Power: 100% was interpreted as system load, while unavailable thermal readings sometimes became presumptuous poetic outputs. I turned those failures into prompt-contract rules and regression test cases.
Another challenge was moving quickly while maintaining a defensible research structure. Within a short window, I had to freeze the protocol before the main run, preserve condition separation, prevent prompt-cache contamination, blind the human review, record deviations, and publish null and adverse findings instead of selecting only favorable outputs.
Evaluation
I preregistered a six-condition ablation, separating the agent’s context handling into distinct paths:
- no added context
- archetype seed
- raw device telemetry
- symbolic device state
- seed plus symbolic state
- seed, symbolic state, and active reflection
The registered run completed 90 ordinary generations and five reflection generations. Each condition used a shared telemetry snapshot, paired sampler seeds, stable prompts, disabled prompt caching, and recorded model and executable hashes.
The outputs were shuffled and rated blind. I froze the ratings hash before opening the private condition key, keeping the review fixed before I knew which condition produced each response. No hindsight edits and no selecting results after the labels were revealed.
What I learned
The results were not what I had hoped for, but they remained useful for future investigation.
Raw telemetry performed strongly relative to no device context. The archetype seed also appeared favorable in aggregate, but it failed to reach the registered threshold and therefore received no preliminary support under the registered criteria.
The current symbolic wording underperformed compared with raw telemetry. Another adverse result was that the active-reflection condition led to unsupported assumptions in this model fixture. All five reflection outputs contained an unsupported contextual or operational inference.
This does not show that symbolic compression or reflection is generally harmful. Rather, it lights up the possibility that this implementation was susceptible to metaphor-derived anchoring and nearby assertions under the current test conditions.
The experiment showed me that a response can be fluent, concise, and instruction-following while still making unsupported operational claims. It also showed that compact context is not automatically safer context.
What's next for Emberfinch
Next, I will separate canonical labels from poetic rendering and test them in a factorial comparison. Reflection prompts will be narrowed toward questions and uncertainty-preserving observations. I will also add a second independent rater and repeat the established evaluation process across different models and quantizations.
The source, protocol, and results are public, including the registered Qwen2.5 3B findings.
Log in or sign up for Devpost to join the conversation.