Sacha

A Simple Project: Self-Adjusting CHat Agent.

Comment

Sacha: Self-Adjusting Chat Agent

Inspiration

I wanted to explore a simple yet powerful question: What if an AI could rewrite its own instructions based on user feedback? Most chat interfaces are static—you get what you get. But what if the AI could evolve its personality, style, and behavior in real-time through natural conversation?

The idea came from noticing how often I wished Claude or ChatGPT would "just be more concise" or "add more examples." Instead of repeating that preference in every conversation, why not let the AI learn and adapt?

What I Learned

Meta-prompting is powerful. By using the LLM itself as a "prompt engineer," I discovered that Claude is remarkably good at understanding vague feedback like "be more casual" and translating it into concrete system prompt improvements.

Per-chat isolation matters. Initially, I had one global system prompt. This meant changing the style in one conversation affected all others. The breakthrough was realizing each chat thread should have its own unique prompt—like different personas for different contexts.

UI simplicity wins. The core interaction is dead simple: chat → click "Edit Style" → describe what you want changed → continue chatting. No complex forms or sliders needed.

How I Built It

Architecture

Frontend (React + TypeScript)
    ↓
Backend (Node.js + Express)
    ↓
Claude 3.5 Sonnet (Anthropic API)
    ↓
Weave (W&B) for tracing

The Two-Level Prompt System

This is the key innovation:

  1. Meta Prompt (immutable): Explains the self-improvement concept to Claude
  2. System Prompt (mutable): Defines current behavior—gets rewritten based on feedback

When a user gives feedback, I send this to Claude:

"You are a prompt engineer. Improve this system prompt based on user feedback:

Current prompt: [...]
Feedback: 'Be more concise'

Return ONLY the new prompt."

Claude rewrites its own instructions, and future responses use the improved prompt.

Data Storage

Each chat has its own system prompt stored in chat-prompts.json:

{
  "chat_12345": {
    "prompt": "You are a concise, technical assistant...",
    "updatedAt": "2025-10-12T..."
  }
}

Challenges Faced

Challenge 1: The LLM was too creative

Early versions would add unnecessary preambles like "Here's your improved prompt:" before the actual prompt. Solution: Explicitly instruct it to return "ONLY the new system prompt text, nothing else."

Challenge 2: Feedback was too abstract

Users would say "be better" or "improve yourself." The meta-prompt needed examples and structure to handle vague feedback gracefully.

Challenge 3: Prompt bloat

After several iterations, prompts became verbose and contradictory. I added instructions to "maintain clarity and coherence" and "keep successful elements" to prevent this.

Challenge 4: State management

Managing per-chat prompts while keeping the UI responsive required careful state management. React's useState and proper API design (passing chatId everywhere) solved this.

The Result

A chat interface where the AI genuinely adapts to you. In one conversation, you can make it technical and terse. In another, friendly and verbose. The AI evolves per conversation, creating truly personalized experiences.

The best part? Users don't need to understand "system prompts" or "meta-prompting." They just chat and suggest improvements in plain English.

Future Ideas

  • A/B testing prompts: Compare two evolved prompts side-by-side
  • Prompt templates: Share successful prompts with the community
  • Automatic optimization: Let the AI suggest improvements proactively
  • Voice interface: "Hey, be more casual" while walking

Built with: React, TypeScript, Node.js, Claude 3.5 Sonnet, Weave

Core insight: The best interface for customizing AI isn't sliders and checkboxes—it's natural language feedback that lets the AI improve itself.

Share this project:

Updates