PAAI- AI Personal Assistant

The AI executive assistant that learns all your preferences except how you take your coffee.

Comment

PAAI: The AI Executive Assistant that learns your every preference, except how you take your coffee.

Inspiration

The idea came from a conversation with my father, who works as a senior executive. He described the weeks it takes to onboard a new executive assistant - the back-and-forth of teaching them which emails matter, how to prioritize meetings, which senders to always flag. A good EA does not just follow instructions, they learn your working style over time.

I decided to test whether an AI could do that. I started small: one command to summarize his inbox. When he saw all his emails condensed and organized in seconds, his reaction told me everything. He was genuinely impressed, not in a "that is a neat demo" way but in a "this actually helps me" way. That was the reassurance I needed to keep building.

What It Does

PAAI is an AI executive assistant that connects to your Gmail and Google Calendar and orchestrates multiple specialized agents to handle your workload.

  • Summarize your inbox for any date, filtered to what matters
  • Prioritize emails into an actionable todo list ranked by urgency and importance
  • Draft professional email replies with the right tone
  • Schedule and fetch calendar events
  • Remember past conversations so you can say "tell me more about point 8" or "take yesterday's priority list and add it to my calendar"
  • Learn your preferences over time without you needing to re-explain how you work

The system uses Alibaba Cloud's qwen-plus model via DashScope as the backbone for all reasoning, planning, and evaluation steps. Every agent call, every preference extraction, every step evaluation routes through Qwen.

How We Built It

Starting point: LangChain

The project started as a simple LangChain agent with a few Gmail tools. It could fetch emails and summarize them in a single chain. The problem became obvious quickly: a single chain cannot handle multi-step tasks reliably. It would either try to do everything in one pass or lose context between steps.

The pivot to LangGraph

Moving to LangGraph was the first major architectural decision. LangGraph gave us explicit state, conditional routing, and retry loops. Instead of one agent trying to do everything, we built a graph where each node is a specialist:

  • Planner reads the user's request and generates a JSON step plan
  • Summarizer, Priority, Email, Calendar each handle exactly one job
  • Step evaluator checks each output against its specific goal before advancing
  • Final evaluator verifies the overall result

The evaluation loop was not in the original design. We added it after noticing that agents would produce technically valid but incomplete outputs. The evaluator creates a self-correction mechanism without human intervention, capped at 3 retries per step to prevent infinite loops.

The iteration count bug

One of the earliest architectural mistakes: the iteration counter was global across the entire plan. So if step 1 used 2 retries, step 2 only had 1 left before the system force-forwarded to final evaluation. The fix was resetting the counter when advancing between steps.

Adding memory: from sessions to ChromaDB

The first memory system was a flat SQLite sessions table - just saving the user input and final output per run. This worked for simple history display but failed completely for follow-up questions. When a user asked "what was point 8 from last time?", the system had no way to retrieve the right context because it was doing keyword search on a plain text blob.

We replaced this with ChromaDB vector embeddings using all-MiniLM-L6-v2 for semantic retrieval. Now when the user asks a follow-up, the history agent gets the most semantically relevant past messages rather than just the most recent ones. Two retrieval tools were exposed: GetRecentMessages for references to the last response, and SearchMessages for topic-based retrieval across all sessions.

The classifier problem

The original classifier was pure regex: if the message contains "always" or "never", treat it as a preference. This was brittle in obvious ways. "I don't really care about Chase emails" is a preference but contains no trigger words. "No, LeetCode should be high priority" is both a correction and a preference update but reads as neither to a keyword matcher.

We replaced the regex with an LLM call to qwen-plus. The classifier now returns structured JSON: types (task, preference, correction), whether a correction is present, and the contradiction strength. This let us handle hybrid messages naturally: "always add LeetCode to priority and summarize today's emails" routes to preference agent first, then the planner with the updated preference already active.

Building the preference system

The preference system went through three generations:

Generation 1: Flat key-value in SQLite. Save a rule, apply it globally. Simple but no concept of confidence or which agent the rule applies to.

Generation 2: Added scope so preferences target specific agents. "Keep emails brief" only applies to email_agent, not summarizer_agent. Same category, different scopes, stored as separate rows with PRIMARY KEY (user_id, category, scope).

Generation 3: Added confidence scoring, passive extraction, and decay. Every interaction now runs a passive extractor LLM call that looks for implicit preference signals in how the user phrased their request. Preferences start with a base confidence score by source type, get reinforced each time the same signal appears, and decay slowly when that preference domain goes unmentioned. The injection prompt separates hard rules (confidence above 0.7, seen at least twice) from soft suggestions (confidence above 0.5) so agents know which preferences are firm and which are flexible.

Switching to Qwen via Alibaba Cloud DashScope

The final infrastructure change was moving from OpenAI to Alibaba Cloud's qwen-plus via DashScope. Because DashScope exposes an OpenAI-compatible endpoint, the code change was three lines in llm.py: base_url, api_key, and model. Everything else (tool calling, structured output, LangChain integration) worked identically.

One Qwen-specific issue we hit: tool call arguments were returned as a JSON string instead of a parsed dict. LangChain expects a dict and throws a Pydantic validation error. The fix was a one-line guard: if isinstance(tool_args, str): tool_args = json.loads(tool_args).

Challenges We Ran Into

1. The LangChain tool wrapping conflict

When adding history retrieval tools, we decorated functions with @tool from langchain_core.tools to get StructuredTool objects, then wrapped those objects again inside Tool(). LangChain throws a Pydantic validation error because Tool(func=...) expects a plain callable, not another tool object. The fix was simple once understood: drop the decorator, use plain Python functions, pass them directly to Tool().

2. The evaluation loop confusing step goals with overall goals

The step evaluator was checking each agent's output against the user's original request, not the step's specific goal. This caused a correct summarizer output to be rejected because "it doesn't include a prioritized list" - even though that was the next step's job. The prompt now explicitly says: "evaluate ONLY against the step goal above, ignore whether the overall request is satisfied." The evaluator went from a source of confusion to genuinely useful quality control.

3. Making preferences actually robust

The hardest problem was making the preference system learn naturally without being brittle. The first version only triggered on "always" or "never". The second version used an LLM but still saved preferences as flat strings with no concept of whether they were reliable or one-off. The third version introduced: $$confidence_{\text{new}} = \min(1.0, confidence_{\text{old}} + \Delta_{\text{source}})$$ where $$\Delta_{\text{source}}$$is the reinforcement delta by signal type:

Source $$\Delta$$
Explicit statement 0.20
Correction of output 0.15
Implicit phrasing 0.08

And decay applied lazily on load:

$$confidence_{\text{decayed}} = \max(floor_{\text{source}}, confidence - 0.02\times n_{\text{interactions}})$$

where $$n_{\text{interactions}}$$ is the number of runs since the preference was last reinforced. This means explicit preferences never decay below 0.7 even if unused for a long time, while implicit ones can fade to noise if never repeated.

What We Learned

Evaluation is a first-class citizen. The step evaluator was an afterthought that became load-bearing. Without it, agents silently produce incomplete outputs. With it, the system catches and corrects its own mistakes before they propagate downstream.

Agentic systems need scoped memory. A global preference that applies to every agent creates conflicts. "Keep it brief" makes sense for email drafts but is wrong for email summaries where you want complete information. Scoping preferences per agent was not obvious at the start but became essential.

LLM classifiers outperform regex at natural language boundaries. The moment you need to understand intent rather than detect keywords, regex breaks. An LLM call that returns structured JSON is slightly slower but handles the full range of how people actually phrase things.

Simple architectural decisions compound. Resetting the iteration counter per step (not globally), preserving artifacts across planner replans, clearing stale step output before retries - none of these were complex changes individually, but each one fixed a category of failures that looked like model problems but were actually state management problems.

Starting with a real user's problem keeps you grounded. Every time the architecture got complicated, going back to "would my father find this useful?" was the right question to ask.

What's Next for PAAI

Active roadmap tracked in GitHub Issues:

  • Make a pathway for general questions - right now the system only handles structured task types; open-ended questions need a direct response path that does not force everything through the planner
  • Make a pathway for agents to ask questions - agents currently make assumptions when information is missing; they should be able to surface a clarifying question to the user before proceeding
  • Authorization and user management - the entire system currently runs as a single hardcoded user; multi-user support with proper auth is the prerequisite for any real deployment

Beyond the open issues: a skills system (saved named workflows triggered by a phrase), deeper calendar intelligence for scheduling across time zones, and a web search agent for context that lives outside Gmail.

Built With

  • chromadb
  • langchain
  • langraph
  • mcp
  • qwen
Share this project:

Updates