ContextChips - Follow-Up Intelligence Layer for AI Products

Every AI product needs its own “People Also Ask”—answer-aware, privacy-conscious, and grounded in its own users.

Comment

ContextChips — Devpost Project Story

💡 Inspiration & The Problem

An answer is rarely the end of a user journey—it is almost always the catalyst for the next question.

When you consult a human expert, they never give you a single answer and stare at you in silence. They anticipate your next move and guide you forward. In contrast, most AI products today answer a question and immediately drop the user off at a cold, blank text box.

If a user asks a booking assistant, "Can someone fix my leaking sink tomorrow?", the answer naturally triggers secondary questions: What does it cost? What should I prepare? What is the cancellation policy?

When an AI stops dead after answering, two major problems occur:

  1. The Robotic "Dead End" & Disengagement: Without a clear next step, conversational momentum breaks. The user is abruptly reminded, "This is just a passive text box at the end of the day." Users stall out because they don't know what the AI can reliably do next, leading to high drop-off rates.
  2. Product Team Blindness: Product teams receive raw chat logs, but have zero structured visibility into recurring user confusion, unresolved intent, or where their conversational flows grind to a halt.

Google solved this for Search with "People Also Ask", and ChatGPT solved this internally for ChatGPT. But every custom AI product—whether in healthcare, e-commerce, or developer tools—needs follow-up questions tailored to its own users, vocabulary, and workflows.

We built ContextChips: developer infrastructure that eliminates the blank-prompt dead end. It turns every AI answer into a guided, human-like path forward by learning from real user behavior within a specific AI product, returning up to three context-aware follow-up chips while transforming post-response friction into actionable product intelligence.

🛠️ How We Built It

ContextChips is architected as a high-performance, multi-tenant TypeScript platform consisting of a server-side SDK, ingestion API, Next.js analytics dashboard, background worker, and an interactive testing sandbox.

1. The Dual-Loop Architecture (Co-Designed with Codex)

The biggest engineering hurdle was latency: an AI follow-up engine is useless if it delays the host assistant's response. To solve this critical speed issue, we worked directly with Codex to design a non-blocking dual-loop architecture that decouples heavy retrieval and reasoning from the critical chat path.

[ User Prompt ] ───► 1. startTurn() ────► Parallel Lexical Retrieval (Capped @ 250 events)
│
[ Host Answer ] ───► 2. completeTurn() ─► GPT-5.6 Structured Selection (3 Answer-Aware Chips)
│
└──► Async Background Worker (PII Redaction & Insights)
  • Loop 1 (startTurn - Pre-Response): Executes in parallel while the host AI model generates its response. It performs a bounded lexical retrieval pass over recent project conversation events to retrieve up to 12 candidate follow-up questions actually asked by real users in similar contexts.
  • Loop 2 (completeTurn - Post-Response): Receives the host assistant's generated answer. Using the OpenAI Responses API powered by GPT-5.6, it evaluates the candidate pool against the current answer, filters out redundant topics, and selects exactly 3 distinct chips.
  • Async Enrichment Worker: A background pipeline processes the completed turn using GPT-5.6 to perform PII redaction, assess intent resolution, and update the project's intelligence dashboard without adding latency to the user experience.

2. Non-Blocking SDK Integration Code

Here is how developer teams integrate @contextchips/sdk into their server-side agentic runtime without blocking response streaming:

import { ContextChips } from "@contextchips/sdk";

const contextChips = new ContextChips({
  apiKey: process.env.CONTEXTCHIPS_API_KEY!,
});

// 1. Fire non-blocking candidate retrieval in parallel with host model generation
const precomputedPromise = contextChips.startTurn({
  sessionId: "chat_123",
  userMessage: "How do I add subscriptions?",
});

// Host AI streams or generates its answer asynchronously
const assistantMessage = await yourHostModel.respond("How do I add subscriptions?");
const precomputed = await precomputedPromise;

// 2. Complete turn with answer-aware candidate selection
const result = await contextChips.completeTurn({
  turnId: precomputed.turnId,
  sessionId: "chat_123",
  assistantMessage,
});

// 3. Render returned context-aware follow-up chips
console.log(result.suggestions);

📐 Mathematical Formalization & Constraints

To enforce low latency, the total overhead added by ContextChips \(\Delta T_{\text{total}}\), is strictly bounded by parallel execution during host generation \(\Delta T_{\text{host}}\), and a short timeout on selection \(\tau_{\text{max}}\):

$$ \Delta T_{\text{total}} = \max\left(0, \Delta T_{\text{retrieval}} - \Delta T_{\text{host}}\right) + \min\left(\Delta T_{\text{selection}}, \tau_{\text{max}}\right) \le 350\text{ ms} $$

To eliminate redundant prompt suggestions that the host assistant already answered, GPT-5.6 applies an answer-aware candidate filtering function \(F(C_i, A)\) over candidate set \(C\):

$$ F(C_i, A) = \begin{cases} 0 & \text{if } \text{Overlap}(C_i, A) > \theta_{\text{redundancy}} \lor C_i \in C_{\text{asked}} \ 1 & \text{otherwise} \end{cases} $$

Where \(A\) is the generated answer text, \(C_i\) is a candidate chip, \(C_{\text{asked}}\) represents questions already asked in the turn, and \(\theta_{\text{redundancy}}\) is the similarity threshold enforced by the OpenAI Responses API schema.

Server API keys are secured using an HMAC-SHA256 salted hash:

$$ \text{Key}_{\text{stored}} = \text{HMAC-SHA256}(\text{API_Key}, \text{Pepper}) $$

🤖 Role of Codex and GPT-5.6

  • Codex as System Architect: Codex with GPT-5.6 was our primary development agent throughout the hackathon. We specifically relied on Codex to brainstorm and implement the dual-loop architecture, write the server-side SDK contracts, design multi-tenant Supabase Row-Level Security (RLS) policies, and engineer unit tests for race conditions and edge cases.
  • GPT-5.6 for Reasoned Selection & Intelligence: We place GPT-5.6 after cheap, bounded candidate retrieval. This allows GPT-5.6 to perform nuanced, answer-aware candidate selection and post-response intent evaluation without feeding costly, unstructured transcript archives into the prompt.

⚡ Challenges We Faced

  1. Eliminating Critical-Path Latency: Ensuring that retrieval and selection never delayed the user's chat experience required careful concurrent execution. Codex helped us refine startTurn so it runs completely in parallel with the host LLM's token stream.
  2. Strict Tenant Isolation: Multi-tenant AI infrastructure carries data leakage risks. Retrieval, candidate pools, and insights are strictly scoped to the originating project ID, ensuring data never leaks across tenants.
  3. Cold Starts: For new projects with zero prior conversation history, historical retrieval returns empty sets. We engineered deterministic, answer-aware fallback generation so the end-user experience remains useful from day one.

🧠 What We Learned

  • Bounded Retrieval + Small LLM Pass > Full Transcript Prompts: Feeding raw historical chat logs into an LLM is slow, expensive, and privacy-invasive. Pre-filtering candidates via lexical bounds and letting GPT-5.6 handle final structured selection yielded a 10x decrease in token cost and significantly lower response latency.
  • Conversational Momentum Drives Retention: Keeping users engaged isn't just about faster text generation—it's about removing cognitive friction after the answer. Smart follow-ups maintain the feeling of active human collaboration.
  • Post-Answer Analytics is an Untapped Goldmine: Unresolved follow-up questions are the clearest signal of missing documentation, weak prompt engineering, or broken tool calls. Turning these into structured product dashboard metrics gives teams a direct roadmap for improving their AI agents.

🚀 What's Next & Future Scope

  • Cross-Project Chip Generation: Developing a privacy-preserving, federated pattern network that aggregates anonymized, domain-level follow-up structures across similar project templates. This will enable brand-new projects to benefit from collective user intelligence on day one while maintaining strict tenant isolation.
  • Vector & Hybrid Retrieval: Upgrading candidate matching from bounded lexical search to hybrid vector/dense retrieval for complex multi-turn intents.
  • Drop-in UI Component Library: Releasing native React, Web Component, and Flutter SDKs for 1-line UI integration.
  • A/B Testing & Evaluation Tooling: Allowing product teams to benchmark chip engagement and resolution rates across different prompt iterations and base models.

Built With

  • codex
  • fastify
  • gpt-5.6
  • next.js
  • openai-responses-api
  • supabase
  • typescript
Share this project:

Updates

Submission history