AI Expert Assistant

LLMs hold vast knowledge. Imagine an app with AI, that configures itself based on natural language input. Every user can be a power user, with a personal AI in the driver’s seat of their app.

Comment

Inspiration

The idea for AI Expert Assistant was sparked by the increasing use of AI in coding tasks. Programming is the art of translating knowledge into functional systems while adhering to the rigid rules of a programming language.

On one hand, large language models (LLMs) possess vast domain knowledge and excel at responding in natural language. However, when it comes to generating large, functional codebases, their limitations become apparent. The intricacies of programming syntax and semantics mean that human developers often need to refine and assemble AI-generated code.

This limitation led to a pivotal insight: what if a simpler programming language could bridge this gap? Enter domain-specific languages (DSLs)—programming languages designed for simplicity and natural language resemblance. DSLs enable LLMs to express their domain expertise effectively, overcoming many of the challenges associated with conventional programming languages.

What It Does

The AI Expert Assistant utilizes the Chrome Built-In Prompt API to interact with both:

  • Humans: Using natural language for intuitive conversations.
  • Systems: Using a DSL to execute commands effectively.

The Assistant is embedded in a simple TODO toy-application. This does not diminish the innovation of the application. On the contrary, it highlights the innovation by keeping the focus on the Assistant’s functionality rather than the app itself.

Key Features

  • Clarifying User Intent: The Assistant refines vague user inputs by asking follow-up questions.
  • Task Execution: Once clarified, the Assistant translates the user’s intent into a DSL command. For instance:
    • A user expresses, "I want to read comedy books."
    • The Assistant asks for specifics, recommends books, and, once a choice is made, schedules daily tasks to read a chapter from the selected book.

This seamless interaction ensures tasks are not only understood but also actionable within the app.

How We Built It

  1. DSL Specification:

    • Created using the Peg.js library to define a syntax closely resembling natural language.
    • The DSL parser generates JavaScript objects from DSL commands for execution.

  2. AI Integration:

    • The Prompt API is instructed via the systemPrompt about the DSL syntax and its role in assisting users.
    • The Assistant alternates between providing human-like replies and generating DSL commands.

  3. Chat-Like Interface:

    • A user-friendly UI enables users to communicate with the Assistant.
    • Replies are either displayed as text in the chat or passed to the DSL parser for execution.

  4. Command Execution:

    • DSL commands are parsed, validated, and executed within the TODO application, creating actionable items.

Challenges We Faced

  1. Overly Talkative AI:

    • The Prompt API occasionally generates verbose replies or appends unnecessary text to DSL commands, causing parsing errors.
    • This was addressed through string-cleanup processes that extract valid DSL commands.

  2. Hallucination Issues:

    • The AI sometimes becomes overly engaged in conversations, leading to inappropriate responses. For instance:
      • A user states, "I want to eat a pizza."
      • Instead of creating a TODO task like ordering or baking pizza, the Assistant asks irrelevant questions about pizza size or delivery address, imagining itself as a pizza delivery operator.
    • The solution involves crafting more assertive prompts.

Accomplishments We’re Proud Of

The project introduces a feedback loop between the AI and the DSL parser:

  • When the Assistant generates a malformed DSL command, the parser returns a detailed error.
  • The Assistant uses this feedback to self-correct its syntax, ensuring successful execution.

This capability is significant because while DSL syntax errors are common, semantic errors are rare. The ability to self-correct means the Assistant reliably taps into its domain knowledge to execute tasks.

What We Learned

  1. Unlocking New Possibilities:

    • Bringing LLM capabilities to the browser opens a world of innovation. The other Chrome Built-In APIs are already powerful, but combining the Prompt API with DSLs unlocks unprecedented potential for super-user-centric applications.

  2. Handling Free-Form AI:

    • The flexibility of the Prompt API comes with challenges, such as unexpected tangents or hallucinations. However, with refined prompts and post-processing, these issues can be mitigated.

  3. Innovative Progress:

    • While the demo app isn’t perfect, it marks a step forward in integrating AI into practical workflows, demonstrating the promise of this approach.

What’s Next for AI Expert Assistant

Priority: Unified DSL Definition

Currently, the DSL definition and the AI’s systemPrompt duplicate information. A more efficient grammar definition is needed—one that can:

  • Generate a programmatic parser.
  • Describe the DSL syntax to the AI.

This will streamline DSL creation for diverse applications.

Additional Goals:

  • Enhance The Feedback Loop: Improve the efficiency of the AI-parser communication cycle.
  • Expand the UI DSL Capabilities: Develop more robust DSLs for building user interfaces.
  • Add User Consent: Implement safeguards requiring user approval before executing commands.

Built With

Share this project:

Updates