Mark.ai

• 

  Example Q&A with text highlight

Inspiration

While preparing for the hackathon, we encountered Alexis, the ElevenLabs AI agent embedded in their documentation website. After interacting with it, we saw opportunities for improvement.

Problem

Let’s face it! Conversational AIs often produce inaccurate or misleading responses. This leads to low user trust in their answers.

Solution

Mark is similar to Alexis. He answers to questions about the ElevenLabs documentation. However, he supports his verbal responses with visual proof. He scrolls through the webpage of the documentation and highlights relevant content that directly addresses your questions.

Value Proposition

• Build user trust in AI-generated responses.
• Save time and minimize effort needed to navigate documentation.

How We Built It

We equipped Mark with a client-side tool that highlights text on the active page. The page’s HTML content is passed to Mark’s system prompt via dynamic variables, enabling him to analyze the article and generate human-like responses.

We integrated three ElevenLabs documentation articles into our Next.js application, embedded Mark through the ElevenLabs widget (<elevenlabs-convai>), and implemented client-side tools using TypeScript.

We deployed the app using Vercel, set up analytics using PostHog, and used ElevenLabs Sound Effects to spice up our video demonstration.

Challenges

We attempted to give Mark a feature that redirects users to another URL if the answer isn’t in the currently viewed article. However, the ElevenLabs AI agent widget has a limitation: its dynamic variables cannot be modified during runtime. Changing them mid-conversation causes the connection to drop.

Accomplishments

Despite challenges, Mark successfully delivers AI-driven document exploration, pioneering a new UX paradigm.

Lessons Learned

The ElevenLabs Conversational AI framework offers vast potential across various use cases.

What's next for Mark.ai

Mark has potential for:

• long-term success, since he can be implemented in any text-heavy environment (e.g., API documentation, legal texts, academic research, technical manuals, textbooks);
• scalability, since he can scale to thousands of pages efficiently using embedding-based retrieval;
• societal impact, since he reduces cognitive load associated with navigating documentation.

Mark is:

• unique, since existing AI agents either return links to sources or answer questions without references;
• innovative, since he uses a modern cross-browser API called text fragments;
• creative, since he enables building user trust in AI-generated responses using an unconventional approach.

Team & Contributions

• Wojciech Mroczyński (prompt engineering, backend development)
• Marko Golovko (full stack development)

Compliance

We adhere to hackathon rules and deadlines.

Built With

• elevenlabs
• nextjs
• posthog
• text-fragments-api
• typescript
• vercel