Boring Brains - Bb

WorkFlow
Screenshot of Chrome Extension
AI Agents
Prompt chaining

Inspiration

The idea for Bb originated from my fascination with Microsoft's Windows Recall feature—a tool that created a searchable timeline of activities like apps and websites. While this concept was groundbreaking, it drew intense criticism over privacy concerns, with many comparing it to spyware. Though I shared the excitement for such a tool, I was equally committed to protecting user privacy. This led me to develop a similar tool that puts user autonomy first. The result is Bb: a Chrome extension that brings powerful recall to your browsing experience while keeping your data secure and self-hosted.

What it does

Bb transforms your browsing history into a powerful searchable timeline. It captures screenshots of your activities, such as online shopping, studying, or researching, and allows you to retrieve information using natural language queries. For example, you can ask Bb, "Where was that red T-shirt from for 499 rupees?" or "What was the price of the television I saw on Xiaomi's website?" Within seconds, Bb provides the answer along with the relevant image.

The best part? Bb is entirely self-hosted, keeping your data under your control. It features a secure authentication system that requires a login token—provided only once during server setup—for accessing your data across all your devices. Your data remains exclusively on your device or server and is only processed by open-source models that never train on your personal information. Whether you're searching for details from weeks ago or trying to find a forgotten webpage, Bb makes the process effortless while maintaining your privacy.

How I built it

Bb leverages Llama-3.2-90B-Vision-Instruct powered by SambaNova, with the backend hosted locally on your device or server. The core components include:

LangChain: Powers natural language processing through agent-based and prompt chaining techniques.
Docker: Simplifies deployment with commands like docker-compose build and docker-compose up, making it easy for users to set up their server.
Chrome Extension: Acts as the user interface for managing screenshots and issuing queries.
Codebase: Built for production with scalability and simplicity at its core.

Challenges I ran into

Building Bb was not without its challenges. Some of the key hurdles included:

Balancing privacy and functionality: Ensuring Bb's features were robust while keeping data safe from intruders and processing entirely local.
AI integration: Optimizing LangChain’s agent-based architecture for fast and accurate responses to user queries.
User-friendly setup: Simplifying the self-hosting process, particularly for users unfamiliar with Docker or server management.

Accomplishments that I am proud of

Completed this entire project in just 10 days.
Privacy-first design: Successfully built a tool that respects user privacy by keeping all data self-hosted and encrypted.
Fast AI-powered recall: Seamlessly integrated advanced AI to deliver fast and accurate responses to complex queries.
Ease of use: Designed a straightforward setup process with clear documentation, making it accessible to a wide range of users.

What I learned

This project taught me invaluable lessons about the intersection of privacy and AI. I learned how to:

Build scalable AI systems with LangChain.
Navigate user concerns about data security and address them effectively in the design.

What's next for Bb

I plan to enhance Bb with:

Multi-platform support: Bb was designed to be cross-platform, allowing users to access and query content from any device with a screen. While the original plan included biometric authentication on all devices too, time constraints and Flutter development environment issues led me to focus solely on the Chrome extension.
Customizable data retention: Allowing users to set their own parameters for data capture and storage duration.
Community-driven improvements: Open-sourcing the project to encourage worldwide developer contributions and create an implementation that surpasses Microsoft's version.