MetaMind

Tech Stack
Functionality
Mind Map Generation Process
Mind Map
To Do List
Mind Chat

Inspiration

The idea behind MetaMind is to act as a "digital twin" — a personal assistant that learns about you while helping you learn more about yourself. In today's chaotic world, it’s challenging to keep track of thoughts, deliverables, and ideas. We envisioned a tool that could organize your mind, acting as an intuitive, AI-driven companion. Initially, we focused on the concept of a mind map: imagine if you could just speak into a mic and instantly generate an aesthetic, structured representation of your thoughts. This tool could serve as a memory refresher or something you could even share with friends.

While there are similar tools available, most of them only contain one of the many functionalities we thought of, and so we set out to build a one-stop platform. That initial idea evolved, driving us to integrate a wider range of features to create a comprehensive tool offering mind-mapping and much more.

What It Does

MetaMind is a web app that allows users to seamlessly organize their thoughts and life. Key features include:

Mind Mapping: Users can input their thoughts via voice or text to generate a mind map, with topics and subtopics organized for clarity. It connects thought as nodes to each other based on Open AI embeddings and similarity checks and also adds on to pre-existing knowledge seamlessly.
Task Management: MetaMind extracts action items from your thoughts, creating a personalized "To-Do" list.
MindChat: Users can chat with MetaMind which leverages insights from all your past conversations (via embeddings) and voice/photo inputs to answer questions like: "What do i have to show John at our meeting tonight?" or "What was the name of that guy i met at the airport?"
Image Analysis: Users can upload images containing text, which MetaMind converts into data, adding to your mind map or task list.

How We Built It

We built MetaMind using the T3 stack for the frontend and Django for the backend, hosted on GCP Cloud Run. Core elements include:

Supabase with PostgreSQL for structured data storage, paired with Pinecone as a vector database for efficient memory retrieval.
Voice Processing: We use Google’s Speech-to-Text API to transcribe voice input, extracting topics and action items to populate both the mind map and to-do list.
Image Processing: Google’s OCR technology converts text in uploaded images, which then integrates into the mind map and task list creation process.
GPT-4: The OpenAI GPT API helped us with key functions like identifying action items, topics, and subtopics for mind maps, as well as enabling dynamic interactions in our MindChat feature.
Mind Map: GPT-4 splits transcriptions into main topics, subtopics, and details, referencing prior data to prevent redundancy. It updates existing nodes with new details and creates new nodes for unique topics and subtopics. Using OpenAI embeddings, MetaMind detects and connects similar subtopics, expanding the web of ideas. Finally, users can explore their ideas in a navigable, interactive thought network.

We containerized the backend using Docker and deployed it on Google Cloud Run for scalability and streamlined performance. The frontend is hosted on **Vercel, with data management through Supabase.

Challenges We Ran Into

Integration: Integration the application involving T3.js and Django for a seamless pipeline from voice/text input to mind map creation. Also ensuring Supabase's python client works for Django when adding and retrieving through knowledge base.
Deployment Complexities: Hosting across various platforms and ensuring integration between our frontend, backend, and database introduced unexpected bugs and delays.
User Experience: Creating a seamless and intuitive UX/UI experience with multiple input methods (text, voice, and image) required careful design and testing.
Account Login/Authentication with Google OAuth: Implementing Google OAuth for user authentication presented challenges, as complexities surrounding the testing credentials occasionally disrupted the login experience and required extensive troubleshooting.

Accomplishments That We're Proud Of

Live Deployment: MetaMind is live at https://aiatl-aga.vercel.app/. Deploying within a hackathon timeframe required managing the complexities of multi-platform integration and adapting to various hosting requirements.
Feature-Rich Output: From voice input to image analysis, we successfully ideated, build and shipped multiple complex functionalities.
Speed of Development: Starting with an idea on Friday night and deploying a live, functional app by Saturday midnight required intense collaboration and quick iterations.

What We Learned

This project taught us the importance of rapid prototyping, deployment pipelines, and the challenges of building a multi-modal AI-driven application. We also learned a lot about integrating vector databases for memory storage and the intricacies of cross-platform hosting.

As individuals with backend experience, we deepened our knowledge in building and deploying full-stack applications. This experience enhanced our skills in coordinating front-end and back-end systems, optimizing data flow, and managing the complexities of deploying a comprehensive, user-facing product.

What's Next for MetaMind

Moving forward, we aim to:

Become a one-stop platform: we want to enable integrations with popular tools as a such as Google Calendar, Notion, email, and task managers to seamlessly connect your schedules, reminders, and to-dos.
User Testing: Actively gather feedback from early users to refine MetaMind’s features and user experience based on real-world usage.
Mobile App Version for on-the-go accessibility and ease of use.
Personalization: Make MetaMind more adaptable to individual user styles and preferences, allowing users to customize how their digital twin organizes, presents, and interacts with their data.
Social AI Interactions: Introduce a unique social layer where users' AIs can interact with each other, sharing insights or updates that reflect individual preferences. This could add a new dimension of personalized interaction and knowledge exchange.
Backend Scalability: Strengthen backend support to handle more users and data, ensuring smooth performance as we scale.