Lyra - Personal Ai

• 

  Creating Todo List with lyra

• 

  Commanding lyra to search mobile under 30000

Inspiration

We were inspired by the gap between AI that can answer questions and AI that can actually get things done. Most assistants generate text or provide instructions, while users still have to manually open apps, search websites, create files, and complete repetitive tasks. We wanted to build an assistant that understands what the user wants and takes action on their behalf.

What it does

LYRA — Multimodal Autonomous AI Mentor is a voice-first AI agent that converts natural-language commands into real actions. Users can ask LYRA to:

• Open and interact with applications.
• Search and navigate the web.
• Create files and folders.
• Automate browser tasks.
• Control supported system functions.
• Analyze documents and resumes.
• Remember context and user preferences.
• Create personalized learning paths and recommendations. LYRA follows an Understand → Plan → Execute → Verify approach rather than simply generating a response. ## How we built it LYRA is built primarily with Python using a modular agent architecture. Our stack includes:
• Google Gemini for AI reasoning.
• Ollama for local AI capabilities.
• Multiple AI providers through a provider abstraction and fallback system.
• Anakin API for additional AI workflows.
• Speech-to-Text and Text-to-Speech for natural voice interaction.
• macOS automation for real computer actions.
• Browser automation for web interaction.
• Persistent memory for context and personalization.
• Document processing for resumes and other files. The architecture separates AI reasoning from actual task execution, allowing LYRA to use specialized tools for different tasks. ## Challenges we ran into The biggest challenge was moving from text generation to reliable real-world execution. Some major challenges included:
• Making voice commands reliably translate into the correct actions.
• Interacting with dynamic websites such as YouTube.
• Controlling desktop applications reliably.
• Handling macOS permissions and authentication.
• Verifying that an action was actually completed.
• Managing failures and switching between AI providers.
• Maintaining context across multiple commands.
• Designing the system so new tools and capabilities can be added easily. ## Accomplishments that we're proud of We are proud that LYRA has evolved beyond a conventional chatbot into an agentic computer assistant. Our key accomplishments include:
• Built a voice-based interaction system.
• Integrated AI reasoning with real computer automation.
• Developed a modular tool-based architecture.
• Implemented persistent contextual memory.
• Added multi-provider AI architecture with fallback capabilities.
• Enabled browser and system-level automation.
• Started building an adaptive learning capability.
• Designed an architecture where LYRA can continuously gain new skills through additional tools and integrations. ## What we learned We learned that building an AI agent is much more than connecting an LLM to a chatbot. The most important lessons were:
• Understanding intent is different from executing intent.
• Reliable automation requires planning, error handling, and verification.
• AI models should not directly control everything; specialized tools should execute actions.
• Context and memory are essential for natural interactions.
• Provider abstraction makes AI systems more resilient.
• Building an autonomous agent requires balancing capability, reliability, security, and user control. ## What's next for LYRA — Multimodal Autonomous AI Mentor Our vision is to make LYRA a truly adaptive personal AI mentor and computer agent. Next, we plan to add:
• 👁️ Advanced screen and visual understanding
• 🧠 Deeper long-term memory and personalization
• 📚 Adaptive learning plans based on skills and progress
• 💻 More powerful coding and development automation
• 🌐 Advanced multi-step browser agents
• 📧 Email and calendar integration
• 🔌 More applications, APIs, and AI providers
• 🤖 Long-running autonomous workflows
• 🖥️ Cross-platform support beyond macOS

Built With

• ai
• api
• document
• git
• github
• llm
• python