NeuroPilot

🌌 Inspiration

Inspired by the "Jarvis" interface from Iron Man, I wanted to build an AI that doesn't just talk, but actually operates. My goal was to bridge the gap between high-level LLM reasoning and local system execution, creating a true digital wingman that lives on your desktop.

🤖 What it does

NeuroPilot is an autonomous agent that sits behind a futuristic holographic HUD. It monitors your system telemetry while executing complex goals. Key capabilities include:

Autonomous Goal Mode: Decomposes high-level requests into multi-step executable plans.
Computer Control: Hands-free application launching and keyboard automation via pyautogui.
Web Intelligence: Real-time web search with AI-powered summarization briefings.
Bi-Directional Voice: Seamless interaction using Web Speech API and Wake-Word detection.
Secure File Assistant: Intelligent file management with built-in path protection and risk-confirmation layers.

🛠️ How we built it

The "brain" is Google Gemini 2.5 Flash, leveraging its superior speed and reasoning for real-time planning.

Backend: Flask (Python 3.12) managing the execution engine and automation services.
Frontend: A high-performance Vanilla JS and CSS3 overlay with 3D Sketchfab integration for the holographic robot.
Automation: Powered by pyautogui for system interaction and psutil for live telemetry.

🧠 Challenges we ran into

The primary challenge was balancing power with safety. We implemented a "Risk Confirmation Layer" that ensures the AI never performs dangerous actions (like deleting files or typing) without explicit user approval via the HUD.

⭐ Accomplishments that we're proud of

Successfully creating a "Zero-Glass" holographic UI where the 3D robot and floating telemetry panels feel like a single, cohesive AR experience. Watching the agent analyze a complex goal and generate a multi-step plan in real-time is truly rewarding.

📖 What we learned

We learned how to effectively use Gemini's JSON mode to drive a local system execution pipeline, and the importance of low-latency feedback in voice-controlled interfaces.