Marty Supreme vscode extension

Make Development Great Again with this high-stakes VSCode extension featuring computer vision and catastrophic git penalties and most importantly, MARTY SUPREME!!!

Comment

💡 Inspiration

We've all been there: staring at a terminal, waiting for an installation to finish or a Docker container to build. It’s low energy. It’s boring. We asked ourselves: Why should a developer ever have to leave VSCode to have fun?

Inspired by the "Sillycon Valley" theme, we decided to create Marty Supreme—an extension that brings casino-tier gambling, computer vision, and catastrophic consequences directly into your development environment. We wanted to build something that wasn't just a distraction, but a high-stakes experience.

🏗️ What it does

Marty Supreme is a VSCode extension that launches Python-powered games directly inside your editor. But these aren't just normal games.

  • Touchless Gaming: We implemented Pong and Fruit Ninja. You play by waving your hands at your webcam, controlling the paddle or slicing fruit without touching your mouse.
  • The Casino: We built fully functional Blackjack and Slot Machines.
  • Hardcore Mode: This is where it gets real. If you enable Hardcore Mode and lose a game, the extension executes real system penalties.

🧠 How it uses Artificial Intelligence

Since this is an AI hackathon, we wanted to integrate Artificial Intelligence in two distinct, creative ways:

  1. Applied Machine Learning (Computer Vision): We utilized MediaPipe's pre-trained Machine Learning models and OpenCV for real-time hand landmark detection. The AI processes RGB camera frames to map 21 3D landmarks of a human hand. We extract the hand's centroid coordinates () to dynamically control the game physics (like the paddle in Pong or the blade in Fruit Ninja) entirely hands-free.
  2. Agentic AI (Model Context Protocol): We built a custom MCP Server to expose our VSCode extension as a set of callable tools for Large Language Models. This turns an AI assistant (like Claude) into an active participant. You can prompt your AI agent to "launch Tetris" or "check if Blackjack is running," and the LLM will reason about the available tools and execute the sub-processes autonomously.

🤪 Why it’s Silly & Creative

This project is the definition of "over-engineered chaos."

  • The Branding: The entire experience is wrapped in "Marty Supreme" branding, featuring Trump-themed audio effects. You get greeted with "Made in China" clips, and if you win a jackpot, you hear "So tired of winning".
  • The Danger: Most hackathon projects try to help you. Ours tries to delete your code. We programmed the "Hardcore Mode" to execute actual Git commands on loss. It can force-push your secrets to the main branch, delete random lines of code, or permanently delete your .env file.
  • The "Productivity" Tool: We bundled this as a developer tool, but it includes a Slot Machine that screams at you when you run out of credits. It is the absolute antithesis of productivity.

⚙️ How we built it

We built a hybrid architecture using TypeScript for the VSCode extension frontend and Python 3.8+ for the heavy lifting backend.

  1. Game Engine: We used pygame to render the games and handle the audio mixing for the voice clips.
  2. System Integration: We used Python's subprocess module to handle the "Hardcore" Git operations and file deletions.
  3. AI Integration: We combined the @modelcontextprotocol/sdk for the agentic LLM connections with Python-based Computer Vision libraries (opencv-python and mediapipe) for the interactive elements.

😲 Challenges & Surprises

The biggest technical hurdle was implementing the Model Context Protocol (MCP) Server to allow AI agents to control the games. Spawning a GUI application (Pygame) from a background Node.js process required complex process management, especially on Windows where we had to use specific flags (detached: true, stdio: 'ignore') to prevent the game windows from being hidden or instantly killed by the parent process.

We also faced the terrifying challenge of testing Hardcore Mode. It is genuinely scary to code a feature designed to delete files inside the very repository you are building. We had to implement a strict "Safe Mode" that targets dummy text files (penalty_test_1, etc.) just so we didn't accidentally wipe our own source code 30 minutes before the deadline!

What surprised us most was how seamless the final experience felt. Seeing an AI agent natively "understand" that it could launch games via our custom MCP tools felt like a glimpse into the future of agentic workflows.

Share this project:

Updates