Bolt.new Detector

Instantly verify hackathon submissions. BoltRadar uses AI to analyze GitHub repos, detect bolt.new usage, and ensure fair play for the World's Largest Hackathon.

Comment

Inspiration

In any competitive event, fairness is paramount. For the World's Largest Hackathon, where participants are encouraged to use powerful tools like bolt.new, ensuring that projects are built within the hackathon's spirit and timeframe is a massive challenge for organizers. Manually verifying hundreds or thousands of repositories is impossible. We were inspired to build a tool for the judges and the community—a "lie detector" for repositories. Our goal was to create a fast, data-driven, and intelligent way to verify if a project was genuinely developed for the hackathon using bolt.new, ensuring a level playing field for all participants. We also found it fitting to use bolt.new to build the very tool designed to analyze its own creations.

What it does

Bolt.new Detector is a powerful web-based analysis tool that provides instant insights into any public GitHub repository. A user simply pastes a repository URL and clicks "Analyze." The tool then performs a comprehensive, multi-layered analysis and returns a beautiful, easy-to-digest report that includes:

  1. Hackathon Eligibility Check: It instantly verifies if the repository's creation date complies with the hackathon's rules (e.g., created after December 1, 2024).
  2. bolt.new Verification Score: It analyzes commit history to determine the percentage of "verified" commits—a strong indicator of AI-assisted development, as bolt.new produces signed commits. This gives a quantifiable score of how much of the project was likely generated.
  3. AI-Powered Analysis Report: Leveraging the Google Gemini AI, the tool provides a sophisticated report with:
    • An Executive Summary of the repository's development patterns.
    • A calculated AI Likelihood Score.
    • Key Findings based on commit messages, frequency, and verification status.
    • Actionable Recommendations for judges.
  4. Detailed Commit Patterns: It surfaces critical metrics like the total number of commits, the ratio of verified commits, and the detection of "rapid commit" sequences, which often point to automated processes.
  5. Polished UI: All results are presented in a clean, responsive interface with dark mode, progress bars, and clear visual cues to make complex data understandable at a glance.

How we built it

We architected the Bolt.new Detector as a modern, full-stack serverless application, leveraging a powerful combination of technologies:

  • Frontend:

    • Scaffolding: The entire project was initiated with bolt.new, using the bolt-vite-react-ts template.
    • Framework: Built with React and TypeScript for a robust, type-safe user interface.
    • Build Tool: Vite provided a blazing-fast development experience and optimized production builds.
    • Styling: Tailwind CSS was used for its utility-first approach, allowing us to rapidly build a beautiful, responsive, and custom design with dark mode.
    • Icons: We used Lucide React for crisp, lightweight, and consistent iconography throughout the app.

  • Backend:

    • Platform: We used a Supabase Edge Function written in Deno (TypeScript). This provided a scalable, high-performance, serverless backend that could handle API requests efficiently from anywhere in the world.
    • Core Logic: The edge function orchestrates the entire analysis process.

  • APIs & Services:

    • GitHub API: This is our primary data source. We use it to fetch repository metadata (like creation date) and the full history of the latest 200 commits for analysis.
    • Google Gemini AI API: This is the brains of our operation. We engineered a detailed prompt that feeds the raw GitHub data (commit stats, messages, timing) to Gemini and instructs it to return a structured JSON object containing its analysis, which we then parse and display on the frontend.

Challenges we ran into

  1. Reliable AI Output: Getting a consistently structured JSON response from an LLM can be tricky. We invested significant time in prompt engineering, providing clear instructions, examples, and a strict schema to Gemini. We also built robust error handling and a fallback analysis generator in our backend, so if the AI response is malformed or fails, the user still gets a useful, data-driven report based on commit metrics alone.
  2. Defining "AI-Generated": How do you prove a project was AI-generated without parsing all its code? We landed on a clever proxy: GitHub's commit verification status. Since bolt.new generates signed commits, a high percentage of "verified" commits is a strong signal of its use. This allowed us to create a quantifiable and objective metric.
  3. API Rate Limiting: Both the GitHub and Gemini APIs have rate limits. We had to be mindful of this in our design, fetching only the necessary data (the last 200 commits) to provide a statistically significant sample without overwhelming the APIs. This lays the groundwork for future caching implementations.

Accomplishments that we're proud of

  • A Full-Featured, Production-Ready Tool: We didn't just build a script; we shipped a complete, polished, and user-friendly web application with a robust backend and a beautiful frontend.
  • Intelligent and Meaningful AI Integration: We went beyond a simple chatbot. Our tool uses Gemini for genuine data analysis and summarization, turning raw commit logs into actionable intelligence for hackathon judges.
  • The Meta-Achievement: We are incredibly proud of using bolt.new to build a tool that serves and supports the bolt.new and hackathon ecosystem. It's a testament to the power and speed of the platform.
  • Flawless UX: The seamless experience—from the loading animations to the clear, color-coded results and dark mode—is something we're very proud of. It makes a data-heavy tool feel light and intuitive.

What we learned

  • The Power of Serverless: Building on Supabase Edge Functions taught us how incredibly fast and scalable a serverless architecture can be. Deploying our backend was seamless.
  • The Art of Prompt Engineering: We learned that the quality of AI output is directly proportional to the quality of the prompt. Crafting precise instructions and providing structured context is key to building reliable AI-powered features.
  • GitHub API Nuances: We gained a deep understanding of the GitHub API, especially the commit data structure and the significance of the verification object, which became the cornerstone of our analysis.
  • Data-Driven Design: This project reinforced the importance of translating raw data into a compelling visual narrative. The challenge was not just getting the data, but presenting it in a way that tells a clear story.

What's next for Bolt.new Detector

We see a bright future for this tool, with many potential enhancements:

  • Backend Caching: To improve performance and respect API rate limits, we plan to implement a caching layer (e.g., using a Supabase table or in-memory cache) for frequently analyzed repositories.
  • Deeper Code Analysis: Move beyond metadata and have Gemini analyze actual code snippets for boilerplate, complexity, and other AI-generated code signatures.
  • Organization-Level Dashboard: Create a dashboard for hackathon organizers to view analytics across all submitted projects, identifying trends and outliers.
  • Browser Extension: Develop a browser extension so judges can get an instant analysis with a single click while viewing a repository on GitHub.com.
  • Support for Other Git Platforms: Extend functionality to support GitLab and Bitbucket repositories.

Built With

  • bolt.new
  • deno
  • github-api
  • google-gemini-ai
  • lucide-react
  • react
  • supabase
  • tailwind-css
  • typescript
  • vite
Share this project:

Updates