CodeSight AI

Inspiration

As developers, we've all felt the pressure of a growing codebase. Technical debt silently accumulates, critical code reviews become a bottleneck, and it's often unclear which parts of the system are the most fragile. We found ourselves asking the same questions on every project: "Where should we focus our refactoring efforts?" and "How can we catch subtle bugs before they hit production?"

The inspiration for CodeSight AI came from the desire to bridge the gap between high-level project management metrics and low-level code quality. We envisioned a tool that could not only tell us that there was risk but could show us precisely where it was and what it looked like, combining quantitative data analysis with the qualitative power of modern AI.

What it does

CodeSight AI is an intelligent health analysis tool for GitHub repositories. With a single URL, it provides a comprehensive, multi-faceted dashboard that helps developers and managers make informed decisions.

Here's how it works:

1. Macro Analysis (The "Where"): It analyzes the last 100 commits to calculate a "risk score" for each file based on its modification frequency and the number of contributing authors. This data is visualized as an interactive Treemap, where larger blocks represent higher-risk files that need the most attention.
2. Micro Analysis (The "What"): It takes the top 3 highest-risk files and sends them to a sophisticated AI model (powered by GPT-4) for an in-depth code review.
3. Interactive Dashboard: The results are displayed on a clean dashboard. Users can click any file in the Treemap to instantly see the detailed AI review, which identifies potential bugs, security vulnerabilities, and bad practices. Clicking on a specific issue automatically scrolls to and highlights the exact line of code, transforming a static report into an actionable, interactive tool.

In short, CodeSight AI turns a complex codebase into a clear, prioritized list of actions.

How we built it

CodeSight AI is a full-stack application built with a modern, performant technology stack:

• Frontend: The user interface is built with Vue 3 using the Composition API for clean, modular code. We used Tailwind CSS for rapid, utility-first styling and ECharts to render the beautiful and interactive Treemap visualization.

• Backend: The server is a high-performance FastAPI application written in Python. Its asynchronous capabilities were essential for handling long-running analysis tasks without blocking. We used the PyGithub library to interact with the GitHub API and the OpenAI API for the intelligent code review functionality.

• Core Logic:

  • The risk score is calculated using a weighted formula: risk_score = (modifications * 0.6) + (authors_count * 0.4).
  • To ensure a fast user experience, the AI reviews for the top 3 files are executed concurrently using Python's asyncio.gather, significantly reducing the total analysis time.

Challenges we ran into

1. AI Unpredictability: Our biggest challenge was handling the output from the OpenAI API. Despite using json_object mode, the AI would sometimes return slightly different JSON structures, missing keys, or data in an unexpected format. We solved this by implementing robust server-side validation and, most importantly, refining our prompt engineering. We had to become extremely specific in our instructions to the AI to guarantee a consistent and parseable response.

2. Performance: The initial sequential analysis took far too long. Sending three files to the AI one by one resulted in a frustrating wait time. The switch to a fully asynchronous backend using asyncio and running the API calls in parallel was a game-changer, cutting the analysis time by more than half.

3. UI/UX for Data Density: Presenting a large amount of information without overwhelming the user was difficult. Our initial left-right layout proved too cramped for the code view. We iterated on the design and landed on the current top-down structure, which gives the code review panel the full width it needs, dramatically improving readability and usability. We also faced a persistent bug with line number highlighting, which we solved by completely refactoring the component to render and highlight each line individually, ensuring 100% accuracy.

Accomplishments that we're proud of

• The Macro-to-Micro Connection: We're incredibly proud of how the Treemap seamlessly connects to the AI Review Panel. This feedback loop—showing you where the problem areas are and then letting you dive into the code with one click—is the core value of our application.

• The Interactive Code Highlighting: The feature where clicking an issue instantly scrolls to and highlights the corresponding line of code makes the tool feel powerful and intuitive. It elevates it from a simple report to a dynamic analysis tool.

• A Truly Asynchronous Backend: Implementing a non-blocking, concurrent analysis pipeline was a significant technical achievement that directly translated to a better user experience.

What we learned

• Prompt Engineering is a Superpower: We learned that the quality of an AI's output is directly proportional to the quality and specificity of the prompt. Aligning our prompts with our API parameters and desired data structure was a critical lesson.

• Asynchronous is Not Just for Speed: While performance was the initial driver, building with async/await from the ground up forced us to think about I/O operations and program flow in a more modern and efficient way.

• Never Underestimate UI/UX: A powerful backend is useless if the frontend is confusing. The journey of refining our layout taught us that user experience is just as important as the underlying technology.

What's next for CodeSight AI

We're just getting started! Here are some of the features we're excited to explore next:

• Support for Private Repositories: Integrate GitHub OAuth to allow users to securely analyze their private projects.
• Historical Trend Analysis: Track a repository's health over time. Imagine graphs showing your Technical Debt Index decreasing after a big refactor!
• Direct GitHub Integration: Allow users to create a GitHub Issue directly from a finding in the AI Review Panel with a single click.
• Customizable Analysis: Allow users to tweak the risk score weightings or provide their own keywords for identifying technical debt.

Built With

• nuxtjs
• openai