Code investigator

Find out the issue from the repo

Comment

Code Investigator

Inspiration

Developers spend a significant amount of time understanding bug reports before writing a single line of code. Even with modern LLMs, asking "Which files should I change?" often results in hallucinated filenames, incorrect evidence, or suggestions that aren't grounded in the repository.

I wanted to build a tool that bridges that gap—one that can take a GitHub issue and produce an investigation backed by real source code instead of model assumptions. The goal wasn't to replace developers, but to eliminate the repetitive process of searching through large codebases and provide a reliable How GPT-5.6 was used

GPT-5.6 was used throughout development as an engineering assistant for implementation, debugging, and architectural iteration. Rather than generating the entire application automatically, it accelerated development by helping refine complex components while the overall design and final decisions remained under my control.

Some of the areas where GPT-5.6 contributed include:

Implementing portions of the investigation pipeline. Refactoring the repository context builder into smaller, reusable modules. Improving prompt engineering for repository analysis. Helping integrate GitHub APIs, Prisma, Auth.js, and the pluggable LLM provider architecture. Debugging TypeScript, Prisma, and Next.js issues during development. Reviewing code for consistency and suggesting improvements to validation logic. Assisting with deployment preparation, environment configuration, and production debugging.

The project went through multiple iterations as GPT-5.6 helped identify implementation issues while I evaluated the suggestions, modified the architecture, and tested the results. This iterative workflow made it possible to move from an initial prototype to a more robust investigation pipeline while keeping the final design decisions under my control.

How Codex was used

Codex was primarily used as an AI coding assistant directly inside the development workflow. It accelerated repetitive engineering tasks such as:

generating boilerplate components and API routes, explaining unfamiliar code during development, suggesting refactors, producing initial implementations that were later reviewed and modified, assisting with repository navigation and debugging, helping verify implementation details while iterating on the investigation engine.

The majority of the application was developed through an iterative workflow where I reviewed, tested, modified, and integrated Codex-generated suggestions rather than accepting generated code unchanged.starting point for implementing a fix.

What it does

Code Investigator analyzes a GitHub issue and generates an evidence-backed investigation.

Given a repository and an issue, it:

  • Retrieves the repository structure from GitHub.
  • Builds a filtered repository context.
  • Uses an LLM to analyze the issue.
  • Identifies the most relevant files.
  • Extracts supporting evidence directly from the repository.
  • Produces an implementation plan explaining where and how the issue can be fixed.

Instead of returning generic suggestions, every finding is tied to actual files and code within the repository.

How I built it

The application is built using:

  • Next.js 16 (App Router)
  • React 19
  • TypeScript
  • Prisma 7
  • PostgreSQL (Supabase)
  • Auth.js with GitHub OAuth
  • OpenAI GPT-5.6, Anthropic, and Ollama (pluggable LLM providers)

The backend is centered around an investigation engine that orchestrates the complete workflow:

  1. Authenticate with GitHub.
  2. Fetch the repository tree.
  3. Build a filtered repository context.
  4. Run the investigation using the selected LLM.
  5. Validate every response against the repository.
  6. Store the investigation in PostgreSQL for future viewing.

Each investigation is persisted, allowing users to revisit completed analyses without rerunning the model.

Challenges I ran into

The biggest challenge was LLM hallucination.

Early versions frequently returned:

  • Nonexistent filenames
  • Incorrect evidence
  • Invalid line numbers
  • References that didn't exist in the repository

Simply improving prompts wasn't enough. Local models especially tended to default to common filename conventions instead of the repository's actual files.

To improve reliability, I redesigned the investigation pipeline so it relied more on deterministic backend logic than on free-form model output. Explicit file references from the issue are resolved by the backend, repository context is generated directly from GitHub, and every investigation passes through strict validation before any results are shown.

This required several iterations before reaching an approach that consistently produced grounded investigations.

What I learned

This project reinforced an important lesson about building AI-powered developer tools:

The most reliable AI systems combine LLM reasoning with deterministic software engineering.

Carefully controlling model inputs, validating outputs, and grounding every result in real repository data made a significant difference in both accuracy and user trust.

I also gained hands-on experience building a full-stack application integrating GitHub APIs, authentication, database persistence, and multiple LLM providers into a single workflow.

What's next

Planned improvements include:

  • Multi-file dependency analysis
  • Interactive implementation suggestions
  • Pull request generation
  • Semantic repository search
  • Incremental indexing and retrieval for larger repositories
  • Support for repository providers beyond GitHub
  • Support for additional LLM providers and investigation strategies

My long-term vision is to make Code Investigator a practical assistant that helps developers move from a GitHub issue to an implementation plan faster while keeping every recommendation grounded in the actual codebase.

Built With

Share this project:

Updates