Git-Help

Ask your repo why. GitHelp turns GitHub and Slack history into answers with sources, and when nobody wrote the reason down, its Slack bot asks. Powered by NVIDIA Nemotron on Nebius.

Comment

Inspiration

On every software team I've worked with, the same question comes up in standups: "Why is this late?" The answer usually exists somewhere. It might be in a pull request review, buried in a Slack thread, or only in one person's head. GitHub shows what happened and Slack holds why, but nothing connects them, so people spend meetings rebuilding context that was already written down, and the reasons nobody wrote down get lost.

I wanted a tool you can ask "why did we keep JSON instead of moving to SQLite?" or "why isn't #6 done?" and get an answer built from the actual issues, reviews and messages, with a link to each source. When the answer was never written down, I wanted the tool to go and ask the person who knows.

What it does

GitHelp is a read-only GitHub dashboard with a memory.

  • Ask anything about the project. It indexes issues, pull requests, reviews, comments, READMEs and linked Slack channels. Answers cite their sources inline as [1], [2], and each citation links to the original.
  • Fills the gaps. When an issue passes its milestone's due date, the GitHelp Slack bot messages the assignee directly, asks what's holding it up and when they expect to finish, and adds the reply to the memory. If there's no reply, it sends one reminder after 3 days and then stops.
  • Shows what's late and why. The Overview lists every past-due item with how many days late it is and either the latest reason or exactly why there isn't one ("asked on Oct 2, no reply yet").
  • Posts a digest to the channel. A daily or weekly Slack post covers what's late, why, and what's due next.

How I built it

The backend is Express on Node 22, using the built-in node:sqlite with FTS5, and the frontend is a React single-page app served by the same Vite server. GitHub access goes through an allowlisted, GET-only proxy, so the app stays read-only even though GitHub's repo scope would allow writes.

Models: NVIDIA Nemotron on Nebius Token Factory. I added Token Factory as a first-class provider, reusing the OpenAI-compatible client:

Job Model
Routing and answers nvidia/nemotron-3-super-120b-a12b
Semantic search Qwen/Qwen3-Embedding-8B (4096-dimensional vectors)

Retrieval is hybrid. Each question runs through FTS5 BM25 keyword search and through cosine similarity over embeddings, and the two rankings are merged with reciprocal rank fusion:

$$\text{score}(d) = \sum_{r \in {\text{BM25},\ \text{cosine}}} \frac{1}{k + \text{rank}_r(d)}, \qquad k = 60$$

The top 8 chunks, up to 24k characters, go to the model as numbered <source> tags. Chunks follow how people communicate: a Slack thread is one chunk, other messages are grouped into conversations split at 15-minute gaps, and each issue or PR is a chunk with a separate chunk for its comments. Chunks are keyed by content hash, so unchanged text keeps its embedding.

Routing. Every message first gets a one-word PROJECT/CHAT classification from Nemotron. Small talk and general git questions skip retrieval entirely. As a safety net that doesn't depend on the model, a CHAT verdict is overridden when 2 or more of the message's keywords appear together in one indexed chunk.

The Slack bot uses Socket Mode with Node's built-in WebSocket. It matches GitHub logins to Slack users by commit email, then full name, then handle, and only messages people whose match is confirmed. A small date parser reads replies like "sick, done by Friday", "in 3 days" or "end of week" into a date, so the bot knows when to nudge again.

Challenges

Nemotron's reasoning was slowing down routing and getting it wrong. With reasoning on, the routing call took about 2.3 s, and Nemotron 3 Super classified "how many open issues are there?" as small talk. Setting reasoning_effort: "none" cut routing to about 0.4–0.6 s and fixed the misroute. Answers are grounded in retrieved sources, so reasoning added latency without adding accuracy. I also strip any <think> blocks from the stream in case a model sends them inline.

Citations went missing. Nemotron gave correct answers but dropped the [n] markers, even though the system prompt required them. Repeating the rule in one line right after the question fixed it. That one change took my evaluation from 13/19 to 18/19, and the remaining miss is a test case whose dates have since passed.

Models can't count from 8 chunks. "How many open PRs?" can't be answered from a handful of retrieved documents. I added an exact overview source built from the database, with a labelled counts block where totals say "open and closed" explicitly. A small model once read "6 pull requests" as six open ones. Counting questions get only the overview, because PR discussions in the context pulled the model toward the wrong number.

Reasons attached to the wrong issue. Asked about #8, a model borrowed #6's reason ("I was sick") even when told not to. Instructions didn't fix it. Removing the other issues' material from the context did. Questions that name an issue now get an exact fact sheet for that issue.

Dates. Milestone due dates are calendar dates stored as midnight UTC, so treated as timestamps, "Sep 29" became Sep 28 for anyone west of UTC. Due dates are now compared by date only, while event times are shown in local time.

What I learned

  • Removing material works better than adding instructions. When a model misused context, taking that context out fixed it more reliably than telling the model not to.
  • An evaluation set is essential. A 19-question suite against real demo data caught every regression when I switched from a local 4B model to Nemotron, and showed exactly what to fix.
  • Reasoning isn't free. For grounded RAG answers, turning off Nemotron's reasoning made it faster and more accurate.
  • The useful data often isn't written down. The bot asking a person was the feature that made the answers complete.

What's next

  • A guest demo mode with read-only sample data, so anyone can try it without connecting their own GitHub and Slack.
  • GitHub Projects date fields as deadlines, not just milestones.
  • A vector index (sqlite-vec) once a team passes about 50k chunks.

Built With

Share this project:

Updates

Submission history