Inspiration
Every time I wanna participate in a hackathon or have an idea to build, I always struggle to find truly committed team members. It has happened several times and has made me wonder why it is that way. Furthermore, I discussed it with other software engineers and discovered that it was a major problem in the ecosystem that most founders, software engineers, and builders face when seeking teammates. I also found research that Harvard Business School professor Noam Wasserman, who studied 10,000 founders, found that 65% of high-potential startups fail due to co-founders or employees not delivering as expected. One documented hackathon started with 50 participants and finished with 26 — a 48% overall dropout rate in that specific event, with participants explicitly describing being left to carry a project alone after teammates disappeared. A separate university hackathon saw dropouts throughout the first week specifically tied to reduced team cohesion and accountability in a remote/online format. An analysis of online hackathon communication logs found a pattern of teams dropping out either by explicitly telling a mentor, or — more commonly — simply going silent and stopping all communication without any announcement at all. That specific "just goes quiet" pattern is exactly the failure mode TruMatch's commitment verification is built to catch in advance.
What it does
TruMatch is two systems that share a brain.
Commitment Verification & Intelligence Layer. Any developer can sign up via GitHub OAuth to verify their developer commitment profile. Behind the request is an automated multi-stage intelligence pipeline that pulls quantitative GitHub activity signals (commit history, pull requests, issues, repo rosters) via @octokit/rest and @octokit/graphql, conducts a live context-aware AI interview streaming over Socket.IO powered by Gemini 3.6 Flash in JSON schema mode, and probes discrepancies between self-reported experience and raw code metrics with full disconnection state recovery (interview:resume). It fuses objective GitHub consistency metrics and AI interview evaluations into a transparent composite Commitment Score (0–100) accompanied by a confidence tier (HIGH, MODERATE, LOW, or INSUFFICIENT_DATA). If a user has a brand-new GitHub account with minimal history, the system handles it gracefully with informative messaging rather than an error screen. Every score is updated over time by post-project peer ratings (Delivery, Communication, Would Work Again), creating a self-correcting feedback loop for developer reliability.
AI Team Matchmaker & Execution Hub. A developer joining a hackathon can activate AI matching to assemble a high-commitment team. The matching engine evaluates the event pool and ranks candidate matches based on commitment alignment, tech stack synergy, and complementary role tags (Backend, Frontend, Mobile, AI/ML, DevOps, UI/UX, PM). Gemini 3.6 Flash constructs structured JSON AI Team Charters defining unified vision statements, role complementarity, availability agreements, communication protocols, and commitment promises. Reciprocal invite blocking prevents duplicate invite loops and excludes candidates with pending invites in either direction. Invites fan out in real time across Socket.IO WebSocket alerts, header notification bell badges, and hackathon charter banners. Upon accepting an invite, an interactive Acceptance Success Modal fires, reverse invites are auto-settled, and the Team Roster & Contact Hub unlocks — displaying teammate emails and GitHub profiles so developers can coordinate project architecture and complete official event registration together.
Platform Foundation. Around these sit the platform pieces a real product needs: GitHub OAuth 2.0 with native Node.js AES-256-GCM token encryption at rest, Supabase (managed PostgreSQL + pgvector) for all state and vector embeddings, BullMQ + Redis async task queuing for background sync jobs and repeatable rating reminder cron processors, and a Next.js 16 / React 19 frontend surfacing all of it through a sleek, dark glassmorphism developer dashboard.
How we built it
The product framing is simple. The architecture is where the work is.
Inputs (what data goes in)
Every evaluation begins with a developer signing up via GitHub OAuth. From that single seed, the system collects:
- GitHub Activity Signals — Commit history, pull requests, issue interactions, qualifying repository counts, contribution consistency, and account age fetched via
@octokit/restand@octokit/graphql, processed through a background BullMQ queue. - Encrypted Credentials & Memory — AES-256-GCM encrypted GitHub access tokens decrypted strictly in memory, plus per-user historical commitment signals and post-project peer ratings loaded from Supabase.
- Contextual Interview Transcript — Real-time turn-by-turn developer responses gathered through an interactive, multi-turn AI interview streaming over WebSockets.
- Hackathon & Candidate Pool Metadata — Event title, target team size, role tag requirements, location/venue, and participant pool rosters.
- Historical Peer Ratings — Post-project evaluations from previous teammates measuring delivery, communication, and "would work with again" sentiment, used to dynamically scale long-term commitment scores.
AI capabilities used
TruMatch employs four distinct AI capabilities across its pipeline:
| Capability | Where it's used | Models / Technologies |
|---|---|---|
| Contextual Discrepancy Probing | Conducting dynamic developer interviews, cross-examining self-reported availability against GitHub metrics | Gemini 3.6 Flash, Socket.IO |
| Structured JSON Schema Synthesis | Generating validated Commitment Score breakdowns and tailored multi-party AI Team Charters | Gemini 3.6 Flash (Structured Output Mode) |
| Candidate Compatibility Ranking | Evaluating role complementarity, stack synergy, and commitment alignment across developer pools | NestJS Matching Engine + Gemini 3.6 Flash |
| Autonomous Tool & Signal Extraction | Executing GraphQL multi-field queries, pagination, and numeric metric calculation | Octokit GraphQL, BullMQ, Redis |
None of these are chatbots or free-text classifiers. They are agentic workflows — LLMs orchestrated with deterministic data pipelines, structured JSON schemas, and mechanical validation gates that strictly constrain what the models are allowed to conclude.
What processing happens
The multi-stage pipeline processes data through a sequence of deterministic and agentic stages:
- GitHub Data Extraction (deterministic) — Authenticates via OAuth, decrypts token with AES-256-GCM, and fetches repository metrics via GraphQL round-trips. No LLM involved.
- Signal Aggregation & Confidence Grading (deterministic) — Calculates quantitative consistency score, qualifying repo count, and assigns a GitHub Confidence Tier. If data is minimal, assigns
insufficient_datawithout throwing an error. - Live AI Interview (agentic) — Socket.IO streams user turns to Gemini 3.6 Flash. Gemini compares self-reports against GitHub metrics and probes discrepancies. Supports
interview:resumestate recovery if connection drops. - Interview Analysis & Discrepancy Scoring (agentic) — Gemini generates a JSON object evaluating specificity, declared hours per day, communication notes, and flagged discrepancy explanations.
- Composite Commitment Scoring Engine (deterministic, no LLM) — Aggregates GitHub signal score and interview score using dynamic confidence-weighted formulas into a final 0–100 Commitment Score.
- Candidate Pool Filtering & Reciprocal Invite Check (deterministic, no LLM) — Filters hackathon participants. Excludes users with active incoming or outgoing pending/accepted invites in either direction to prevent reciprocal invite loops.
- Candidate Ranking & Team Charter Generation (agentic) — Gemini receives candidate profiles and generates ranked recommendations alongside structured AI Team Charters.
- Notification & Real-Time Alert Dispatch (deterministic) — Creates
Notificationrecords in Supabase and emitsnotification:newWebSocket events to the candidate's socket room. - Team Contact & Roster Activation (deterministic) — Accepts invite, creates
TeamMemberrows in Supabase, auto-settles reverse pending invites between the two users, and renders the Team Roster & Contact Hub with directmailto:links. - Post-Project Peer Rating Feedback Loop — Post-hackathon peer reviews dynamically recalibrate future commitment scores stored in Supabase and update the user's long-term reliability track record.
Outputs (what the user receives)
From the Intelligence & Matching Layer:
- Composite Commitment Score Card with final score (0–100), GitHub Confidence Tier, and flagged discrepancy breakdown.
- AI Team Charter with Vision Statement, Role Complementarity, Availability Agreement, Communication Protocol, and Commitment Promise.
From the Team Matcher & Contact Hub:
- Persistent Header Notification Center with unread badges, floating notification dropdown, and inline Accept/Decline actions.
- Hackathon Pending Invite Banner with full Team Charter preview card.
- Interactive Acceptance Success Modal guiding developers to coordinate with teammates and complete official event registration.
- Team Roster & Contact Hub with teammate avatars, role tags, commitment scores, GitHub links, and clickable Email Teammate
mailto:actions.
Human-in-the-Loop Decision
The AI never decides who joins your team or signs you up for a hackathon. TruMatch's intelligence layer recommends teammates and drafts structured Team Charters — but the system cannot lock a user into a team autonomously. A human must explicitly choose which hackathon to join, which candidate to invite, whether to accept or decline an incoming charter, and when to complete official registration on the hackathon event site. The team perimeter is always human-drawn. The AI operates inside it.
Responsible AI Guardrail
Risk: Over-reliance or gaming of commitment scores leading to team abandonment.
If developers could game their score or treat AI matches as absolute guarantees, teams would still experience ghosting. We mitigate this through four structural mechanisms:
- The system handles edge cases gracefully without hallucinating. When a developer has a fresh GitHub account with minimal commits, TruMatch assigns
insufficient_dataand relies on interview depth rather than assigning an arbitrary low score or failing. - Discrepancies are surfaced, not hidden. If a user claims 8 hours/day availability but their GitHub commit history shows sporadic 1-day activity every 3 weeks, the flagged discrepancy is displayed transparently on their score card.
- Reciprocal invite blocking eliminates invite loops. If User A invites User B, candidate recommendations for both exclude each other while pending. Attempting a reciprocal invite guides the user to accept the existing pending charter.
- Post-Project Peer Ratings create calibrated long-term accountability. Initial commitment scores are dynamically adjusted post-event based on real teammate ratings (Delivery, Communication, Would Work Again). Teammates who ghost suffer score degradation, making reliability provable over time.
Stack at a glance
- Database & Cloud Storage: Supabase (Managed PostgreSQL + pgvector vector extension + Supabase JS SDK), managed via Prisma ORM
- Backend: NestJS v11 (TypeScript Modular Monolith), Node.js, Bun
- Realtime & Queuing: Socket.IO WebSockets, BullMQ, Redis (ioredis)
- AI Runtime: Google Gemini API (gemini-3.6-flash, Structured JSON Output Mode)
- GitHub Integration: @octokit/rest, @octokit/graphql
- Security: AES-256-GCM token encryption (Node.js native crypto)
- Frontend: Next.js 16 (App Router), React 19, TypeScript, Vanilla CSS + Tailwind CSS v4, Motion (framer-motion), Three.js, react-globe.gl
Challenges we ran into
The commitment paradox. "Objective GitHub activity" and "developer capability" sound synonymous, but raw commit counts lie. Early prototypes over-indexed on public commit volume, which unfairly penalized developers working in private enterprise orgs or fresh open-source accounts. The breakthrough was separating raw metric collection from context evaluation — introducing the insufficient_data graceful path and utilizing Gemini discrepancy probing in structured JSON mode to evaluate developer reasoning rather than raw commit volume alone.
LLM determinism is a fantasy; pipeline determinism is achievable. Early attempts to regex-parse free-text interview summaries and team charters failed at the tail with broken markdown fences and unparseable JSON. We stopped asking the LLM to format text manually and instead bracketed it with deterministic stages — Gemini responseSchema enforcement, Zod DTO validation, reciprocal invite checks, and fallback handling. The LLM generates the analysis and charter proposals; the pipeline strictly enforces structure and execution rules.
The reciprocal invite loop and candidate pool anchoring. In early team matching, if User A invited User B, User B could still see User A in their candidate pool and send an invite back. This created reciprocal pending invite deadlocks where neither user received clear accept/decline banners. The fix was bi-directional candidate exclusion in the matching service combined with automatic reciprocal invite detection in the invite sender, guiding users to accept existing charters instead of generating duplicate invites.
Live WebSocket disconnects mid-interview and state loss. Real-time AI interviews streaming over WebSockets experienced occasional network drops during long turns, leaving users stranded with hanging spinners or forcing them to restart the Interview. We built the interview:resume gateway event with mid-stream turn rollback, allowing developers to reconnect and immediately restore their exact transcript state without duplicate prompts or lost history.
Payload dereferencing and silent Supabase notification gaps. Storing notification metadata inside Supabase JSON columns caused frontend notifications to silently render blank strings when accessing top-level fields directly. We resolved this by implementing safe payload dereferencing, paired with real-time WebSocket fan-out so online users receive instant alerts without polling delays.
Cold-start developers vs. high-signal veterans. Developers with brand-new GitHub accounts initially risked being categorized as zero-commitment users. We introduced dynamic weight scaling across four explicit confidence tiers (HIGH, MODERATE, LOW, INSUFFICIENT_DATA), allowing the AI interview score to automatically assume higher weight when GitHub history is sparse.
Peer rating feedback loop without toxicity or bias. Designing post-project peer ratings risked creating toxic feedback loops if unmoderated. We built anonymous post-project evaluation vectors (Delivery, Communication, Would Work Again) backed by statistical weighting persisted in Supabase, ensuring peer reviews dynamically recalibrate scores over time without single-event score destruction.
Accomplishments that we're proud of
Turning GitHub activity into a verifiable trust signal — without NLP. The entire commitment scoring system is grounded in quantitative, objective metrics only. No sentiment analysis, no self-reported badges. A developer's commit consistency, PR contribution record, and repo diversity produce a transparent, reproducible score that candidates can understand and improve.
A live AI interview that resumes mid-conversation. Building a stateful, socket-streamed AI interview that survives network disconnections and resumes exactly where it left off — without regenerating questions or duplicating context — required careful architectural thinking around session state, socket room management, and BullMQ job idempotency. The resulting experience feels seamless even on unstable connections.
An invite system that truly prevents team formation deadlocks. The reciprocal invite detection and bi-directional candidate exclusion system eliminated an entire class of team formation bugs (the endless invite loop) that would have made the matching experience unusable. The system now routes users intelligently toward accepting existing charters rather than generating stale duplicates.
Structured AI Team Charters as first-class product artifacts. Rather than letting a chatbot suggest "you two should work together," TruMatch generates structured, multi-section Team Charters with explicit availability agreements and commitment promises. These become binding shared documents between teammates, setting expectations before a single line of project code is written.
A post-project feedback loop that makes scores self-correcting over time. Most commitment signals are frozen at sign-up. TruMatch's peer rating system continuously updates developer scores post-hackathon using real teammate evaluations, meaning a developer's long-term reliability track record — not just their GitHub snapshot — determines how they're matched in future events.
A premium UI that matches the technical ambition. Building a real-time developer platform with glassmorphism, 3D globe visualizations, animated score cards, live notification bells with unread badge counters, and interactive Team Roster & Contact Hub cards while maintaining strict TypeScript across a monorepo took discipline. The result is a dashboard that feels genuinely premium alongside the feature depth.
What we learned
Structured JSON output mode is non-negotiable for agentic pipelines. Letting the LLM produce free text and parsing it downstream is a tax on reliability. Every AI output in TruMatch — interview analysis, discrepancy evaluation, team charters, score breakdowns — enforces a Gemini responseSchema. The LLM generates content; the schema enforces shape. We never trusted free-text output in production paths.
Supabase's pgvector extension opens architectural doors that a plain relational database can't. Storing candidate embeddings directly alongside relational data in Supabase — without spinning up a separate vector database — dramatically simplified the matching architecture. A single Supabase query can filter by role tags, sort by commitment score, and apply vector similarity simultaneously.
WebSocket rooms must be treated as first-class state, not an afterthought. Early notification implementations broadcast events globally and relied on user ID checks in the frontend. Rebuilding around named socket rooms (one per user) made real-time delivery reliable, eliminated notification leakage between accounts, and made interview:resume state recovery straightforward.
Graceful degradation is a feature, not error handling. Every edge case — sparse GitHub history, LLM schema violations, mid-interview disconnections, missing peer rating data — required a deliberate fallback path. Building those paths explicitly (rather than letting errors propagate) made TruMatch meaningfully more trustworthy than systems that crash or silently produce wrong results.
Monorepo DTO discipline pays compound interest. Keeping backend DTOs (/backend/src/**/dto/) and frontend API types (/frontend/src/lib/api/types.ts) synchronized across every schema change eliminated an entire class of runtime type mismatch bugs. The discipline was painful upfront and saved significant debugging time downstream.
Separating agent decision authority from enforcement authority is the right mental model for AI systems. The AI recommends candidates and generates charters. The application pipeline enforces invite rules, validates schemas, blocks duplicate invites, and gates team formation on explicit human actions. This separation made it easier to reason about safety properties and easier to debug when either layer misbehaved independently.
What's next for TruMatch
In-app team chat. The Team Roster & Contact Hub currently provides email and GitHub contact links. The logical next step is an in-app messaging layer built directly on Socket.IO, giving teammates a persistent communication thread tied to their shared hackathon project without leaving the platform.
Commitment Score history and trend charts. Right now the score is a point-in-time snapshot. Surfacing how a developer's score has evolved across events — with granular breakdowns of which peer ratings impacted it and when — transforms the score from a static number into a meaningful, auditable track record.
Automated hackathon discovery and ingestion. Hackathons are currently created manually. The next version automates event ingestion by crawling Devpost, MLH, and major hackathon aggregators via Cheerio, populating the event pool automatically so developers discover relevant events rather than waiting for manual curation.
Score verification badges for external profiles. A verifiable TruMatch Commitment Score badge embeddable in GitHub READMEs, LinkedIn profiles, and Devpost pages — letting developers carry their reliability signal beyond the platform and making TruMatch scores a portable developer credential.
Multi-event team continuity. Developers who worked well together across one hackathon should be able to pre-form teams for future events with a single click. Adding team continuity tracking — so peer ratings feed directly into future matching preferences — closes the full feedback loop between evaluation and team formation.
Expanded AI charter negotiation. Rather than generating a static charter, Gemini should facilitate an asynchronous charter negotiation between matched teammates — proposing terms, surfacing availability conflicts, and suggesting role adjustments until both parties confirm the final agreement.
Who the Application is for
This application is for developers, engineers, creators and builders who want to increase their productivity and quality of their products because with stronger teammates, anything immaginable can be built.
Log in or sign up for Devpost to join the conversation.