OpenHaq

Inspiration

Hackathons suffer from an opaque "black box" judging experience. Organizers drown in hundreds of submissions with broken repositories, invalid video links, and messy spreadsheets. Judges rush through evaluations under severe time pressure, constantly switching across separate tabs for video pitches, GitHub repos, and live deployments. Worst of all, participants who invest weeks building projects receive zero feedback when eliminated.

We asked: What if hackathon judging had the collaborative fidelity of a modern engineering workspace, automated rule compliance, multimodal AI assistance, and transparent feedback loops for every builder? This inspired openHaq—an automated evaluation engine and collaborative workspace designed to replace chaotic spreadsheets with transparent, multi-stage judging.

What it does

openHaq is an automated judging platform and microservice for high-throughput competitive hackathons:

• Automated Stage 1 Compliance Triage: Automatically verifies mandatory rules (open-source license, public GitHub repo, demo video, required skill packs) to eliminate broken submissions before judges spend time reviewing them.
• Configurable Multi-Stage Pipeline: Organizers configure stage progressions (e.g., Compliance Filter → Technical Review → Finalists) with custom advancement rules (Top N, Top %, Score Threshold), written-month date validation, and timezone auto-detection.
• Figma-Inspired Judge Canvas: Unifies the submission description, live web deployment iframe, and embedded video pitch into a single focused workspace with collapsible sidebars.
• Gemini Multimodal Video Timeline: Analyzes pitch video streams with Google Gemini 3.8 Flash to generate interactive, clickable timestamp markers (0:00–3:00), letting judges skip talking heads and jump straight to live demos.
• Grounded AI Handover Feedback: Evaluates numeric rubric scores and judge notes to synthesize structured, constructive participant feedback (Rubric Breakdown, Key Strengths, Next Steps) with strict judge sign-off and 422 validation preventing blank-score hallucinations.
• Human-in-the-Loop Advancement Review: Provides hosts with a qualifying vs. eliminated review modal before advancing stages, ensuring human oversight over automated cutoffs.
• Real-Time Participant Transparency: Dedicated participant view displaying live stage transitions, masked finalist leaderboards, and instant feedback release upon elimination.

How we built it

• Frontend: React 19, TypeScript, and Tailwind CSS with a clean, high-density Devpost-aligned interface. Includes interactive rubric sliders, YouTube iframe player controller, live search, and review modals.
• Backend Microservice: FastAPI (Python 3.13) providing REST endpoints, secure session authentication, role-based authorization (host vs. judge), and stage state transition guards.
• Primary AI Engine: Google Gemini 3.8 Flash via the google-genai SDK for multimodal video stream parsing, natural language rule extraction, and rubric-anchored feedback generation.
• Database & Storage: SQLite with Write-Ahead Logging (PRAGMA journal_mode=WAL) for atomic relational persistence of hackathons, stages, rubrics, submissions, and evaluations.
• Semantic Vector Search: ChromaDB persistent vector database indexing submissions by text embeddings for instant natural-language similarity search.

Challenges we ran into

• Streaming Video Latency & Synchronization: Ingesting and analyzing full YouTube streams dynamically with multimodal LLMs can take up to a minute. We solved this by implementing background persistence and caching video timeline markers in SQLite, ensuring that once analyzed, video jump-points load instantly (<10ms) for subsequent reviews.
• Preventing AI Hallucinations in Feedback: Generating feedback without strong guardrails risks generic platitudes. We designed the AI Handover service with strict validation: the API rejects requests if rubric scores are missing, grounding all output directly in the judge’s numeric ratings and criteria weights.
• Multi-Stage Advancement Complexity: Building flexible advancement rules (percentages, score cutoffs, top N) across both Pass/Fail compliance stages and weighted-criteria scoring stages required dynamic UI adaptation, hiding irrelevant numeric sliders on Pass/Fail stages and validating 100% rubric weight sums.

Accomplishments that we're proud of

• Zero-Friction Ingestion: Seeded and indexed 100 realistic Devpost submissions across multiple tracks with live ChromaDB semantic search and instant filtering.
• Eliminating Tab Fatigue: Built a complete, all-in-one judging canvas that keeps video, repo inspection, live demo webview, and rubric grading in a single unified view.
• Auditable Human-in-the-Loop AI: Created an AI assistance pipeline that accelerates human reviewers rather than replacing them, requiring explicit judge sign-off on AI-drafted feedback and host review on stage advancements.

What we learned

• Multimodal Video Understanding Is Transformative for Judging: Giving judges direct timestamp jump links to key architectural and demo moments cuts review time by more than half without sacrificing thoroughness.
• Brevity and Grounding Matter: Participants value specific, constructive feedback tied to the competition rubric far more than generic praise. Grounding LLM generation strictly to judge scoring creates genuine educational value for hackathon learners.

What's next for openHaq

• Direct Devpost Webhook Ingestion: Bi-directional sync with the Devpost API for automated project intake and real-time score publishing.
• Live Deliberation Rooms: Synchronized multi-judge breakout sessions with real-time score calibration and conflict resolution tools.
• Automated Code Artifact Verification: Deep AST parsing and commit timeline verification to confirm that submitted codebase commits were authored strictly within the hackathon window.

Built With

• chromadb
• fastapi
• google-gemini
• python
• react
• sqlite
• tailwind-css
• typescript
• vite
• websockets