Inspiration
A recurring problem was observed within our own group chats: a tech headline would be shared, someone would ask whether it was actually true or what the source was, and the response would then rest on whatever an LLM produced from memory, hallucinations included. The objective was therefore to build something small and inspectable that demonstrates an LLM can answer real questions about real news while showing its work — citing the exact snippet from which a claim was drawn and acknowledging when it does not know. AG News's Sci/Tech split was identified as the ideal small, clean sandbox for proving that pattern quickly during the hackathon.
What It Does
SciSift pulls the Sci/Tech-labeled slice of the AG News dataset live from Hugging Face, chunks and embeds the articles, and stores them in a ChromaDB vector database. When a question is asked — "what's new with AI chips," "which security stories came up" — the most relevant article snippets are retrieved and handed to Gemma, which answers using only that retrieved context and cites which snippet backs each claim. If the indexed articles do not contain the answer, this is stated rather than fabricated.
How It Was Built
The pipeline was split into clear stages: a Hugging Face datasets-server API loader that filters AG News down to Sci/Tech only, a LlamaIndex-based chunker, an ai-sdk-powered embedding step (text-embedding-004), ChromaDB for vector storage and retrieval, and a Gemma-powered generation step (called through the Vercel AI SDK's @ai-sdk/google provider) that turns retrieved chunks into a cited answer. The result is wrapped in a small Express app with a single dashboard page for ingesting the dataset and asking questions, deployable to Vercel.
Challenges Encountered
Getting AG News's raw text field (title and body concatenated with a backslash) into clean, citable snippets required custom parsing. The design was also constrained by Vercel's stateless serverless functions, since ChromaDB normally expects a persistent server — this pushed the team toward documenting a hosted-Chroma deployment path rather than assuming local storage. Tuning chunk size for already-short news articles took several passes to avoid over-fragmenting context.
Accomplishments
A working end-to-end RAG loop — live dataset pull, chunking, embedding, vector search, and Gemma-generated, cited answers — was built and wired together within the hackathon window, together with a deployment-ready structure (Vercel config, env template, README) so that it is not merely a notebook demo.
Lessons Learned
It was observed how cleanly a "citation-or-refusal" system prompt curbs hallucination compared to open-ended prompting, and how much of a RAG pipeline's reliability derives from the unglamorous parts — dataset cleaning and chunking — rather than from the generation model itself.
What's Next for SciSift
Planned directions include multi-source cross-referencing as a small agentic loop, an offline-friendly mode using a smaller Gemma variant via Ollama, substitution of a live news feed for the static dataset, and an automated groundedness eval to score answers against their cited sources.
Built With
- ag-news
- ai-sdk
- chromadb
- express.js
- gemma
- google-ai-studio
- graphics
- html
- huggingface
- javascript
- llamaindex
- node.js
- rag
- swiftui
- vercel
Log in or sign up for Devpost to join the conversation.