legal_agent

Inspiration

Ordinary people facing legal issues — labor disputes, contract conflicts, consumer rights — often hit the same wall: lawyers are expensive, and searching through legal statutes on your own is overwhelming. Could AI fill this gap? A 24/7 assistant that costs nothing, understands natural language, and pulls real legal sources? That's where this project started.

What it does

Users describe their legal problem in plain language (e.g., "My employer hasn't paid me for three months — what can I do?"). The agent first checks whether the user has provided enough information — asking follow-up questions if not. Once the picture is clear, it searches a local legal knowledge base and the web, then delivers a structured answer with cited legal sources. Responses stream token-by-token via SSE, just like ChatGPT.

How we built it

Agent Framework: LangGraph hand-written StateGraph with a custom Planner node (information completeness check) + ReAct loop (LLM autonomously decides which tools to invoke)
Dual Retrieval: Local FAISS vector store (legal PDFs) + DuckDuckGo web search, with a CrossEncoder Reranker for precision
Two-layer Memory: Short-term (full conversation history) + Long-term (LLM incrementally extracts structured case summary as JSON)
Backend: FastAPI + SSE streaming
Frontend: React + TypeScript + Tailwind CSS + Framer Motion
LLM: GLM-4.7 (Zhipu AI API)
Embedding: Nomic Embed Text (Ollama, local deployment)

Challenges we ran into

Ollama client version incompatibility: LangChain's OllamaEmbeddings didn't support the newer Ollama API. Built a custom lightweight wrapper using httpx with trust_env=False to bypass Windows proxy interference.
LLM hallucinating legal statutes: When users gave vague descriptions, the model would fabricate non-existent laws. Solved by enforcing RAG + web search — every claim must cite a source.
Long conversations → LLM forgets early details: As history grew, the model lost track of case facts mentioned earlier. Fixed by introducing incremental case_summary extraction injected into every SystemMessage.
RAG retrieval accuracy: FAISS top-K results contained noise. Added a CrossEncoder Reranker (BAAI/bge-reranker-base) for second-stage scoring — 20 candidates → top 5, all running locally on CPU.

Accomplishments that we're proud of

Designed and integrated the Planner node before the ReAct loop — not a library feature, but a custom state graph extension driven by real UX needs
Two-stage retrieval (FAISS coarse + Reranker fine) runs entirely locally — zero API cost
Full-stack, solo-built: from Python agent orchestration to React streaming UI, a complete demo-ready product

What we learned

Hand-writing LangGraph StateGraph beats create_agent() black boxes — full control over nodes, edges, and conditional transitions makes debugging and extending trivial
A Reranker is the highest-ROI optimization for RAG — 20 → 5 documents, significant accuracy gain, CPU-only
SSE streaming is a UX game-changer — the difference between waiting 30 seconds for a full response vs. watching it appear word by word
The real value isn't how many technologies you use — it's being able to articulate why you made each decision