Graphly

First Frontend UI
Second Frontend UI
Data scraping and augmentation notebook

Video Demo

Inspiration

Current search is broken. Search engines rely on string matching and keyword tuning, not true understanding. With LLMs and graphs, we can rebuild search from the ground up—semantic, contextual, and interactive. We're starting with a digital twin of the real world, beginning in Berkeley.

What it does

Graph is a world-scale knowledge engine where everything is a node. People, places, facts, and memories are embedded in a graph through LLMs and GraphRAG. Users can search the entire graph in natural language, visualize the connections, and even place themselves inside it. You can query a building, a person, or an event, and get a response that knows how it's all linked—contextually and relationally.

How we built it

We used:

Python and various LLMs to intensely augment the data
FastAPI to handle LLM requests and orchestrate backend services
GraphRAG to build context-aware graph data structures
Azure OpenAI and Claude to fetch embeddings and perform agentic workflows
React + TypeScript for a fast, smooth frontend
Gemini for multimodal agent capabilities
GraphQL to structure graph queries
Cloudflare + Azure Foundry for scalable hosting and caching

Each node in the graph holds a vector, powered by Azure’s text-embedding-3-large. Our server connects these embeddings to LLM outputs in a real-time interactive interface.

🤝 Sponsors + Products We Used

Azure OpenAI – GPT-4o + o4-mini for embedding, classification, and multi-modal understanding
Anthropic – Claude for rich, long-form scraping and agentic workflows
Gemini – Pro model used for image + text understanding, redundancy checking
Letta AI – Agent routing, orchestration, and multi-agent memory handling
Vapi – Voice interface (in progress) for real-time audio interactions
Fetch AI – Used for real-time planning agents and knowledge graph updates

Challenges we ran into

Graph complexity: Making vectorized nodes traversable in real time while keeping contextual awareness
LLM integration: Getting Claude, Gemini, and Azure GPT-4o to work smoothly in parallel agent workflows
Latency and cost: Balancing responsiveness with API rate limits and inference pricing
Frontend–graph sync: Visualizing real-time graph updates was tricky to optimize

Accomplishments that we're proud of

Built a full LLM-powered GraphRAG stack end-to-end in one weekend
Fully integrated Claude, GPT-4o, Gemini, and Fetch into agent workflows
Designed a real-time graph search UX where users can explore knowledge as a map
Used cutting-edge tools like Azure Foundry and Cloudflare Vectorize to scale intelligently

What we learned

Agent workflows are powerful but chaotic. Memory, planning, and tools need strict control
Vector databases and graph databases are very different beasts, but we merged them effectively
Building with multiple LLM providers (OpenAI, Anthropic, Google) unlocked new use cases and capabilities

What's next for Graph

Make it multiplayer: users can collaborate in a shared knowledge space
Zoom out: bring in more cities, institutions, and data sets
Decentralize: users can own their nodes and control what they share
Release a public search portal to explore the Berkeley knowledge graph in real time
Add interactive voice agent integration using Vapi or other tools