What's Between...?

Turn curiosity into discovery. Scan everyday objects and see what connects them. Follow the most surprising connection with AI research agents, check the sources, and ask your own questions.

Comment

Inspiration

What do a banana and a shoe have in common? What about a computer and a skeleton?

The world is full of things we recognize but rarely question. We know their names and what we use them for, but we don’t always notice the materials, ideas, and discoveries that connect them.

I love getting lost in Wikipedia rabbit holes. I open an article to look up one thing, follow a link that catches my attention, and suddenly I’m reading about Japanese folklore and Polynesian architecture. For "What’s Between…?", I wanted to bring that experience into the physical world.

The problem

Search tools are useful when you know what you’re looking for. Exploring an unfamiliar subject is harder: you might not know the right terms, how it relates to other subjects, or where to begin.

For example, an orange and a water bottle might lead you to investigate packaging materials, but you might never think to search for polyethylene without first encountering that connection.

What it does

Scan two objects around you or upload a photo. Choose the Wikipedia articles that best match them, then explore a map of the topics and paths that connect them.

When a topic catches your attention, use Deep Research to ask a question, inspect the sources behind the answer, and follow up on what you find. You can save the investigation in your Research log and pick it up later with your findings and sources still attached.

In my demo (see images/video), an orange and a water bottle led me to polyethylene. I asked which types are used in reusable bottles, then followed up on why some are more flexible than others. The app suggested a next question I wouldn’t have thought to search for at the start: how does the plastic’s molecular structure affect its rigidity?

How I built it

I built the project by combining a React and TypeScript interface with a TypeScript API and a separate Python graph service.

From physical objects to Wikipedia

The Gemini API, with Gemini 3.1 Flash-Lite, recognizes objects in camera frames or uploaded images. Users then select or correct the article matches, keeping them in control of what they investigate. Gemini also helps explain connections and powers the research workflow’s reasoning.

The Wikipedia API retrieves article candidates and source passages for exploration and research.

The project builds on the open-source Six Degrees of Wikipedia (SDOW), created by Jacob Wenger, which supplies the foundation for exploring Wikipedia’s link graph. SDOW is credited under its MIT license. My contribution extends that foundation with camera-based interaction, object recognition and selection, AI explanations, research agents, and saved investigations.

Deep research with LangChain and Wolfram Alpha

The main workflow uses two LangChain Deep Agents running in parallel:

  • A historical investigator explores historical and cultural connections.
  • A design investigator explores mechanisms, functions, and design principles.

Both use Wikipedia tools to collect evidence and submit structured claims linked to that evidence.

A conditional research coordinator handles numerical questions or provides a fallback when the main investigation cannot finish. It can delegate calculations to a calculator subagent using Wolfram Alpha, or source verification to a fact-checker subagent. These agents run as needed; all five do not run for every question. Wolfram Alpha adds a computational resource to the exploration, allowing relevant numerical questions to be addressed through calculations rather than generated prose alone.

Three separate model stages (a citation verifier, skeptical reviewer, and final editor) assess proposed claims and assemble the response. These are review and synthesis stages rather than independent tool-using Deep Agents. Execution limits, or failures, can prevent every stage from completing.

LangSmith provides traces for inspecting model calls, tool use, investigator branches, and review stages.

Preserving investigations with MongoDB Atlas

MongoDB Atlas is an essential part of my project. It stores selected articles, connection context, versioned findings, supporting evidence, and research runs in the Research Log.

Follow-up context is loaded from saved server records, allowing an investigation to build on earlier findings. This preserves more than a final answer: it connects findings with their supporting evidence. Saved investigations are currently tied to the user’s browser, and research-run history is retained for 30 days.

Making it publicly accessible

The prototype is available at whatsbetween.us, using a custom domain I registered through GoDaddy and public HTTPS access through Cloudflare Tunnel. The application and graph service currently run on my computer, so availability depends on that computer and its services staying online.

Challenges I ran into

One challenge was deciding what the app could reasonably infer from a Wikipedia link. A path between articles gives users a lead to follow. Explaining why two objects are connected takes more evidence, so I built a research workflow that retrieves source passages and checks proposed explanations against them.

Deployment was another challenge. The Python graph service needs access to a 13.5 GB SQLite database, so publishing the frontend alone wouldn’t make pathfinding available. The prototype currently runs on my computer and is publicly accessible through Cloudflare Tunnel.

Accomplishments that I’m proud of

I’m proud I built the full experience on my own, from scanning two objects to investigating a connection and saving the findings and sources. It was a different experience from my first hackathon, in which I was on a team. But nevertheless, I had an excellent time and enjoyed the challenge!

What I learned

Building What’s Between…? reinforced my interest in how interface design shapes exploration. Starting with a camera lets people choose something they can see before deciding what they want to ask. I tested the interface with other hackers to see how they moved from selecting objects to exploring the map and asking follow-up questions.

I also learned to work with LangChain and MongoDB Atlas. The LangChain presentation at HackPrinceton helped me learn how to coordinate research agents, and MongoDB University helped me learn how to store investigations with their supporting evidence.

What’s next

My first priority is to host the Python graph service and its database on a server with persistent storage so the site can stay online independently of my computer. I estimate hosting will cost about $24 USD per month, plus API usage.

I want to see which connections people open, how often they ask follow-up questions, and whether they return to saved investigations. I’d pair those usage patterns with user feedback to understand which explanations need more context and what makes a topic worth exploring further.

Longer term, I’d like to add research sources beyond Wikipedia, let users access saved investigations through an account, and make it easier to see how findings changed as they asked new questions.

Built With

Share this project:

Updates

Submission history