Tree Detective

Scan a QR tag on a campus tree and it wakes up, tells its true local history out loud, and shows its spot in 1930. Spot pests or damage? Your photo alerts the grounds team.

Comment

Inspiration

We walk past the same trees on campus every day without knowing a single thing about them. Some of them have stood beside NJIT buildings through decades of history. At the same time, the grounds crew can't be everywhere at once, and invasive pests like the spotted lanternfly are spreading across New Jersey. We wanted to make people actually notice the trees, and turn that attention into something useful.

What it does

Tree Detective turns campus into a storybook map. Every tagged tree starts out asleep. Scan the QR tag on its trunk and it wakes up on the map, and tells you its story out loud, in its own voice.

  • True local history: every story is written from sourced facts about the spot (Eberhardt Hall, Weston Hall, the school's founding), with the sources linked.
  • Time travel: drag a slider back through real aerial and satellite photos of campus, from New Jersey's 1930s survey to today, and compare "then" with "today".
  • Ask the tree: type or hold the mic to ask a tree a question. Campus history answers only come from its sourced facts.
  • 5 languages: English, Spanish, Chinese, Hindi and Gujarati, each with its own voice.
  • A game: points, a grove book of trees you've met, badges, and a walking route to the trees still asleep.
  • Help the grounds team: snap a photo and flag pests, damage or a dying tree. A problem only becomes confirmed once 5 different people report it with a photo, and it shows up on a dashboard the grounds crew can actually use, with a CSV export. Lanternfly reports link straight to New Jersey's official reporting tool.
  • Works over iMessage too: no app or website needed. Text a photo of the QR tag and the tree replies with a voice note.

How we built it

  • Frontend: plain HTML, CSS and JavaScript with Leaflet for the map, and a canvas for the fireflies and falling leaves.
  • Backend: Node.js and Express.
  • Voices: ElevenLabs gives each of the 9 tree personalities its own voice, with Azure Speech as the backup.
  • Writing and Q&A: Azure OpenAI, with Google Gemini as the backup. Facts are stored with vector embeddings in TigerData (TimescaleDB + pgvector), so a tree pulls up the most relevant sourced facts before it answers.
  • Translations: Azure Translator.
  • Photos: Pl@ntNet for species guesses, and Custom Vision or Gemini to check photos for pests and damage.
  • Data: Azure Table and Blob Storage for visits, reports and photos. TigerData hypertables power the week-by-week charts and "season firsts" on the grounds dashboard.
  • History: real 1930s aerial photography from the NJ Office of GIS, and Esri World Imagery Wayback for satellite photos.
  • iMessage: Photon.

Challenges we ran into

  • Keeping it honest. It's easy for AI to make up history. Every story is built only from sourced facts, unchecked stories are labelled as drafts, and AI-generated "then" pictures are clearly labelled as impressions, not real photos.
  • Preventing cheating. Anyone could type a tree's link at home, so each QR tag carries a secret key. Also, one person shouldn't be able to raise a false alarm, so a problem needs 5 different people with photos before it counts.
  • Real-world phones. Big iPhone HEIC photos broke uploads, so photos are shrunk to JPEGs on the phone first. GPS is often off, so the QR code alone counts.
  • Bad wifi. Venue and campus networks block or slow down outside servers, so we serve the map library from our own site and fall back to the browser's voice when a voice service doesn't answer.

Accomplishments that we're proud of

  • A tree that wakes up, glows and talks to you in under a few seconds of scanning.
  • Real 1930s aerial photos of the exact spot you're standing on.
  • Data the grounds team can actually use.

What we learned

How much history is hiding in plain sight on campus, and how to build features that rely on AI without letting it make things up.

What's next for Tree Detective

  • A "resolved" button for the grounds team, so everyone who reported a tree hears "you helped save it".
  • Sending season sightings to the USA National Phenology Network so every student becomes a citizen scientist.
  • Teams and leaderboards by dorm or club.
  • Tagging every tree on campus. ## Accomplishments that we're proud of

Built With

Share this project:

Updates

Submission history