Tree Walk Guide

• 

  Companion website landing page

• 

  Settings

• 

  Info of the walk route stop

• 

  The stop red pins with the green geo fence pole markers

Inspiration

The CSU Fresno campus contains a lot of wonderful trees. The existing tree walk route was decades old, many trees were relocated or replaced and the information was in a PDF format. Students redesigned a new tree walk route which focuses on drought tolerant, native, and indigenous trees (still work in progress). The Tree Walk Guide project submitted here aims to create interactive tools to support the redesigned walk the most modern way possible.

What it does

The Tree Walk Guide is an augmented reality based mobile application helper. With the help of Geospatial API the visitors can locate the target trees along the route, easily access information about the walk stops. Even more information is accessible on a companion website which leverages 2D Google Maps API and contains extra data, links and 360 photos of the stops.

The guide application also offers gamification: the user unlocks Game services achievements when the walk stop pin is clicked. On top of that anyone can gather extra points by virtually watering surrounding trees at the stops: clicking the shower icon the trees on of the current camera snapshot are identified and clicking on any triggers a virtual watering along with extra experience points earned.

I hope that the on one hand the information cards and on the other hand the gamification can capture the attention of all types of users and demographics. Accessibility is top priority for the project so both the companion website and the mobile application is bi-lingual (English and Spanish). The augmented reality app is also able to voice narrate messages and information if the feature is turned on.

How we built it

• The augmented reality applications most important pillars are the ARCore and the Goespatial API. That allows the placement of the tree walk route stops by GPS coordinates into the real world space. Each stop also has a geo-fenced rectangular area marked by four posts where the user can perform the virtual watering.
• Semantics API helps to identify potential trees the player can get experience points for by virtual watering. This is an EAP feature and only available on Android right now, so the project currently is Android only.
• The gamification is based on the Google Play Game Services API.
• The application dynamically downloads the current walk route from the companion website. Therefore a new app release won't be needed if the route changes: the website meta-data can be modified and the mobile app should pick that up. The mobile application technically is a Jekyll (Ruby on Rails) blog and the meta data is human readable: the non localizable portion is in yaml and the localizable texts are in markup front-end-matter (yaml style) or markup / light HTML.

Challenges we ran into

• The ARCore engine was not able to handle the creation of all the anchors (even if they are not visible), this would be 24 map pins and 24 * 4 posts, so 120 Earth anchors. Therefore I refactored the code to only keep the current target stop and the anchors and the main anchor of the next stops created at a time.
• I had trouble developing a shader for the Semantic API where I'd keep only the "tree" semantic pixels as a stencil and blend the camera texture in a way that it'd tint the tree areas lightly with a color. I completely redesigned the watering workflow, and will revisit the shader.
• Since the walk is specific to the CSU Fresno campus I needed to develop a local debug mode.

Accomplishments that we're proud of

• Despite how hard was it to

What we learned

• Geospatial API and Semantics API makes features possible which were hard (image semantic analysis) or impossible (Geospatial Terrain Anchors) before
• First year experience students will use the website and the app later this semester (a few already contributed issues to the website repository). I'm very excited about the feedback I'll get and the future of the project
• There's still a ton of work ahead of us to enhance, develop, and refactor the source

What's next for Tree Walk Guide

• Integrate fun facts about the walk route stops (the students are currently assembling the data), this will require data format / meta data change
• Enhance the features I could not tackle with the minimum viable product, such as the Semantics shader or UX / usability issues
• Add new languages, for example Hmong which is a prominent minority in the valley (latinos are the biggest though)
• Gather feedback from the students and the professors to cater to their exact needs

Built With

• android
• games
• geospatial
• html
• javascript
• jekyll
• kotlin
• terrain