1. The fitness industry has attracted a lot of consumers.
  2. Fitness watches, bands, and trackers make consumers easy to track their activity and burnt calories/energy.
  3. But energy intake is hard to track. Currently, users enter data manually.
    • time-consuming
    • hard to estimate
    • no useful functions
    • ultimately discourage from using

What it does

  • Allowing people to get a quick access to know the foods' nutrition.
  • Providing a different view of displaying information as usual.

Target users

  • Individuals who willing to spend little time for a brief nutrition overview.
  • Athletes, professional trainers who track their nutrition intake seriously.

How I built it


  • Big Data: gather product images, nutrition facts
  • Machine Learning: calculate cascade classifiers, recognize objects
  • Augmented Reality (AR): display / show nutrition facts
  • Object Tracking
  • React Native: deploy web app to both iOS and Android platforms


We are really collaborative!

  • Wangyue Wang: Front-end design and implementation, packing web app to different platforms
  • Xiaofan Ni: Object tracking and AR part
  • Tianyu Wang: Building objects' cascade classifiers and gathering nutrition information

Open Source Projects We Used

  • tracking.js
  • react-native
  • OpenCV

Challenges I ran into

Training the object model is some sort of difficult in the 48 hours.

Accomplishments that I'm proud of

But we've done!

What's next for Nutrition Lens

  • Nutrition overview
  • Item size detection for better accuracy.
  • Reorder like Amazon Dash (re-order)
  • Advertisers (recommend competitive products)
  • Sponsors (recommend sponsored products)
  • Connect w/ apps (API)
  • Reminder

Built With

Share this project: