FrenchFry

Prize Categories

  • #### Vermeer Topic: Road to Autonomy: Best Machine Vision App

Prize: MP Select Mini 3D Printer V2

  • #### Rockwell Collins Topic: Best Use of Communications Over Devices

Prize: Drones

  • #### TradeBot Topic: Best Newbie Hardware Hack

Prize: Raspberry Pis

  • #### MLH Prizes Topic: Best IoT Hack Using A Qualcomm Device

Prize: Qualcomm Dragonboard 410c

Team Members

Thomas Han

Colby McKinley

I worked with Android Studio to develop an Android application that pairs an Android phone with another device and transfers data between the two devices. The phone is the slave and the Qualcomm Dragonboard 410c is the master. When an object is correctly detected, a notification is generated to alert the user the camera has found something of interest.

Alex Powers

I learned how to interact with the JeVois and load predefined algorithms onto the camera. I learned to ride a unicycle. I learned that I have a lot to learn about hardware and also everything else in the world. I read the entire user documentation of JeVois and am now fully prepared to try again at my next hackathon.

Kevin Blicharski

I wrote the Python code using the pyserial library to interface with the JeVois camera's USB serial connection. This allowed us to transmit the data from the camera into the Dragonboard. I spent most of my time learning how the Dragonboard operates and debugging bizarre configuration issues.

Inspiration

FrenchFry is a hardware and software hack that was heavily inspired by our curiosity and desire to learn about new, cutting-edge technologies. We initially had no intentions of doing a hardware hack, but this quickly changes after stopping by the Vermeer table and seeing what the JeVois camera was capable of. None of our group members had any extensive experience with hardware, but we were motivated by our interest in computer vision, machine learning, and IoT applications. We wanted to expand our knowledge in these fields and get hands-on experience and the JeVois camera and Qualcomm Dragonboard 410c were great options.

Our motivation in developing FrenchFry was to create an IoT, computer vision application to assist the visually impaired in an otherwise mundane activity: crossing the road. Although we could not finish the project in time to implement it on a wearable device, our work can serve as a proof-of-concept to be improved upon in future iterations.

What it does

  • JeVois camera observes its surroundings and detects when it is safe to cross the street
  • It does this by detecting when cars to the left and to the front of the viewport are no longer moving
  • The JeVois then feeds a filtered data stream into the Qualcomm Dragonboard 410c
  • This data is then transmitted via Bluetooth to a Bluetooth-enabled phone running our Android app
  • This app can determine when it is safe to cross the street through a push notification/vibration on an Android application
  • In the future, this could be expanded with screen reading capabilities
  • We needed the Dragonboard for the data processing and WiFi and Bluetooth capabilities

How we built it

  • Node.js for the serial port transmission code
  • Qualcomm Dragonboard
  • JeVois
  • Android using Java8

Challenges we ran into

  • New experience, none of us knew anything about the Qualcomm dragonboard, JeVois, Debian Linux, serial ports, or hardware in general

  • The dragonboard could not recognize any USB connected devices so we could not directly interface with it

  • Instead, we had to use the intermediary of the laptop

  • None of us knew anything about anything

Accomplishments that we're proud of

  • Actually following through and finishing the project

What we learned

  • How the fuck hardware works (not really)
  • That hardware is really complicated
  • That Bluetooth is really complicated
  • That minor version differences in an operating system can assassinate an entire hackathon project

What's next for FrenchFry

-Putting it in a beret

  • Having expanded functionality for object/image detection
  • Fixing the problem of the dragonboard not detecting the device dev path

Built With

Share this project:
×

Updates