- #### Vermeer Topic: Road to Autonomy: Best Machine Vision App
Prize: MP Select Mini 3D Printer V2
- #### Rockwell Collins Topic: Best Use of Communications Over Devices
- #### TradeBot Topic: Best Newbie Hardware Hack
Prize: Raspberry Pis
- #### MLH Prizes Topic: Best IoT Hack Using A Qualcomm Device
Prize: Qualcomm Dragonboard 410c
I worked with Android Studio to develop an Android application that pairs an Android phone with another device and transfers data between the two devices. The phone is the slave and the Qualcomm Dragonboard 410c is the master. When an object is correctly detected, a notification is generated to alert the user the camera has found something of interest.
I learned how to interact with the JeVois and load predefined algorithms onto the camera. I learned to ride a unicycle. I learned that I have a lot to learn about hardware and also everything else in the world. I read the entire user documentation of JeVois and am now fully prepared to try again at my next hackathon.
I wrote the Python code using the pyserial library to interface with the JeVois camera's USB serial connection. This allowed us to transmit the data from the camera into the Dragonboard. I spent most of my time learning how the Dragonboard operates and debugging bizarre configuration issues.
FrenchFry is a hardware and software hack that was heavily inspired by our curiosity and desire to learn about new, cutting-edge technologies. We initially had no intentions of doing a hardware hack, but this quickly changes after stopping by the Vermeer table and seeing what the JeVois camera was capable of. None of our group members had any extensive experience with hardware, but we were motivated by our interest in computer vision, machine learning, and IoT applications. We wanted to expand our knowledge in these fields and get hands-on experience and the JeVois camera and Qualcomm Dragonboard 410c were great options.
Our motivation in developing FrenchFry was to create an IoT, computer vision application to assist the visually impaired in an otherwise mundane activity: crossing the road. Although we could not finish the project in time to implement it on a wearable device, our work can serve as a proof-of-concept to be improved upon in future iterations.
What it does
- JeVois camera observes its surroundings and detects when it is safe to cross the street
- It does this by detecting when cars to the left and to the front of the viewport are no longer moving
- The JeVois then feeds a filtered data stream into the Qualcomm Dragonboard 410c
- This data is then transmitted via Bluetooth to a Bluetooth-enabled phone running our Android app
- This app can determine when it is safe to cross the street through a push notification/vibration on an Android application
- In the future, this could be expanded with screen reading capabilities
- We needed the Dragonboard for the data processing and WiFi and Bluetooth capabilities
How we built it
- Node.js for the serial port transmission code
- Qualcomm Dragonboard
- Android using Java8
Challenges we ran into
New experience, none of us knew anything about the Qualcomm dragonboard, JeVois, Debian Linux, serial ports, or hardware in general
The dragonboard could not recognize any USB connected devices so we could not directly interface with it
Instead, we had to use the intermediary of the laptop
None of us knew anything about anything
Accomplishments that we're proud of
- Actually following through and finishing the project
What we learned
- How the fuck hardware works (not really)
- That hardware is really complicated
- That Bluetooth is really complicated
- That minor version differences in an operating system can assassinate an entire hackathon project
What's next for FrenchFry
-Putting it in a beret
- Having expanded functionality for object/image detection
- Fixing the problem of the dragonboard not detecting the device dev path