We knew from the offset that we wanted to do a hardware-based hack. We began with the several sensors available to us that could interface with the Qualcomm Dragonboard. Of particular interest were vibrational sensors. After considering our options, we settled on using a sound sensor to deduce structural integrity and composition of materials. We decided that a great way to test this technique would be with watermelons. After finding a 2004 paper that empirically demonstrated a relationship between vibrational modes and watermelon quality, we realized that this project would be feasible. While the authors of the paper conducted acoustical analyses, we decided, based on the time available to us and because we felt it was more aligned with the spirit of HackTJ, to use a neural network to differentiate watermelon quality.

What It Does

We prototyped a portable watermelon quality sensing device that delivers a sound producing impulse through a servo and records the corresponding sound. The recorded sound is fed into a trained neural network that determines whether or not the sound corresponds to a higher quality watermelon.

How We Built It

First, we opened Qualcomm and tried to boot it up. After figuring out how to use the DragonBoard, we began researching how to use the electrical components while also making the neural network. While this was happening, we also brainstormed a procedure to gather data and prototyped an arm that would be attached to the servo, which we later 3D printed. Initially, we used a spy camera and microphone, but because it didn’t work, we switched to using a webcam. Afterward, we collected data using the sensor, fed the data into the neural network to train it, and wired up the actual apparatus.

Challenges and Accomplishments

  • The microphone that came with spy camera did not function correctly; the signal obtained from the microphone was not clean, as a result, the .wav file was full of beeping noises that rendered data useless
  • The 96Boards that came with Qualcomm DragonBoard was causing kernel panics on the DragonBoard; we solved this by giving the 96boards an additional USB power supply
  • Wanted to do a project on watermelons; did not have watermelons; solved by getting watermelons 3D printed arm did not fit onto servo, filed down components so that they fit

  • Overcoming hardware issues by employing gracious professionalism among peers: borrowed camera due to nonfunctional camera

  • Neural network arrays did not align up: kept fiddling till worked

  • Successfully collected data

  • Artificially created a controlled simulation for bad watermelon

  • Servo was not strong enough so we used knuckles to apply consistent sound generating impulse to the watermelon

  • Came up with a fun and challenging project idea

    What We Learned

  • Hardware problems that can’t be fixed

  • Gathering and processing audio data from microphones

  • Learned that python code can be super annoying

  • Using GitHub, transferring data

  • Difficult to roll watermelons

  • Learning how to use Qualcomm board

  • Library-specific sensors to DragonBoard

  • Important to be patient when debugging

    What's Next For IoT HackTJ Project

  • Upgrading hammer/servo

  • A more comprehensive training set

  • Expanding fruit type

  • Creating a more portable device

Built With

+ 18 more
Share this project: