I started off by trying to train a recurrent neural network to be a C compiler. Rather than waste days waiting for the model to train and missing the rest of the hackathon, I decided to use my newfound love of Gabor transforms to try and transform images into sound. It produces very interesting results: areas of the image with high color frequency (blue-ish) tend to produce high frequency sounds and vice versa, areas of the image with rapid changes in color tend to produce high frequency sounds and vice versa. In addition to single image to sound production, I also implemented real time audio from webcam video (which can serve as a sort of sensory substitution) and built a small web interface.

Built With

Share this project:
×

Updates