We wanted a simple way to back up our music and movies. Cloud storage providers, such as Dropbox, only provide a few gigabytes of storage space; built for text files and images, but it lacks the capacity for large multimedia files. Storage space is expensive and media files are difficult to compress. But what if there was a way to save space when storing media files for thousands of users? And what if this savings could be transferred onto the user to provide them with more storage all while saving bandwidth?
What it does
MediaBin takes file deduplication to the next level by way of a process called Audio FIngerprinting. This is the same process by which Shazaam can identify songs. MediaBin generates audio fingerprints for any music or movies that are uploaded. It then compares the fingerprint against a database. If a matching file is found, the user is granted access to that file instead of having to upload their own. If not, the file is uploaded. So if 500 people upload a copy of The Shawshank Redemption, only one copy is actually stored. All of the other users simply have access to that copy. When a user "deletes" a file, their access to the file is simply removed. On a small scale, the space savings are marginal but on a larger scale, think a service like Dropbox that has tens of thousands of users, this deduplication can be monumental. MediaBin users can then reap the benefits of these savings. Instead of being able to upload 5 gigabytes, they could upload (for example) 40 hours of video and 100 hours of audio.
How I built it
The backbone of the project was build with Node.js, with a client-side web app to go along with it. With that being said, the project was designed such that the web app could easily be converted to a mobile or native application. The application is hosted on AWS. Amazon EC2 provides the computing power, DynamoDB provides the database service, the files are stored in Amazon S3, and Amazon Cloudfront is able to provide streaming services for the application
Challenges we ran into
Handling upload of large files proved to be very challenging, as did setting up AWS authentication credentials, authenticated users, etc.
Accomplishments that we're proud of
Each of our team members were equally involved in the implementation of the front end, client side and server side development, regardless of experience level.
What we've learned
Simplest problems can be the most complicated ones. In a hackathon, the more planning and awareness you have of the tasks to be accomplished, the more fun you'll have making the product. When you assume simple tasks, like uploading files to a server, are moved to the last minute, the rabbit holes of a project become overwhelmingly complex. Fortunately, we all successfully worked together without merge conflicts through our quick awareness and planning.
What's next for MediaBits
After today, we'll be working on user authentication, streaming media, quality analysis of media files, favoriting files.