Inspiration
Everyone spends hours on Facebook Messenger from chatting, to collaborating on work, to sending links, and to sharing photos. What if there was a way to super-charge your Facebook Messenger and make these tasks even easier and more integrated into the Facebook Messenger experience.
What it does
We implemented a bunch of new features into Facebook Messenger that will take your Facebook usage to the next level. For better collaboration we created an interactive canvas inside a chat. Users will be able to whiteboard their ideas with their friends without ever having to leave Facebook.
We can stream local videos from one user to the other through Facebook allowing for instant sharing without the manual tasks of uploading it to Youtube or Drive before hand.
Using the Chrome WebAudio API we have integrated voice commands into Facebook Messenger. At the tap of a button, users can speak to Facebook and have their text displayed on the screen.
For those users that love the sharing aspect of Facebook, we created tools that allow you to share content much faster. Using the commands /giphy and /youtube, users can search through Giphy and Youtube for content that represents what they are feeling when words can't.
How We built it
We attached an event listener to every element in the html hierarchy and used the react-id to get message text. From here we just associated specific message commands delimited by " / " to get the commands.
We inserted custom css by injecting it with content scripts by reverse engineering the hierarchical and reactive structure of Facebook's Messenger page. This includes the canvas, the extra button functionalities, as well as anything we pushed directly into the chat.
For our canvas, we used Socket.IO to create unique rooms using the Facebook IDs of users that we received from the self-referencing profile account: facebook.com/profile.php?=73322363. From here we had access to our own vanity, our user-id, as well as all of our recent chat history which we could scrape for information about other users. We went through the page to find the ids and used them to add elements to the messenger-thread.
Challenges We ran into
As we were modifying elements on Facebook's Messenger.com, it was difficult to see past the obfuscation created by ReactJS - it required a lot of trial-and-error testing and reverse-engineering. Sending http and websocket data within Facebook also proved difficult, as we were required to use SSL and TLS-enabled sockets, and had to deal with CORS and other cross-origin-related headers.
Deploying on Azure also proved difficult originally: we experienced problem with a compiler dependency of a concurrency library for Python/Flask. We ultimately switched to an Ubuntu Box and successfully configured NGINX to handle both SSL and TLS sockets.
We had an additional RESTful server deployed to Heroku. Early Sunday morning, however, several Heroku data centers experienced significant problems, preventing us from deploying: we ultimately transitioned to a VM on the Google Cloud Platform, which solved our problems.
Accomplishments that I'm proud of
We are very proud of how much we finished because of the huge amount of obfuscation that Facebook has on their messenger page. We were still able to integrate a lot of APIs including Youtube, Giphy, Google Microphone, and Instagram, without the need for external authentication or inconvenience to the user.
Log in or sign up for Devpost to join the conversation.