Sticker Bot - Chatbot for making, sharing user-made stickers

A Messenger bot for creating and sharing user-made stickers. Sticker Bot makes it possible for Messenger users to create, edit, manage and share stickers.

Comment 2

Inspiration

A picture they say contains a thousand words. Most times, responding with a sticker is the perfect reply that conveys the sender's unexplainable expression to a chat message. It spices things up and also an excellent medium to send messages you can't tell with words, for example; memes, surprise expressions, funny moments.

Interestingly, Messenger sticker feature is limited. As I write Messenger out of the box does not support user-made stickers, only the predefined ones from Facebook compared to other popular chat apps like Whatsapp and Telegram that supports user-made Stickers. We decided we could solve this somehow; Our solution was to build a chatbot that brings this feature to Messenger.

What it does

Sticker Bot is a Messenger chatbot that enables it, users, to make, manage and share user-made stickers. Under the hood, it uses Machine Learning (ML) to make stickers from the picture you send to it. Also provides its users with a sticker editor to edit their Sticker. With the editor, users can add texts, emojis and also create animated Stickers.

"Any product that needs a manual to work is broken" (Elon Musk) In other to make the bot easy to use and very engaging, we heavily use Facebook Quick Replies:

  • To suggest what the user wants to do.
  • To give users options.
  • To confirm an action, for example, deleting a Sticker. Ultimately, using Quick replies, we were able to make the bot very simple by eliminating the need for typing text commands. We believe text commands are only for savvy power users. If a user likes to use text commands for example, "edit", "delete", "manage", help e.t.c the bot understands and will reply as well.

How we built it

The Messenger bot, developed with Node.JS, Facebook Send API and Quick replies, Goggle Cloud storage to backup stickers and MongoDB Atlas to keep records of Stickers made by users.

The Sticker making module is a Python server which provides a deep learning network for Image Segmentation. It extracts objects from a picture, then uses OpenCV to add sticker effects to the extracted images. The resulting Stickers returned to the user as an attachment.

We also build a Sticker editor. We paid attention to details; the editor is a replica of Messenger's in-app picture editor. We did this to give our users a familiar experience. The editor is developed with ReactJS and HTML5 Canvas API to provide users with a cross-platform facility to edit their Stickers. The editor is deployed on the Web so that our mobile users can use Messenger's in-app browser to launch the editor. Also, users using Messenger on a Web browser can equally use the editor to edit their stickers right there in a browser tab.

Sticker Bot relies on Messenger's share feature for sharing stickers. It allows users to share them with other users in their contact list.

Challenges we ran into

Some of the challenges we ran into were mostly optimization issues. We were able to solve them with extra effort. An example is when we wanted our Sticker Creator module to extract up to four objects in an image and be able to do this as fast as possible. We were able to achieve this by fine-tuning our model. As a result, up to four stickers stencilled out of a picture in 30 seconds.

What's next for Sticker Bot

We hope to make Sticker bot into a fully-fledged product after the hack-a-ton. Ultimately, we wish to continue to run the service.

Built With

+ 2 more
Share this project:

Updates