Modern social activism needs tools to excite new generations.

With the emergence of mixed reality technologies, protesters have access to new mediums to expand upon what normal signage can do. These technology-enabled signs allow for much larger visual spaces with dynamic content. We want to take advantage of and give regular people access to these advanced technologies without the need to understand how they work.

A chatbot that generates mixed reality protest signs.

Our facebook messenger chatbot connects people to facebook pages or events. If you're not sure what social activism to be involved in telling us what you are passionate about. We will connect you with events and pages that you might be interested in. Once you have an event to attend our chatbot will walk you through making a custom mixed reality sign. Simply upload your image and we will give the user a QR code to print out as the sign. Other Protesters can open your mixed reality sign via the printed QR code. Your mixed reality sign will be mapped to the target QR code turning your small sign into a dynamic experience.

Building the app

We use Facebook messenger to manage our user interface and DialogFlow to understand and respond to users. We spent a fair amount of time understanding DialogFlow to help us maximize our user experience. The Mixed Reality platform we used is SparkAR because it is user-friendly and allows us to do both target tracking and to display different images based on the User.

Our biggest Challenges

We spent a bunch of time trying using DialogFlows hosting system as our backend for our application. It was initially very successful allowing us to quickly deploy a chatbot which took our data. Unfortunately, it also had some technical challenges associated with the platform. specifically, we had trouble storing the custom URLs for files and applications. With some help and a great deal of careful evaluation of the documentation, we were able to overcome our challenges and served a real poster to our user.

Accomplishments that we're proud of

Using SparkAR we were able to quickly build dynamic posters that image track the printed QR code. This allowed us to have our mixed reality posters working within an hour. We are also very happy with how our identity protection filter works. The filter uses face tracking to cover up to five faces from the camera's view. This way if a protestor doesn't want his or her sign associated with their face the identity filter blocks them. The QR code is generated by an API which means its totally possible to resize it to any size poster.

What we learned

We learned how to use the SparkAR platform to load dynamic data from our post API. This allows the same Two Filters to be applied to multiple users Posters leading to a different result. We simply serve the filter with different data based on when and who is requesting the filter. We also did a deep dive into how DialogFlow allows chatbots to be built using Machine learning to respond dynamically to requests

What's next for SociAR

We really want to finish building a more advanced backend so we can anonymously link our users to their specific posters with more accuracy. Our current solution relies to heavily on time and Users could potentially receive other people posters if we get too many requests. We also want to build a Create-react-app which would walk people through designing their own protest poster and generate a static image from the resulting data. A big motivation for us is connecting activists and their posters to the groups they represent. We want to revamp the Community recommendation systems so people who want to protest have resources to do so.

Built With

Share this project: