Wherever we go in the world, in train stations, in metros, in shopping complexes, in shops; we see digital signage or digital screens showing advertisements. One of the biggest problems for these digital signage advertising is to target it to audiences and to measure actually how many people have seen these advertisements and how their responses are to the advertisements. So we though of finding a solution to this issue during LauzHack using Microsoft Cognitive Services APIs where we can combine human elements to the apps we are creating.
What it does
Our system IntelliAd is an intelligent digital signage software solution. IntelliAd can be used in any digital signage and a camera should be placed on top of the digital display facing the audience. IntelliAd software analyses the crowd in-front using Microsoft Cognitive services and shows advertisements according to the crowd in-front.
Example: Let's say when there are more males in-front of the advertising display, it would automatically play an advertisement targeted for men like Gillete. When there are more girls in-front, it would play something like Ponds.
IntelliAd can guess the age of the people in-front and filter the content as well. Let's say when there are more young people it would play an advertisement targeted for a young audience. When there are small children infront, it would play only ads suitable for small children. IntelliAd is smart by every means.
What explained above is just one side of the application. IntelliAd does more. It gives an web app interface for any advertiser to come and upload an advertisement while selecting the targeting options as well. And while his advertisement is being played, he can view a analytics page with how the advertisement is performing. How many views it got, how many males so it and so on. This would help the advertiser to customize the advertising campaign and to gain a real insight on how well the advertisement is performing.
How I built it
The app consists of 3 main parts. I will describe each of it part by part.
1. Server app for Advertisement displaying This will be what normal public or the viewers would see. They would just see Video and Image advertisements being played in-front of them. This is a Windows presentation foundation app and connects with the online database through sockets to check if new advertisements are available.
2. Client app for image processing with the feed from camera (Gender, Age, Emotion) This app communicates with the server app and at the same time monitors the crowd in-front (image processing) with the feed from the camera. This runs as an app in the background. This analyses the crowd and sends message to the Server app on which advertisement to show where Server app shows ads and send statistics to the online database at the same time.
3. Web app for advertisement uploading People can upload a video or image advertisement. They can select their target audiences as well. The uploaded advertisements will be automatically displayed in the server application. (using sockets) Further advertisers can have analytics on how their advertisement is performing.
Challenges I ran into
Microsoft Cognitive services posed a lot of challenged from the beginning :D For this kind of an application what would be most suitable would be an SDK which can do image processing offline. But since we have cognitive services, we had to run a separate thread in the client app which sends a picture to cognitive services each second and get the response. According to the response, client app communicates with the server app to change advertisements. Implementing this took a lot of time. Implementing the socket between the online database and the server app too was a bit of a challenge.
Accomplishments that I'm proud of
We are a two man team and creating the working demo it self is a good achievement I think. Actually we came to the hackathon from Munich, Germany and spent a lot of time travelling here. We have been awake 2 nights straight for the hack and we are proud of what we have built :)
What I learned
Learnt a lot about cognitive services and REST APIs. Though it seems easy to use, complexity comes when we combine the API calls with all the other functionalities and features. Cognitive services API about Gender recognition is just for images. But what we have is a video stream. Hence we had to develop a way to take frames of the video stream at certain intervals and use them with cognitive services. Learnt a lot about Microsoft Azure as well.
What's next for IntelliAd
We are not sure. Let's see what future holds for IntelliAd. We think the idea is really cool and it has the potential to disrupt the advertising industry in the world.