Our goal was to create an API for connecting different mobile devices of all types to one big screen without huge afford for the users.
Our product is web-based, which means you do not have to install any software on your device to use it (you simply need a web browser). Therefore we used Websockets, so you just have to type in the server’s IP and can connect.
Via our server’s webcam we take pictures of the different devices that should be connected. We want to connect the device's IPs with their actual position - so we know which pixels of which devices cover which areas. Then we can splice videos and images up and send them to the device's screens.
At this point some image processing was required. In order to separate the background from the foreground, we grayscaled our picture and programmed some kind of the Otsu algorithm to get the optimal gray-scale value. All pixels that are darker, are drawn black, and the others are colored white.
The next step was to implement a fill algorithm to find connected areas. Then we wrote our own class to detect the different rectangles (devices). Our projects recognizes devices that are places horizontally, vertically and even rotated ones.
As soon as all devices are recognized different parts of a huge picture are send to the devices and rotated with the same degree the devices are rotated.
Our aim is an software that can be used for example in a football stadium. Nearly everyone has his own mobile device in his pocket and if everybody keeps it up and gets some piece of a huge picture or video it will looks quite impressing and amazing.