We got inspired by Morse code and smoke signals as a way to transfer information in large distances by only requiring line-of-sight between two parties.
What it does
The protocol we devised allows the encoding of any kind of data to a sequence of three different colors and back. Our example implementation is an Android app that automatically decodes text messages by pointing the device's camera at the flashing color sequence. Text can also be encoded, generating the flashing sequence for another device to see.
This could be used in emergency situations where other forms of communication may not be available, to communicate by just requiring line-of-sight to the flashing colors.
We developed a simulation in Unreal Engine, where the protocol is used by ships on sea to send messages to personnel on land and between each other. In the simulation, the ships communicate to avert a dangerous situation.
How we built it
The protocol was first designed on paper. We wanted it to be independent of timing issues, so we couldn't just map 0's and 1's to colours. A sequence of "11" and "111" with all 1's being a single colour would be indistinguishable from each other without precise timing measurements. This is why the colour sequence always changes the colour for every bit. The mapping of bits to colours dynamically changes based on the previously displayed colour, in a way that never generates repeating colours.
An encoder/decoder was then implemented for testing, and after that the Android prototype was created. It is written completely in Kotlin.
As an example, the ship's communication simulation was developed in Unreal Engine, with a working implementation of the protocol in the game engine.
Challenges we ran into
The protocol design process uncovered many issues, that were however quickly fixed, thanks to doing that early in the development process. The implementation was especially hard in Unreal Engine, where we had to use the blueprint system, which is like programming by connecting small blocks of actions. It definitely wasn't designed for data processing.
Accomplishments that I'm proud of
We devised a valid protocol, developed a working prototype with Android and a simulated use case in Unreal Engine. All of it worked in the end.
What we learned
Computer Vision is hard, and even a simple task like differentiating basic colours was an issue to overcome. We definitely appreciate libraries like the Microsoft's one for such tasks now, but it was also fun to try doing it ourselves.
What's next for Ternary Colorbridge
We could expand the Android app to use a real image processing library. We also though of using the protocol in puzzles like games and escape rooms, where it could prove to be an interesting game design tool.