Great hype around virtual and augmented reality. Computing power of today's mobile devices. An old idea we had in mind.
What it does
Render the outside view from an airplane to anyone onboard with an android device (other platforms planned), given that the airplane is fitted with cameras with a combined full 360-degree coverage of the outside.
Live demo of our prototype at Junction hackathon.
How we built it
We have a prototype using an NVIDIA Jetson TK1 development kit and four cheap USB video cameras pointed in different directions. The client runs on a phone/tablet and changes its view according to the device's pose: it's a "window" for all orientations.
Challenges we ran into
Magic command line switches for ffmpeg and gstreamer. Video format compability issues. Perspective projection math for building the view from individual cameras. Stupid front-end bugs we made because of staying up a whole night without sleep.
Accomplishments that we're proud of
The idea has a lot of potential and it is very simple to implement.
What we learned
Mainly ffmpeg and gstreamer usage; others we were already aware of. Studied the state of current mobile APIs for streaming video (which is not quite there yet without issues).
What's next for virtual window seat
Build an actual product out of a prototype hacked together in two nights, and get it installed on every commercial airplane?
The prototype just shows the camera views when rotated, but it'd be simple to add features such as taking photos and mapping country borders or other augmented reality items.
The product would need installation of synchronizable video cameras on an airplane for all directions for full coverage, and a small server for sending the video feed to any connected clients. The commercially available 360-view cameras such as Nokia Ozo or the spherical GoPro are not immediately feasible, since they consist of just one integrated device and thus one direction would be blocked by the airplane body. (And they'd create unnecessary drag; cameras should be installed right on the plane surface.)
Post-party thoughts in Finnish: http://sooda.dy.fi/2015/11/10/virtuaalinen-ikkunapaikka-lentsikoihin/