What it does
A VR gallery environment where photos are displayed on the walls with automatically generated captions based on image content using the Cognitive Services API as well as information extracted based on EXIF data such as location and time - captions are also narrated to the user.
How we built it
Unity VR app talking to python server on AWS that handles caption construction and serves the image, caption and audio caption which are then played in the app.
Challenges we ran into
Server setup on AWS Getting Oculus dev environment working
Accomplishments that we're proud of
Working VR app in <24 hours having never done and VR dev before Captions that (we believe) are better than those provided directly by Microsoft cognitive services API.
What we learned
VR application design How to use Microsoft Cognitive Services API How Unity works
What's next for DJP's VR AI Picture Zone
Integration with android's native photo service to provide user's own photos in the gallery rather than the current placeholder images.