What it does

A VR gallery environment where photos are displayed on the walls with automatically generated captions based on image content using the Cognitive Services API as well as information extracted based on EXIF data such as location and time - captions are also narrated to the user.

How we built it

Unity VR app talking to python server on AWS that handles caption construction and serves the image, caption and audio caption which are then played in the app.

Challenges we ran into

Server setup on AWS Getting Oculus dev environment working

Accomplishments that we're proud of

Working VR app in <24 hours having never done and VR dev before Captions that (we believe) are better than those provided directly by Microsoft cognitive services API.

What we learned

VR application design How to use Microsoft Cognitive Services API How Unity works

What's next for DJP's VR AI Picture Zone

Integration with android's native photo service to provide user's own photos in the gallery rather than the current placeholder images.

Share this project: