I Desire Apple Pie

Inspiration

Since the beginning of time, mankind has looked to the stars for inspiration, often leading to some of the greatest scientific and mathematical discoveries to date. Kepler, Newton, Laplace, Lagrange, and Einstein are all commonplace names in every classroom, but many would be surprised to know that their motivation for making some of the greatest advances in science and technology were born out of a desire to understand our universe and the laws that govern it.

Soon, mankind will begin the efforts of colonizing other planets, and the thirst for discovery will be quenched once again. This project was born out of curiosity for what life will be like once humanity makes that step. What will we see when we look up from other planets? We wanted a way to visualize 3D space and explore the stars and constellations in a way they had not been seen before. This, coupled with the VandyHacks theme to create a work of art, led us to build the first Martian art Gallery. As Carl Sagan once said: “If you wish to make an apple pie from scratch, you must first invent the universe”.

What it does

A Virtual Reality experience that transports the user to an art exhibition on the Martian surface. Exhibits range from sculpture, to monuments, to 3D constellation models. The sky is made with real-time star mapping that represents what the actual night sky will look like when future Martians look to the stars. The size and brightness of each star directly correlates with the distance the star is from our Solar System and its Flux value. The simulation features narration by a neural network version of Carl Sagan, who teaches the user about a variety of concepts that will be crucial to the future of humanity living outside of the Earth system, as well as offbeat facts about some of the celestial bodies that make up many of the worlds favorite constellations.

How we built it

The Virtual Reality simulation was built using Unreal Engine and Oculus hardware, as well as real-time voice synthesis with small data sets, image hyper resolution neural network trained primarily on 2D data sets used to enhance textures provided by NASA and Poliigon, and style transfer with neutral network to make neat art.

The star mapping was created by utilizing SPICE Kernels, ephemeris data which determines the precise location of celestial bodies in space depending on a user-defined epoch and reference frame; and Python programming. Python packages used include Spiceypy, Skyfield, Numpy, Matplotlib, Pandas, and Bpy to interface the code with Blender. SPICE Kernels used include de430, naif0008, and the Hipparcos star catalog, among many others.

Challenges we ran into

Simultaneously running out of VRAM, RAM, GPU, and CPU power; and two complete computer crashes which resulted in file corruption. Additionally, migrating assets from Blender to Unreal Engine and finding ways to interface between all the different project segments proved to be much more difficult than previously anticipated.

On the celestial mechanics side, determining the necessary SPICE Kernels required to view bodies from the Mars Barycenter, proper coordinate transformations to convert from the spherical coordinates used in the celestial models to the cartesian coordinates used by Blender and Unreal Engine was tricky to properly align, and determining how to automate the process of building spheres within Blender that correctly correspond to the size and location of stars making up various constellations.

Accomplishments that we're proud of

The overall aesthetic of the simulation came out great. We started the weekend off with a much smaller vision and are proud of how we allowed that vision to evolve and transform into a much more cohesive piece or art. There were quite steep learning curves associated with many of the tools we utilized but we were able to overcome them and bring all of the pieces together successfully.

What we learned

We came into this project with a basic understanding of Python Programming and Blender. Everything else we learned on the fly along the way. We learned a lot about using virtual reality hardware, neural networks, editing textures and audio clips, visualization, and the process of integrating multiple pieces from different software and building a cohesive creation.

What's next for I Desire Apple Pie

Continuing to expand our collection of deep art, neural network art, and transfer learning art; as well as increasing the amount of user interaction by creating simulated people within the experience and adding more narration for pieces that are not currently supported. Fine touches and the development of a polished executable for future versions.

Built With

• python
• ue4