MLuminate

The problem: how do we compute where the red arrows go?
The method: adding shadowmasking to an unlit, single-pass occluded scene
The results: a ML approximated shadow (to the right). Notice a minor quality reduction and noise.

Inspiration

As a game developer, it's always been apparent that lighting up environments is both the most challenging and important aspect of making traditional immersive experiences. We wonder "is there a faster way"?

What it does

MLuminate attempts to take the complex features of a scene (lights, player, objects) and use a smart ML dimensionality reduction strategy to return a constant-time lightmap representation of the scene.

How I built it

The primary strategy for this app was to use Principal Component analysis to reduce the dimensionality space of a descriptive set that would characterize both the environment objects and the light conditions, before using a CNN to generate subsequent lightmap data.

Challenges I ran into

Big questions to answer were: "How do we represent the scene?", "How do we make this a simpler problem?", "What tradeoffs are we inviting by reducing the runtime to an approximation?", and "Where is the training data?".