Immersive Gaussian Climate Simulation (IGCS)

Inspiration

We noticed that people did not really care about climate change until recently when Canada experienced smog from forest fires, heatwaves, and less snow in Winter. We observed that people pay attention and take action only when experiencing these unusual phenomena. From this project, we aim to increase people’s awareness of climate change by creating immersive experiences of extreme weather via machine learning; in some of the iconic areas of the cities to drive personal connections.

What it does

Simulate extreme environments in real world map generated by 3d Gaussian Splatting via Unreal Engine.

How we built it

We started by capturing video footage, which we then converted into images.
These images were used to train a 3D Gaussian Splatting to create a 3D model of the environment.
We then imported this model into the Unreal Engine and added various weather effects to simulate extreme conditions.
Finally, we made this simulation accessible through a web browser via Arcware.

Challenges we ran into

One of our biggest challenges was converting the images into a format that could be used to train our model. We spent almost 8 hours on this task, only to find that the resulting datasets didn't match the reconstructed point clouds, meaning we couldn't proceed to the next step. Fortunately, we were able to get two of our models to work eventually.

Another challenge was that I (Ben) got kicked out while filming the footage for the project.

Accomplishments that we're proud of

We're proud of using techniques we've never tried before. We learned how to capture, train, and and even deploy our project (simulation) from scratch. Despite facing multiple errors, mistakes, and crashes, we made it to the end!

What we learned

Our team members didn't have solid knowledge of all the technologies used in this project, so we learned together how to develop each implementation part and how to solve all the bugs we encountered along the way.