Introduction:
The primary problem we are trying to solve is whether a deep generative model is capable of conditioning on geospatial data – such as satellite imagery, topographical maps, and other terrain factors – to generate realistic photos taken from the ground. This involves learning a meaningful representation that consolidates these geospatial factors, and using it to synthesize realistic images corresponding to these inputs. This is a type of unsupervised learning problem, specifically within the domain of generative modeling. Unlike traditional tasks, this model leverages unlabeled data to learn the conditional distribution of plausible ground images given local terrain data. The training process harnesses pairs of geospatial and ground image data, teaching the model to generate accurate and realistic depictions based on the input terrain features. The idea for generating photorealistic images from geospatial data arose from an interest in exploring the capabilities of deep generative models to interpret and recreate complex visual representations from varied and abstract data sources, like satellite imagery and topographical maps. Beyond this, we were driven by the potential applications such models could have in fields such as environmental science and urban planning, which could benefit from enhanced visualizations of geographic locations based on this data. Moreover, the training of this model will also develop a highly versatile encoder of the geospatial features, which can become an effective tool for other applications involving this kind of data, beyond what we intend to explore in this project.
Challenges:
In our project, the random selection of datapoints often did not align properly with corresponding Google satellite images. This misalignment complicated our data accuracy: as a result, we further restricted our datapoints to have a reliable dataset to train on. Additionally, we are continuing to update and revise our model as new papers are published - some papers (in particular a published open-source model by allenAI) were published after our initial project proposal was submitted.
We are on track with our project, and are thinking of referencing the allenAI paper for further direction.
Log in or sign up for Devpost to join the conversation.