Deep Learning for Weather Related PDEs

Poster
Solving inverse problem for Lorenz system
Solving Riemann problem with rarefaction waves
Pressure distribution around square by PINN

Title

Deep Learning for Weather Related PDEs

Who

Qian Zhang, qzhang64 Sicheng Liu, sliu167 Ziyao Xu, zxu59

Introduction

Weather is important in our daily life. Correct weather forecasts are critical for agriculture and industry. From fifty years ago, people started to simulate and forecast the weather by solving some simple differential equations like the Lorenz system with computer programs. Later on, with the rapid development of computational hardware and algorithms, it became possible to predict weather by solving Euler equations, a set of partial differential equations (PDEs) which is suitable for describing the dynamics of gas, but very costly for computation.

Nowadays, since deep learning has attained tremendous success in many fields, especially computer vision and natural language processing, we cannot help but think if it is possible to apply this powerful tool in computational science and engineering, like studying the weather related PDEs like Lorenz systems and Euler equations. An innovative idea was proposed in 2017 by Prof. George Karniadakis that we are able to train a neural network to solve the PDEs by adding PDE residuals to the loss function. Here we will follow the idea and use deep learning to study a forward problem and an inverse problem of the weather related PDEs. Although it is essentially a regression problem, the combination of the topics and approaches has not been studied before.

Related Work

An important paper we are referring to is Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, which can be found at https://www.sciencedirect.com/science/article/pii/S0021999118307125.

In this paper, the authors want to encode the physics formulas, which often emerge in the form of PDEs, to the design of neural networks. Specifically, they proposed to add the residual of PDEs, i.e. the MSE between left hand side and right hand side, as long as the initial and boundary value conditions, as a loss function for the neural network. Since partial derivatives of the variables can be handled by auto differentiation in modern deep learning packages, the optimizer can minimize this loss to give us a solution. In our case, we will design the loss function similar to their idea, and use these losses to solve the forward and the inverse problem.

Data

In our project, the data has two parts: temporal and spatial coordinates, and the physical quantities on the corresponding coordinates. For the forward problem in our project, we do not need physical data to train the model, as it is physically constrained. We only need them for evaluation. For the inverse problem, we use both parts of the data to train the model to find out the parameters of the system.

These data are generated by traditional numerical methods. The data size is not fixed, as it depends on the need for training and validation.

Methodology

In general, we train a model that takes the temporal and spatial coordinates as input and produces physical quantities as output. This model will use MLP as its structure because it is sufficient for our problems, even though it’s quite simple.

For the forward problem, we have designed a loss function including initial condition, boundary condition and physics equation. In this loss, we do not need the generated physical quantities. Then we minimize the loss to make the network produce physical quantities which satisfy all these conditions. For the inverse problem, we designed the loss similarly but the loss on data is included. Since the parameter we would like to study participated in the forward computation of the physical quantities, we can optimize the loss between the prediction and the target data to get the parameter.

Metrics

For our forward problem, we use high resolution generated data from traditional numerical solutions as the reference and use MSE as a metric between our results and the reference solution. The smaller MSE means the better our results are. For the inverse problem, since we have the parameters in data generation, we can directly compare the absolute error between the parameter from the inverse problem solver and the “true” parameter we assigned. Since the absolute value of the error may not be important as the values of the physical quantities can be very different for different problems, we can expect an error of magnitude around 10e-2.

Ethics

Deep learning based algorithms are general and flexible. Since the PDEs used to model weather could be extremely computational costly to solve by traditional numerical methods, a simple end-to-end approach like deep learning can be helpful in simulating these equations. While traditional methods need to create fine mesh for complex geometry, which is very expensive, deep learning can handle these situations easily. Meanwhile, conditions and parameters of these PDEs can vary a lot in different cases. Deep learning, which is a general method which does not require much prior knowledge, can be important for solving the inverse problems.

However, since there is little convergence study about deep learning, its robustness cannot be guaranteed. Since weather forecasting institutes may use the results of this project to improve their forecasting service, if this approach goes wrong or unstable sometimes, it can cause harmful results. Hence, traditional methods can still be used in parallel on coarser meshes as a validation, thus the possible risk of wrong prediction can be minimized.