SAAIV: Reconstructing Velocity/Pressure Fields on the Brain.

Removing barriers with scientific machine learning

Comment

Inspiration

The scope of traditional methods usually limits our capability to model physical phenomena. For instance, quantifying the intracranial cerebrospinal fluid (CSF) velocity and pressure is crucial to understand the brain’s glymphatic system (GS). However, obtaining data from the CSF flow is challenging since adapting data acquisition devices inside the brain would severely modify the biological system and potentially kill the animal. Nevertheless, the advent of scientific machine learning represents a unique opportunity to obtain models through hybrid data-driven approaches.

What it does

Artificial intelligence velocimetry (AIV3D) infers 3D continuous CSF velocity and pressure fields from 2D sparse particle image velocimetry (PIV) data obtained from the perivascular space (PVS) inside a murine's brain.

How we built it (Timeline)

  1. Visualize our boundary conditions and residual points. (Due: 11/18/2022)
  2. Normalize the data. (Due: 11/20/2022)
  3. Feature-based data generation. (Due: 11/23/2022)
    • Deal with unbalanced datasets: 9500 PIV data points (t,x,y)->(u,v), 5e5, BCs points (t,x,y,z) -(u,v,w), 5e6, and residual points (t,x,y,z). (Due: 11/23/2022)
  4. Modify the Navier-Stokes Flow nets (NSFNets) to work with real data. (Due: 11/25/2022)
    • Redefine the conservation of mass and momentum equations in x and y and include an additional equation for the momentum in z. (Due: 11/27/2022)
    • The Reynolds number is significantly low inside the brain, so add a scaling parameter inside the PDE loss. (Due: 11/27/2022)
    • Add a supervised loss that minimizes the error from the PIV data. (Due: 11/27/2022)
    • Add smart weights to regularize the multiple loss optimization. (Due: 11/27/2022)
  5. Define a suitable network architecture that takes four inputs (i.e., t,x,y,z) and gives four outputs (i.e., u,v,w,p). (Due: 12/08/2022)
  6. Obtain a model that infers the velocity in z and the pressure. (Due: 12/08/2022)
  7. Deep Learning date (Due: 12/11/2022)

Challenges we ran into

  1. Preprocessing the data. We are working with three datasets: 9500 PIV points, 500000 BCs, and 5000000 residual points. So preprocessing imbalanced data sets was complicated.
  2. Defining a suitable method to use self-adaptive weights for three batched and unbalanced datasets
  3. Including our 3D Navier Stokes equations.
  4. Dealing with the small Reynolds number, it blows up the Laplacian of the velocity and the training becomes unstable.
  5. Defining suitable hyperparameters. Training our NN takes 18h, so finding the right parameters was extremely time-consuming.

Accomplishments that we're proud of

  1. We defined a feature-based data generation that enables us to use self-adaptive weights for batched data.
  2. We found a way to deal with small Reynolds numbers by scaling the inner terms of our loss function.
  3. We solved a min-max problem. For the self-adaptive weights, we need to maximize the weights and minimize the training error. To do it we applied two optimizers that work in parallel during the training process.
  4. We successfully reconstructed the CSF velocity and pressure fields in the PVS.
  5. By applying self-adaptive weights we reduced the MSE of the unsupervised error (PDE) up to 70% and the supervised error by 10%.

What we learned

  1. Physics-informed neural networks.
  2. Self-adaptive physics-informed neural networks.
  3. Multi-loss optimization.
  4. Dealing with unbalanced datasets.
  5. Dealing with real-noisy data.
  6. Solving min-max problems.
  7. The relation between the glymphatic system and the CSF

What's next for SA-AIV: Reconstructing velocity/pressure fields on the brain.

Using the pressure and velocity fields we found to infer the GS inner mechanisms. Also, we have to find a non-invasive way to test it on humans.

Reflection

  • How do you feel your project ultimately turned out? How did you do relative to your base/target/stretch goals?

Answer: I fulfilled my stretch goals!

  • Did your model work out the way you expected it to?

Answer: Yes, it worked even better than expected.

  • How do you feel your project ultimately turned out? How did you do relative to your base/target/stretch goals?

Answer: I am happy, my SA-AIV exceeded my expectations. It was hard but I pulled it off so I am proud of this work

  • How did your approach change over time? What kind of pivots did you make, if any? Would you have done differently if you could do your project over again?

Answer: Initially I just wanted to do the AIV. However, thanks to the Thanksgiving break I could make work the SA for this problem which was tricky.

  • What do you think you can further improve on if you had more time?

Answer:

– I would have used a deep learning approach to segment our data (i.e., find the lambda groups for the SA weights).

– I would have tried the l-bfgs optimization after using Adam to improve the results even more.

– My SA-weights for the residual points did not fluctuate as much as for the other datasets (i.e., PIV, BCs). I would have liked to explore why that happened.

– My SA weights grow unboundedly, I would like to try an approach to control their growth while keeping their performance

*What are your biggest takeaways from this project/what did you learn?

Answer:

– Physics-informed neural networks.

– Self-adaptive physics-informed neural networks.

– Multi-loss optimization.

– Dealing with unbalanced datasets.

– Solving min-max problems

References

Kelley, D., Cai, S., Boster, K., Ladron-de-Guevara, A., Sun, J., Zheng, X., ... & Karniadakis, G. (2022). Artificial intelligence velocimetry reveals in vivo pressure gradients in brain cerebrospinal fluid. Bulletin of the American Physical Society.

Lu, L., Meng, X., Mao, Z., & Karniadakis, G. E. (2021). DeepXDE: A deep learning library for solving differential equations. SIAM Review, 63(1), 208-228.

Jin, X., Cai, S., Li, H., & Karniadakis, G. E. (2021). NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations. Journal of Computational Physics, 426, 109951.

Cai, S., Li, H., Zheng, F., Kong, F., Dao, M., Karniadakis, G. E., & Suresh, S. (2021). Artificial intelligence velocimetry and microaneurysm-on-a-chip for three-dimensional analysis of blood flow in physiology and disease. Proceedings of the National Academy of Sciences, 118(13), e2100697118.

McClenny, L. D., & Braga-Neto, U. (2022). Self-adaptive physics-informed neural networks. Journal of Computational Physics, 111722.

Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686-707

Raissi, M., Yazdani, A., & Karniadakis, G. E. (2020). Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science, 367(6481), 1026-1030.

Thomas, J. H. (2019). Fluid dynamics of cerebrospinal fluid flow in perivascular spaces. Journal of the Royal Society Interface, 16(159), 20190572.

Built With

Share this project:

Updates