Final submission is attached as a pdf file in the additional info section. Codebase: https://github.com/abidart/gflownets-hubbard
Who
Alan Bidart - abidart Kevin Rapp - krapp
Introduction
In the context of physical chemistry, understanding quantum-level interactions between particles is the biggest bottleneck in engineering novel nanomaterials or 2D materials—very small or thin materials. Collaborations of theoretical and experimental groups often drive research in this field. Theoreticians use frameworks that approximate quantum interactions to study what stable configurations of particles could lead to interesting properties and experimentalists try to build them. Unfortunately, most of the computational approaches that theoreticians use to exactly model materials have complexity scalings that are exponential or worse. In recent years, most theoreticians working on material discovery have turned toward approximate (non-exact) models to understand the properties of materials, which have polynomial complexity scalings. This is an active area of research in the field.
We are interested in implementing the framework described in [1] because we believe it might help speed up some of the computation on a popular approximate theory to model quantum dynamics called Auxiliary-Field Quantum Monte Carlo (AFQMC). We also believe that this is a relevant problem to work on since a speed-up on the AFQMC algorithm could accelerate the discovery of novel, useful materials.
We are both students in the Chemistry Department and work with Professor Brenda Rubenstein, a theoretical chemist specializing in Quantum Monte Carlo methods for quantum simulations. The framework described in [1], “Generative Flow Networks,” caught our attention because it claims to be an efficient way to sample from complicated distributions. Since AFQMC requires sampling from complicated distributions, we thought it would be curious to understand if the GFlowNets framework could help us speed up quantum simulations.
The GFlowNets framework outlines the steps to train a generative model that can be used to sample from complex distributions. Throughout Reinforcement-Learning-based training, a deep neural network learns to approximate the Directed Acyclic Graph (DAG) that results from the Markovian Decision Process (MDP). Given a starting state in the MDP, the resulting architecture can be used to efficiently sample terminal states. In other words, it is as if the architecture learns a sparsification of the MDP’s DAG, which otherwise would be too big to store and traverse in memory.
[1] Bengio, Yoshua, et al. "Gflownet foundations." Journal of Machine Learning Research 24.210 (2023): 1-55. Link: link
We plan to create a new implementation of a recently developed deep learning framework called Generative Flow Nets, or GFlowNets. Our implementation would involve a chemical application, by applying the model to a Hubbard model, a benchmark for molecular modeling. We are a part of a chemistry and physics research group at Brown called the Rubenstein group, and this problem is relevant to the work we do in this group. This implementation involves reinforcement learning.
Related Work
Zhang, Dinghuai, et al. "Generative flow networks for discrete probabilistic modeling." International Conference on Machine Learning. PMLR, 2022. Link: https://arxiv.org/abs/2202.01361
This ICML 2022 paper uses Energy-Based GFlowNets to model the ferromagnetic properties of 2D lattices, building on top of a quantum chemical framework called the Ising model (Section 4.1). The Ising model is a framework used to approximate the changes that the surface of a material will undergo at a given temperature as it tends to “relaxation”—in physical chemistry this relaxed state is known as the system's ground state. The approximated path can help us make predictions about the ferromagnetic properties of the material at that temperature. In the paper, the researchers use the Energy-Based GFlowNets framework to train a multi-layer perceptron to “encode” the rules of the Ising model, and generate paths from starting surface states to ground states.
"Generative flow networks for discrete probabilistic modeling" code: link
Data
We do not plan to work using labeled datasets, but we do plan to write code based on quantum chemistry to generate Hubbard model states and calculate how stable a generated state is.
Methodology
We plan to use the novel Generative Flow Networks framework, proposed in 2021 by Yoshua Bengio, to explore our problem. The neural network for this framework can vary—we will treat the architecture itself as a hyper-parameter—but our plan for now is to focus on an MLP.
We will train our model via reinforcement learning, assigning rewards that are proportional to the energy level of the material.
Our work will be inspired by the paper “Generative Flow Networks for Discrete Probabilistic Modeling” by Zhang et al link. Our work will feature the Quantum Chemistry Hubbard modeling for energy calculations, which is considerably more advanced than the Ising modeling used in the paper.
Our backup plan would be to attempt to replicate the Ising model results from the paper, and study how replacing their MLP architecture with other architectures (like CNNs or RNNs) impacts the results.
Metrics
Success is constituted by an implementation that makes accurate predictions for paths for a state to transform and arrive at a ground state or near-ground state energy. Ideally, the closer the predicted path is to the ground state, the better. We would consider anything close to this to be a success.
We plan to run the GFlowNet framework on Hubbard models of different complexities, for example 1D Hubbard models, 2D Hubbard models with a small number of lattice sites, a small number of electrons, or a larger number of either/both. Through these steps we can explore the robustness of our model in handling models of increasing complexity and computational cost.
The notion of accuracy does apply for our project, because it will be the benchmark for if our model works (well) or not.
In terms of base, target and stretch goals, a “minimum functionality” we would be happy with looks like our model learning on a Hubbard model of any complexity and making some kind of prediction for a state to transform through a path that leads to the highest reward and has the most flow. If the accuracy is low, this would still be a good starting point for future work, and would allow us to have some understanding of what is and is not working with our implementation. Ideally, we would like these predictions to be relatively accurate and to reach these predictions quickly. This would be our target goal, for our model to make predictions for paths that are more often right than wrong and to do it quickly. Our stretch goal would involve optimizing our MLP architecture and loss to further increase the accuracy and decrease the runtime of our model to arrive at more accurate predictions faster.
Ethics
Why is Deep Learning a good approach to this problem?
Deep Learning is a good approach to this problem because it could provide an approximate but efficient computational strategy to simulate how a simple model of a molecule will adjust its electron configuration to arrive at its ground state. Typical exact simulations, especially with models of high complexity (i.e. more than 10 lattice sites), involve computer clusters to complete calculations, or are impossible even with the power of supercomputers. With a deep learning strategy, models that were previously inaccessible or extremely inconvenient will hopefully be much easier to work with.
How are you planning to quantify or measure error or success? What implications does your quantification have?
We are planning to quantify error and success through the accuracy of our model in predicting a path for a state to transform to a state with ground state energy. If this is feasible and we reach this goal, we will also incorporate attention to transformations that are more efficient than others that lead to a state with ground state energy. The implications of this quantification mostly involve how we understand and represent our results. This application of a recent deep learning model is relevant to the current state of research in computational and theoretical chemistry and physics, so the way in which our results are represented and their ability to be reproduced is very important.
Division of Labor
Alan will be responsible for implementing the GFlowNet framework. This will involve reading literature about how it functions and implementing it to handle Hubbard model states and its valid transformations to arrive at terminal states, while assigning flow values, reward values, and a loss function that will correctly update flow values of the framework to incorporate the proportion of flow and how the reward of a terminal state influences this update.
Kevin will be responsible for implementing the Hubbard model, specifically to create some state given parameters for the system, to generate valid moves this state can make for the GFlowNet to transform to successive states and reach terminal states, and to calculate the energy of a state to check if it is a terminal state or not.
Together we will work to on the MLP implementation that will train the GFlowNet.
Log in or sign up for Devpost to join the conversation.