Molecular structure is an incredibly important part of chemistry. Manipulating these molecular structures leads to isomers, which can have different reactivity and properties. Identifying these isomers requires a high degree of spatial awareness, a skill which is difficult to teach and often people can visualize it or they can't. This tool aims to help those without spatial awareness visualize molecules and isomers.
What it does
Identifies atoms in a 2D drawing of the molecule then determines which atoms are connected in the drawing and then creates a 3D model of the molecule that can be rotated and manipulated. The atoms are arranged properly in space according to bond angles from VSEPR theory.
How we built it
Using Python and OpenCV, we identified the letters representing atoms and then fit the atoms to a line in order to find each bond in the molecule.
Challenges we ran into
Many computer vision techniques we tried couldn't differentiate between certain letters, like L and I, and the line representing the bond. We also initially intended to integrate this into an Android app and use AR Core to produce a 3D image when the camera was held over the molecule; however, none of the Android devices we had access to were compatible with AR Core, so we were restricted from experimenting with that.
Accomplishments that we're proud of
We were able to make a program that identified key components of a molecular drawing, like the letters representing atoms and lines representing bonds. Additionally, we were able to identify what atoms were present in the molecule by classifying the letter. We also created a way to identify which atoms were connected in what arrangement based on lines drawn between the boxes.
What we learned
We gained more experience with OpenCV and scikit-learn and also got exposure to CV libraries such as Tesseract to classify the letters. We also researched different AR libraries and learned about their integration with Android Studio.
What's next for VSEPR Hero
We hope to be able to get access to a newer Android device compatible with AR Core and build a 3D model within an Android app using AR Core. The accuracy of the atom and bond-identification algorithms can be improved before being integrated into the app so that we can better read the 2D drawings to make more accurate 3D models.