Microscope Assistance (Edu app)

An easy-to-use app to assist students to learn about the vast yet invisible microscopic universe through AI (object detection) and modern technology.

Comment

Inspiration

Microorganisms are interesting, beautiful, and very invisible to the naked eye. Studying them would unravel the very reason for the existence of life on earth. This app that I created uses modern technologies like AI (object detection) to help armatures and students study the vast universe of micro-organisms.

What it does

This app when installed on a device and attached to a microscope as shown in the image, would detect microorganisms from the camera input and display what microorganism it is. microscope with smart-phone

The model was trained using YOLOv5 with the data that was manually annotated using Roboflow which was collected from here. The model generated predicts between the following classes (class numbers):

  1. Actinophrys
  2. Arcella
  3. Aspidisca
  4. Codosiga
  5. Colpoda
  6. Epistylis
  7. Euglypha
  8. Paramecium
  9. Rotifera
  10. Vorticella
  11. Noctiluca
  12. Ceratium
  13. Stentor
  14. Siprostomum
  15. Keratella Quadrala
  16. Euglena
  17. Gymnodinium
  18. Gonyaulax
  19. Phacus
  20. Stylongchia
  21. Synchaet

Further, the user/student can click on any organism with the detected bounding box to get more details about the organism. This would create a very interactive and fun way of exploring and learning about microorganisms

How we built it

First, the images were collected from here. The images were filtered and manually annotated using software called Roboflow. Then the data was exported to Google Colab. YOLOv5 was used to train and export the model. By leveraging the tflite features of TensorFlow, the model was used to predict the microorganisms in real-time. The app was built using react-native using the expo CLI.

Challenges we ran into

One of the major challenges was that we were unable to properly integrate the exported model to the app, as there was less time and very less tutorials about it. Even though the model produced worked perfectly on integrating with the laptop's camera, it was difficult to replicate it in react-native.

Accomplishments that we're proud of

We are proud of the model that we generated with a self-annotated small dataset. results

What's next for Microscope Assistance (Edu app)

The first step would be, finishing the integration of the model with the app would be the first priority. Then we would like to include more micro-organisms and expand the dataset. And ultimately use the app in institutes and allow the amateurs and students to explore and learn more about the vast invisible universe of microorganisms

Built With

Share this project:

Updates