OncoBot

Discord Profile of OncoBot
OncoBot in action (nearby hospitals)
OncoBot in action (yes melanoma)
OncoBot in action (no melanoma)

Inspiration

Seeing the growing number of cancer patients around the world and with two of our team members having multiple family members directly affected by the disease, we thought of making a bot that can preemptively help detect skin cancer at its early stages so that people can get the desired treatment. Annually, around 200,000 individuals globally are diagnosed with melanoma, and despite rapid advances in oncological treatment, many find out too late. This inspired us to design OncoBot as a way to help worried individuals check for skin cancer.

What it does

OncoBot allows patients to inspect suspicious marks on their skin and directs them to nearby medical services. Using computer vision, the program detects the presence of melanoma from the images that the user inputs. It utilizes various image processing techniques to remove noise from images and try to find potentially dangerous lesions. Additionally, the bot uses a Google Maps API to navigate the user to their nearest 5 hospitals for further examination if they type in an address.

How we built it

The project was built entirely using Python, as it allowed for the easiest integration of computer vision libraries, as well as Discord and map APIs. We had one team focused on the computer vision aspect, developing the algorithm and image processing sequence. This was crucial, as each step needs to be in the right order so that important information was preserved. This part involved much exploration, as only one of our members had prior experience with the OpenCV library. Our second team was focused on creating the user experience, integrating the bot using the Discord API and enabling it to access geographical data to find nearby medical services using a Google Maps API.

Challenges we ran into

We faced many significant problems through the course of this project. As this the first hackathon for all of us and none of us are particularly experienced with programming projects, the brainstorming process was uniquely challenging. Working within the constraints of our respective skillsets was something we had to pay close attention to so that no one would be unable to contribute.

Another problem we faced was the privacy of the virtual environment on the GitHub repository for the project. The environment included the unique token for the bot, which was called by the "main.py" file. This was a problem, as publicizing the token was not an option. We inevitably realized that this token was not meant to be published. Other users, should they want to replicate the functionality of the bot, should make their own bot with its own token and replicate the rest of the code; otherwise, they are able to join the server with our bot. Obvious in hindsight, but we were stumped nonetheless.

The most pressing problem we had was near the end of the project, as merging all parts of the project produced many errors apropos recognition of dependencies. After pulling the CV algorithm onto the computer with the Discord bot, the interpreter was throwing false errors for nearly every library that had been installed, saying that it couldn't recognize them. Despite these errors, the program ended up running exactly as intended, but we still don't know why those errors came up, whether or not they're remotely valid, or how to debug them. Even then, we spent several hours racking our heads as to the cause of this issue.

Accomplishments that we're proud of

We accomplished many feats that we didn't even fathom at the start of the project. First, on the CV algorithm, we were able to achieve a cumulative 82% accurate labeling system using solely existing pre-trained models within OpenCV. Due to both technical and technological limitations for us, training a model would not have been feasible, so getting close to what a neural network trained from scratch specifically on this datatset might be able to achieve is something we're incredibly proud of. Beyond that, though, we're proud of our ability to work as a team to see a project to completion. Each one of us has experience starting coding projects that often end up unfinished, so actually having a collaborative effort with multiple moving parts and functionalities we were unfamiliar with end up with something tangible is an amazing feeling for all four of us.

What we learned

This experience was, more than anything else, an amazing learning experience. We all obviously became much more familiar with the tools we were using for our individual part of the project. But there was a constant cross-pollination; the CV team learned about all the parts of the Discord and Google Maps API integration (especially during the final hours-long dependency issue) and vice versa. We all also became familiar with the development process itself, including key components such as GitHub functionalities, how virtual environments work and why they're useful, and more general skills of teamwork, time management, and communication. It was overall an extremely fulfilling 36 hours that's going to have an impact on all our programming lives for the foreseeable future.

What's next for OncoBot

We think that OncoBot can be made even more effective in the future with more extensive capabilities and features. We plan on further refining the CV aspect by training our own model on the specific datatset so that it's more targeted to images of skin, rather than images of anything. Additionally, we'd like to extend it's usage so that it can also detect other skin conditions such as eczema or carcinoma, allowing some sort of future integration with virtual healthcare platforms so that the patients can also directly connect to healthcare professionals for an immediate diagnosis. Most importantly, we would like OncoBot to be a free to use, accessible platform for all people to get instant medical advice and guidance when care might otherwise be immediately available.