Around 6 months ago one of our teammates recognized some interesting patch-like figures on his tongue and started to research possible indications. Then he found that it could be anything and he decided to visit an ENT specialist. That ENT specialist told him that there is nothing severe but it looks like a genetic condition named Geographical tongue.
Then, we started to research and learn more about other genetic conditions and possible bad scenarios like tongue cancer. Through his dentist friend's directives, we learned that there are 21 different mouth conditions and 6 of them can cause serious damage or kill people. Also, we thought that nobody checks his mouth regularly if there is no pain or blood. But these conditions can occur without any symptom and it can be too late before you see a specialist. In the meantime, we realized that a few of these diseases are not that dangerous but they look dangerous and they can influence people's mental health until that person sees a specialist.
On the other hand, we thought that it could be a very strong supportive application for visually impaired people too. Like how we described above, these conditions can occur without any noticeable symptoms.
In fact, finding a correct specialist is another issue because dentists don't deal with such cases regularly. In general, the best option seeing an ENT specialist in the first place and then, getting another appointment with a Dermatologist or Operator Specialist according to the specialist's advice. But there are some cases you should see a Dermatologist directly.
What it does
The current version only checks for 8 common cases and advises you on what to do accordingly. It doesn't aim for an accurate diagnose, it directs you to understand the current situation and find the correct type of specialist and helps you to describe your condition before you get a doctor appointment.
How we built it
We tackled 3 main problems so far.
The first one is finding the mouth location. We used state-of-the-art face detection and heuristic mouth localization techniques.
The second one is mouth segmentation. We trained our own custom tongue segmentation model to just examine the tongue area and not to be influenced by other parts(They will be included in future versions.)
The third one is the classification and similarity search which tells you that what kind of problems the app sees and if there is a dangerous case, which one can be similar to your situation. We also, trained our own custom classifiers.
Challenges we ran into
The main challenge is always finding good amounts and high-quality data. On the internet, there wasn't enough publicly available data so we decided to collect our own data by web-scraping and custom capturing.
Accomplishments that we're proud of
Probably the best tongue segmentation algorithm and the best tongue status(8 classes so far) classification algorithm so far. Also, we managed to put all of them together and collected good results in real-life cases.
What we learned
Clean, well-organized, and diverse data is a must and you can't avoid it.
What's next for BeAware
Scaling it up for all possible oral conditions and make it more robust for possible future scenarios.