avtMonMedExp

avtMonMedExp project searches for experts in the field of medicine on Twitter

Inspiration

The avtMonMedExp project using Amazon Comprehend Medical with Python3.

This project is a qualitatively new stage in the implementation of the avtMonExp project for Medical domain. Thanks to the use of modern natural language processing (NLP) tools from Amazon.

What it does

The avtMonMedExp project includes the following main stages in data processing:

First of all, search criteria (tags, phrases) that characterize the subject area and experts from the Medical domain are determined.
Search, retrieve, and analysis data about experts in Medical domain from Twitter based on pre-defined criteria.
Searching and analyzing experts from the field of medicine among Twitter users consists of two main stages:
- On the first stage of data analysis, an algorithm developed by A.V.T. Software is used to identify a Twitter user as an expert in a Medical domain.
- On the second stage of data analysis, we use Amazon Comprehend Medical DetectEntitiesV2 operation for analysis Twitter user data to identify specific medical entities. To more accurately understand whether the Twitter user whose data we are analyzing is an expert or a patient.
Saves data about found experts from Medical domain that corresponds to the specified criteria in the relational database for future analysis, and visualization.

How we built it

To develop the avtMonMedExpproject, our team used the Visual Studio Code IDE.
As well as the following important programming languages, libraries, and tools:

Python 3.9.5 - Python is a programming language that lets you work quickly
TwitterSearch 1.0.2 by Christian Koepp - A Python library to easily iterate tweets found by the Twitter Search API
AWS SDK for Python (Boto3) to configure, and manage AWS services, such as Amazon Comprehend Medical
Python Geocoder by DenisCarriere - Simple and consistent geocoding library written in Python
MySQL Community Server 5.7 - MySQL Community Edition is a freely downloadable version of the world’s most popular open source database that is supported by an active community of open source developers and enthusiasts
MySQL Connector/Python - MySQL Connector/Python is a standardized database driver for Python platforms and development.

Challenges we ran into

The search for experts according to the algorithm that we have developed requires an analysis of an unstructured text in the bio account twitters users. The user database consists of both medicine professionals and patients. Amazon Comprehend Medical to analyze an unstructured text solved this problem. Amazon Comprehend Medical algorithm for keywords determined diagnoses and we identified doctors and specialists and separately their patients.

Accomplishments that we're proud of

The project for the search for experts on Twitter received new models and development. We expanded it to the field of health, medicine. This gives new practical prospects for the project.

What we learned

We know in practice how Amazon Comprehend Medical works, including its API

What's next for avtMonMedExp

Improving the algorithms for finding and selecting experts in the field of medicine based on various methods of analyzing the data of Twitter users (and in the future, other social networks).

Select and implement a new modern architecture for avtMonMedExp application project. For example, cloud architecture.

Development of a graphical user interface for convenient interaction with the avtMonMedExp application and visualization of the results of data processing and analysis.

Extend the functionality of the avt MonMedExp application.