VectorXRay

VectorXRay helps doctors & researchers in diagnosing patient's X-Ray scans by providing similar images taken from a large dataset of X-Ray scans to help them in identifying diseases better.

Comment

Inspiration

The inspiration behind VectorXRay stems from the need for enhanced diagnostic tools in the medical field. It is recognized that doctors often face challenges when interpreting X-Ray images, especially when for rare conditions. VectorXRay is a tool that can assist doctors by providing them with similar X-Ray images from a vast database, helping them make more informed decisions. By leveraging TIDB vector search functionality, VectorXRay aims to bridge the gap between traditional radiology practices and cutting-edge technological advancements.

What it does

VectorXRay is a web-based platform designed to assist doctors in diagnosing X-Ray images. The platform allows users to upload an X-Ray image, which is then converted into embeddings to be then fed into TiDB vector search engine to returns similar images, enabling doctors to compare and contrast the X-Ray in question with others that have been diagnosed. This can be particularly useful in identifying patterns, anomalies, or conditions that may not be immediately apparent.

How I built it

VectorXRay was built using a combination of modern web technologies and machine learning algorithms. The frontend is developed using React.js, & deployed using Vercel. I utilized Axios for handling HTTP requests, allowing interaction with the backend API. The backend, built with Node.js and Express, handles the image and database interactions. AWS S3 is used for storing and retrieving the X-Ray images. There is a python API which uses pretrained pytorch Deep Neural Network models to convert image into embedding which is used to retrieve the similar images. The first task of this project was to convert the entire database into embedding & insert them into TiDB vector database. The backend & python embedder is deployed using fly.to. Dataset Source - https://www.kaggle.com/datasets/nih-chest-xrays/data

Challenges I ran into

One of the major challenges we encountered was optimizing the image processing algorithm to deliver quick and accurate results. X-Ray images vary significantly in quality and format, making it difficult to achieve consistent comparisons. Integrating the machine learning model with the backend and handling large datasets efficiently posed additional technical challenges. Handling large amount of data & creating embedding & inserting into TiDB & mapping them into s3 path was error prone as requirements in the backend was changing constantly. There were also issues I ran into while deploying the app due to the amount of resources required to run the embedder.

Accomplishments that I'm proud of

I'm proud of creating a functional and reliable tool that has the potential to significantly impact the medical field. I'm also proud of the user interface, which is both aesthetically pleasing and easy to navigate. The ability to offer a robust history feature, allowing doctors to review past queries and results, is another highlight of the project.

What we learned

Throughout the development of VectorXRay, I have learned a great deal about developing various aspects of a full stack app & making sure they work as expected in the production. Working with many components components(frontend, backend, python embedder, dataset preprocessing, TiDB, mongoDB) & making sure they work well together in conjunction in production was a great learning experience.

What's next for VectorXRay

The next step would be to reduce the latencies when fetching images with S3. In addition to that, other information about the patient such as age, gender, previous medical history, etc. can also be used when generating embeddings for better predictions by TiDB vector search engine.

Share this project:

Updates