MedVi

Paper to AI, saving time, saving lives.

Comment

Inspiration

In his third year of study, Victor faced a chronic ailment that left him weak and in pain. After a long wait at the school clinic, the nurses couldn't locate my medical files, denying me care. Thankfully, his father was there to transport him to a private hospital 13.3 km away. This experience made him reflect: what if he or someone else had been in a critical situation, like needing an emergency blood transfusion?

For the past few months, team Medvi has been focused on addressing Nigeria’s healthcare challenges through innovative solutions. A starting point was to make medical history of patients availiable in a secure database for seamless healthcare to patients

What it does

MedVi captures handwritten clinical notes using a camera, extracts the text using Gemini AI, and saves it alongside patient details in a local database through a simple GUI interface.

How we built it

We built MedVi by integrating a Raspberry Pi camera with a Python-based GUI using Tkinter. We captured or uploaded images that were processed using Google Gemini’s vision model to extract the handwritten text. Patient details and reports are then stored locally in an SQLite database.

Challenges we ran into

Administration challenges: We were unable to obtain medical records from the school clinic to create a database of training data for the OCR model. As a result of this we decided to use the 'lstm' training feature of the Tesseract open source library. The 'textimage' command took the open source fonts which we downloaded from dafont.com, the groundtruth text files and size of the text to produce the three files needed to train the Tesseract model (.tif files, .lstmf files and .txt files). This turned out to be the best development as it automated our initial methodology of scanning large volumes of records to create a training dataset and then scan several others for the intended application.

Technical Challenges

  1. Font Tesseract Compatibility. Although our methodology was automated, still the 'textimage' command didn't produce the expected amount of .tif, .lstmf files. This is because of the incompatible font name, location errors and default configuration issues.

  2. OCR Accuracy Altogether the Tesseract OCR failed to produce good results when it came to handwritten inputs. So we moved to using the Gemini 2.5 flash which obtained better extraction but not without internet connection.

  3. Compatibility We encountered several compatibility challenges with our procured Raspberry camera. Our choice of camera which was the module 3 wide arrived only to be non-functional to our disappointment. We procured another camera module - the Raspberry pi zero, which was compatible but had terrible picture quality. We eventually opted for the USB 2.0 camera. Although this was delayed by several weeks of back and forth in the procurement phase.

We encountered similar compatibility issues with the 5” DSI touchscreen, which was packed with the wrong DSI flex. It was unmatchable with the slot on the raspberry pi 5 and we had to sacrifice compute by using an older and slower raspberry pi 3b+. We had no visual output for a while till we found the right orientation for the DSI flex in the DSI slots.

  1. Dependency issues We encountered several issues with poorly maintained opensource documentations when we were in search for an opensource multimodal vision model on huggingface.

  2. Module errors like (Pillow, GenerativeAI, Google API) and simple issues with virtual environment and file locations were also surmounted.

  3. Issues of Power Supply: We encountered flickering of the monitor display connected to our raspberry pi, apparently we needed the proper power supply cable, but it took a while to figure that out. We used VNC protocol to acccess the raspberry remotely, we observed that it reduced the power demands of the system so that it could be powered on a 20,000mAh 5V/3A power bank, which was reliable and an easy to field construct

  4. Integration of UI and OCR Added Gemini OCR to Tkinter UI. Was forced to overcome problems with not processed images on capture and all was ok on the upload side. Enhanced threading and GPIO button to make camera capture happen.

  5. Data Storage Firebase, which was the first choice, eventually blocked our database operations because we lacked a paid subscription. We pivoted to SQLite that is locally stored and offline friendly and added the functionality of downloading reports.

Accomplishments that we're proud of

1.) We are proud to have been recognised as the top medical innovation by the University of Ilorin medical classs of 2025, Aureum Generis.

2.) We finished third place in the maiden edition of the Technology Innovation Challenge at the Unversity of Ilorin, Kwara state, Nigeria.

3.) We finished among the top 20 finalists in the Meta x Nithub University of Lagos competition of over 300 teams. This was a competition for AI innovation in healthcare across Nigeria.

What we learned

We learnt from every challenge we encountered; procurement, administrative and technical challenges. We also learnt about our target market from reviews of judges in the competitions we attended. Our Go-to-market strategy was critiqued and advised.

What's next for MedVi

We have future decisions to host a multimodal vision model on the edge device. We would achieve this by supplementing the raspberry pi with the M.2 HAT to solve the compute challenge of that task. This way our system would fully bypass all resource constraints and would work in areas of no internet connectivity. We also plan to expand the digitization effort to other medical documents such as ECGs. For scalability and security we have plans to implement blockchain with hospitals having their own secure nodes.

Share this project:

Updates