CSV Health Checker

Data is everywhere, but who's to say if it's good vs. bad data?

Comment

Inspiration

As an aspiring data scientist, I know firsthand how crucial it is to assess the quality of data before diving into analysis. Clean, well-structured data is the foundation of any successful data-driven project, and CSV Health Checker is a tool designed to ensure that your data is up to the challenge. Recently, I've sort of been in my self growth phase, and have been connecting with more people to hear their life story. The wanting to add colorblindness mode was inspired by just one small interactions I had with a stranger who has inspired me to build projects that can benefit people from all walks of life.

What it does

CSV Health Checker allows you to quickly evaluate the health of your CSV files. It assesses data quality by checking for issues such as missing values, inconsistencies, and overall structure. All you need to do is have an unzipped folder containing csv files, and upload it into the website (it is not being saved). The tool provides a percentage score and insights into the dataset's condition, helping data scientists like me decide whether the data is ready for analysis or needs further cleaning.

How I built it

I built CSV Health Checker using a combination of modern web technologies and machine learning. The frontend was developed with Next.js and React, while the backend leverages AI models to analyze CSV files and generate meaningful insights. With prompt engineering and feeding the ai the context of the csv, we return a grade similar to handing over homework.

Challenges I ran into

  • Never used openai api before, I had to resort to using Langchain to implement ai into my project.
  • (tiredness as shown in my showcase video lol)

Accomplishments that I'm proud of

Doing this in under 10 hours. If I had to explain my real challenge, it was figuring out how react and canvas work together, you can see a broken web app on my git repo.

What I learned

Openai api calls and data science skills! Although as far as graphics go, I learned a lot more about design choice. When introducing the color blindness mode, I had do a bit of research to get a better understanding of my color choices.

What's next for CSV Health Checker

In the future, I plan to add an auto-cleaner and more interactions with the drag and drop features I implemented using react-dnd. Features like appending, creating, and cleaning data are my next goals. However just limiting myself to just csv feels like a crime. Databases such as MongoDB Atlas is one I’d love to expand my website towards… despite the name of the site.

Built With

  • daisyui
  • langchain
  • mui
  • nextjs
  • openai
  • react
  • react-dnd
  • styled-components
  • tailwindcss
Share this project:

Updates