Dataverse

Inspiration

DataVerse was inspired by the need for an efficient way to extract specific information from large datasets, like CSV files and Google Sheets. Many users struggle with manual data searching, so we aimed to automate the process using AI.

What it does

DataVerse automates the extraction of specific data, such as email addresses and phone numbers, from CSV files or Google Sheets. Users can input customizable queries and get results extracted through AI-powered web searches.

How we built it

We built DataVerse using Python with Flask for the web interface, Pandas for data handling, Google Sheets API for Google Sheets integration, SerpAPI for search capabilities, and Groq API for AI-based information extraction.

Challenges we ran into

Integrating multiple APIs (Google Sheets, SerpAPI, and Groq) required careful handling of authentication and rate limits.
Parsing and extracting relevant information from search results posed accuracy challenges.

Accomplishments that we're proud of

Successfully integrating AI-powered web search for automated data extraction.
Creating a user-friendly interface that allows seamless CSV and Google Sheets uploads, with real-time results display and downloadable CSVs.

What we learned

API integration can be complex but rewarding when everything works together.
Handling large datasets and queries efficiently in real-time requires careful optimization.
Working with AI for data extraction opens up new possibilities for automation.

What's next for DataVerse

Enhance the AI capabilities to support more complex queries and data types.
Improve user experience with better UI/UX design and features like scheduled data extraction.
Expand compatibility with additional data sources like databases and APIs.