Inspiration

Enterprise organizations generate thousands of datasets, but discovering and understanding them remains a challenge for new engineers and analysts. Documentation is often spread across multiple systems, making onboarding slow and knowledge difficult to access.

DataPilot was built to provide a single workspace where users can discover enterprise datasets, understand their purpose, explore documentation, view lineage, and access learning resources in one place.

What it does

DataPilot is an enterprise data discovery and onboarding platform that helps engineers quickly become productive.

The platform includes:

Searchable enterprise data catalog
Dataset metadata and ownership
Business documentation
Data lineage visualization
Sample data and SQL examples
Learning Hub with guided courses and documentation
Interactive onboarding missions
Dataset-specific knowledge resources

The platform is designed to reduce onboarding time while making enterprise data easier to discover and understand.

How I built it

The application was built using Next.js, React, TypeScript and Tailwind CSS.

The backend exposes API routes that serve dataset metadata and documentation while the frontend provides an interactive experience for exploring datasets, learning resources, and onboarding missions.

The project models realistic enterprise datasets across multiple business domains including HR, Finance, Marketing and Engineering.

The application is deployed on Vercel.

Challenges I ran into

One of the biggest challenges was designing a data catalog that felt realistic while remaining easy to navigate.

Another challenge was creating reusable dataset models that could represent different business domains while maintaining a consistent user experience.

Connecting catalog pages, documentation, learning resources, SQL examples and onboarding flows into a cohesive experience required careful planning of the application structure.

Accomplishments that I'm proud of

Built a complete enterprise data catalog from scratch.
Created reusable metadata models for multiple business domains.
Designed an interactive onboarding experience instead of static documentation.
Integrated learning resources directly into dataset exploration.
Successfully deployed the application as a production-ready web application.

What I learned

This project helped me gain deeper experience with modern React development, the Next.js App Router, TypeScript, API design, and building scalable component-based applications.

I also learned how thoughtful information architecture can significantly improve the onboarding experience for engineers working with enterprise data.