-
-
TraceWise Home Page
-
Data Contracts
-
Pipeline Lab
-
Failure Simulation
-
Repair and Replay
-
Blast Radius
-
RiskLens
-
Recovery & Revalidation
Inspiration
TraceWise started with a brainstorming session around a simple but important question:
What happens when the data an organisation depends on cannot be trusted?
During one of my internships, I encountered exactly these kinds of challenges firsthand. Working with data in a real organisational environment made me realise that problems such as inconsistent records, missing values, duplicate entries, and unreliable datasets are not merely technical inconveniences. They can affect reports, downstream systems, analytics, and decisions that depend on that data.
That experience stayed with me. When brainstorming ideas for Next Byte Hacks V5, I wanted to build something around a problem I had actually experienced rather than choose an idea simply because it sounded interesting.
This led me to TraceWise — an attempt to bring data quality validation, failure diagnosis, downstream impact analysis, and controlled recovery into one platform.
What it does
TraceWise is an intelligent data reliability and risk platform that helps users identify unreliable data, understand potential consequences, and take controlled corrective action.
Its key capabilities include:
- Data Quality Monitoring: Analyses datasets, profiles columns, and identifies common quality issues.
- Data Contracts: Checks datasets against expected schema and validation rules.
- Pipeline Lab: Simulates data-quality failures, diagnoses them, suggests repairs, and allows validation to be replayed on an isolated working copy.
- BlastRadius: Maps downstream dependencies to identify which data products and systems could potentially be affected by an upstream issue.
- RiskLens: Provides a heuristic assessment of potential downstream risk.
- Recovery & Audit: Supports controlled remediation through quarantine, revalidation, and an audit trail.
How I built it
I built TraceWise using Python and Streamlit, with Pandas for data processing and Plotly for visualisation. The application is organised into five sections, each addressing a different stage of the data reliability lifecycle.
I used a transaction dataset containing deliberately introduced data-quality issues to demonstrate the platform's capabilities. This allowed me to showcase how TraceWise identifies problems, diagnoses failures, evaluates potential downstream impact, and supports recovery without modifying the original source dataset.
Challenges I faced
One of my main challenges was bringing multiple aspects of data reliability into a single, coherent workflow. Detecting an issue is only one part of the problem. Understanding its potential impact, deciding how to handle affected records, and verifying that a correction actually resolves the issue are equally important.
I also had to ensure that simulated failures and recovery operations remained isolated from the original dataset, while maintaining a meaningful audit trail.
Another challenge was making the platform useful beyond a fixed transaction dataset, which motivated me to add generic data-quality checks and CSV upload support.
What I learned
Building TraceWise reinforced an important lesson: data reliability is not just about finding errors. It is about understanding their consequences and making recovery safer and more transparent.
More importantly, this project allowed me to turn a problem I had encountered in a real-world internship environment into a practical, working prototype during Next Byte Hacks V5.
