DataInsight

Inspiration

Researchers and professionals often have the data they need but not the statistical background or time to clean, explore, and interpret it. They are forced to wait for a specialist before they can answer basic questions about their own datasets.

DataInsight was created to shorten that distance: upload a spreadsheet and get a clear, reproducible first analysis in minutes.

What it does

DataInsight is a Streamlit application for accessible exploratory data analysis.

Users can upload CSV or Excel files and automatically receive:

• A data-health profile with missing values, duplicates, column types, cardinality, normality and outlier signals.
• Conservative data cleaning with smart defaults while preserving the original file.
• Feature recommendations, categorical encoding options, scaling suggestions, redundancy alerts and VIF estimates.
• Descriptive statistics, correlation analysis, group comparisons and categorical associations.
• A ranked “Top Findings” section that explains the most relevant patterns in plain language.
• A configurable visualization area where users choose the chart type, axes, grouping variable and correlation method.
• A downloadable cleaned dataset.

The app supports Portuguese (Brazil) and English. It can use automatic correlation selection or explicit Pearson, Spearman, and Kendall methods.

How we built it

The application was built with Python and Streamlit. Pandas handles tabular data, SciPy and statsmodels provide statistical calculations, scikit-learn supports feature transformations, and Plotly powers interactive visualizations.

Codex with GPT-5.6 accelerated the workflow from an empty workspace to a working application. We used Codex to structure the modules, implement the data pipeline, build the bilingual interface, add statistical safeguards for small or incomplete datasets, debug runtime issues, and iterate on the user experience.

A key design decision was to keep rule-based insights deterministic and auditable. The system explains accessible conclusions first and keeps technical details available for users who want to inspect the method and p-value.

Challenges we ran into

The main challenge was balancing useful automation with user control. Smart defaults make the first result fast, but analysts still need to understand and adjust what the app is doing. We addressed this with visible action logs, conservative outlier handling, a separate working copy of the data, configurable correlation methods, and on-demand charts.

Another challenge was supporting messy real-world files: different encodings, separators, Excel sheets, missing values, constants, probable identifiers, and small samples. The pipeline handles these cases without modifying the uploaded original.

Accomplishments that we're proud of

• A complete upload-to-insight workflow with no statistical setup required.
• Plain-language findings designed for non-specialists.
• Reproducible cleaning decisions and a downloadable treated dataset.
• A bilingual experience designed into the interface from the beginning.
• A heatmap-first analysis view with optional custom charts instead of overwhelming users with every possible plot.
• A modular codebase that can be extended with new statistical tests and visualizations.

What we learned

Good data analysis UX is not only about adding more statistics. It is about choosing sensible defaults, explaining uncertainty, preserving trust in the source data, and giving users a clear path from a pattern to the underlying numbers.

What's next for DataInsight

We plan to add time-series analysis, report export, richer dataset templates, partial correlations for confounder control, and optional assisted explanations while keeping the core calculations transparent and reproducible.

Built With

• codex
• gpt-5.6
• pandas
• plotly
• python
• scikit-learn
• scipy
• statsmodels
• streamlit