AutoPilot DS — Autonomous Data Scientist Agent

An autonomous AI data scientist that turns raw datasets into ML insights, model comparisons, and actionable reports.

Comment

Inspiration

As data scientists, we spend a significant amount of time performing repetitive steps such as inspecting datasets, identifying the target variable, preprocessing data, training multiple models, comparing their performance, and preparing reports.

We wanted to build something beyond an AI chatbot that simply explains how to perform these tasks.

This inspired us to build AutoPilot DS, an autonomous AI Data Scientist that combines agentic reasoning with real machine-learning execution. A user can upload a dataset, provide an objective, and let the system execute the core data-science workflow automatically.

What We Built

AutoPilot DS is a Streamlit-based AI application powered by Gemini 3.5 Flash and the Google Agent Development Kit (ADK).

The application follows an end-to-end workflow:

Upload Dataset → Inspect Data → Identify ML Task → Preprocess → Train Models → Compare Models → Evaluate → Generate Report

The user simply uploads a CSV dataset and provides an objective such as:

"Predict total_sales."

AutoPilot DS then:

  • Inspects the dataset structure and data quality
  • Identifies the target variable
  • Determines whether the problem is regression or classification
  • Handles missing values
  • Encodes categorical variables
  • Trains multiple machine-learning models
  • Uses cross-validation to compare models
  • Evaluates the selected model on a holdout dataset
  • Identifies the best-performing model
  • Generates an executive report containing the results

The Streamlit interface makes the entire workflow visible to the user, allowing them to see what the agent is doing instead of receiving only a final answer.

How We Built It

The project combines an AI reasoning layer, deterministic machine-learning tools, and an interactive web interface.

AI and Agent Layer

Gemini 3.5 Flash acts as the reasoning engine, while Google ADK provides the agent framework and orchestration.

The agent decides which actions need to be performed and calls specialized tools to execute them.

Machine Learning Layer

We use Python, Pandas, NumPy, and Scikit-learn for the actual data-science operations.

The ML pipeline includes:

  • Data inspection
  • Missing-value treatment
  • Categorical encoding
  • Feature preprocessing
  • Model training
  • Cross-validation
  • Model comparison
  • Holdout evaluation

For regression problems, the system evaluates metrics such as R², RMSE, and MAE. For classification problems, it evaluates metrics such as accuracy and weighted F1-score.

Application Layer

We built the user interface using Streamlit.

The dashboard provides a simple workflow where users can:

  1. Upload a CSV dataset
  2. Enter a prediction objective
  3. Start the autonomous workflow
  4. Observe dataset inspection
  5. View model comparisons
  6. See evaluation metrics
  7. Read and download the generated executive report

This makes the agent easy to demonstrate and useful for non-expert users as well.

What We Learned

One of the biggest lessons from this project was that an effective AI agent is not just an LLM with a detailed prompt.

The agent becomes much more useful when the LLM is connected to well-defined tools that perform real operations.

We learned how to combine:

LLM reasoning + agent orchestration + deterministic Python tools + interactive UI

We also learned the importance of reproducibility in machine learning. Instead of allowing an AI model to simply describe hypothetical results, AutoPilot DS performs the actual experiments and returns real metrics.

Another important lesson was designing the system so that AI reasoning and machine-learning execution remain separate. This makes the application easier to test, debug, and extend.

Challenges We Faced

One challenge was deciding how much autonomy to give the agent.

Datasets can contain ambiguous column names and different data types, so blindly allowing an AI model to make every decision can lead to incorrect assumptions. We therefore designed the workflow around an initial dataset inspection step.

Another challenge was preventing data leakage during preprocessing and evaluation. We addressed this by using Scikit-learn pipelines so preprocessing operations are performed correctly during model evaluation.

We also had to balance flexibility with reliability. Instead of attempting to support every possible machine-learning problem, the current version focuses on structured tabular datasets and common regression and classification workflows.

Finally, integrating an agentic backend with a user-friendly interface was an important design challenge. Streamlit provides the visual layer while the AI agent and Python tools handle the reasoning and execution.

Why AutoPilot DS?

AutoPilot DS is designed to reduce the repetitive work between:

"I have a dataset"

and

"I have a defensible machine-learning result."

The goal is not to replace data scientists. Instead, AutoPilot DS acts as an autonomous first-pass data-science teammate that can execute experiments, compare models, and produce evidence that a human can review.

Our future roadmap includes:

  • Automated feature engineering
  • Hyperparameter optimization
  • Feature importance and explainability
  • Experiment tracking
  • Data-drift monitoring
  • Model monitoring
  • Automated deployment recommendations

AutoPilot DS turns an AI assistant from something that explains data science into an agent that actually performs data science.

Built With

Share this project:

Updates

Submission history