Aster - AI Research Scientist

This is your personal AI Researcher that will fetch related research paper, plans a hypothesis and experiments and executes the experimennts on distributed devices to produce results, suggestions.

Comment

Inspiration

When doing ML experiments, I faced a difficulty that I have to manually do everything from finding papers, researching them to code and run experiments and then make result reports. As we do different experiments our work become cluttered and makes it difficult for us to find resulting metrics at the end. In addition to this doing experiments on low end laptops is very difficult due to lack of ram and gpu. This inspired me to build this project, a personal AI Reseach Scientist, which keeps a track of all experiments, logs, allow you to configure number of epochs, experiments etc while running experiments on multiple devices. You can connect other laptops or pc and DMLF will use all devices to run experiments.

What it does

This project takes your question and allows you to configure research loop in which you can configure number of experiments, epochs, seed, DMLF nodes. You can also search for related papers and select them to include them into research plan. After this you can preview plan and run research loop. Now experiments will be divided to available devices as jobs. After training is completed it aggregrates results, show results.

How we built it

We built Distributed AI Research Scientist as a multi-agent research workflow that turns a research question into a literature brief, falsifiable hypothesis, bounded experiment plan, distributed execution, analysis, and an auditable report.

The product uses a Node.js backend and dashboard with native HTTP APIs, JavaScript agent orchestration, asynchronous worker scheduling, paper search through Semantic Scholar/OpenAlex, and optional OpenAI Responses API support. It persists runs, manifests, paper metadata, experiment plans, timelines, and reports as reproducible research artifacts.

Its Python Distributed ML Framework uses PyTorch, PyTorch Distributed Data Parallel (DDP), Gloo for CPU-based distributed communication, Torchvision for MNIST and synthetic data, gRPC with Protocol Buffers for cluster communication, SQLite for durable node/job state, and PowerShell automation for persistent multi-device Windows setup. Node agents run across heterogeneous LAN devices, with live heartbeats, asynchronous job polling, cancellation, unique rendezvous ports, network-interface configuration, real metric collection, and structured centralized logs.

Challenges we ran into

The hardest part was reliable multi-device training on Windows. We encountered GPU capability mismatches, long-running synchronous requests, DDP rendezvous port conflicts, firewall/security-suite interference, stale processes, incorrect network-adapter selection, and ranks completing out of order. We redesigned execution around asynchronous jobs, added polling and cancellation, surfaced worker errors in the dashboard, assigned unique rendezvous ports, added bounded DDP timeouts, persisted node network settings, and ensured every DDP node prepares its own local dataset copy.

Accomplishments that we're proud of

  • Completed a real two-device PyTorch DDP synthetic training run.
  • Collected real distributed metrics, marked separately from simulated/demo metrics.
  • Built an auditable research timeline from question through report.
  • Added paper approval before literature synthesis.
  • Added a plan critic, benchmark provenance, prompt/model metadata, and reproducibility manifests.
  • Created a DMLF configuration dashboard with live node registry, job history, progress, cancellation, and persistent startup scripts.
  • Added a real MNIST distributed benchmark path that records runtime dataset and framework provenance.

What we learned

Agentic research systems need strong boundaries: hypotheses must be falsifiable, experiments must be budgeted, and reports must clearly separate real evidence from simulated demonstrations.

Distributed ML is not only a training problem—it is also a networking, observability, reproducibility, and recovery problem. Clear live logs, durable job state, cancellation, and per-node configuration were essential to making the system trustworthy.

What's next for AI Research Scientist

  • Run repeated real MNIST experiments across two devices and report mean/variance.
  • Add richer citations and literature-grounded experiment recommendations.
  • Add early stopping, adaptive experiment prioritization, and automatic follow-up hypotheses.
  • Expand from MNIST to CIFAR-10 and limited user-provided medical-imaging pipelines. Improve the dashboard with richer cluster visualization and experiment-comparison charts.
Share this project:

Updates