ARES – Autonomous Rover Evaluation System

ARES — Real-time intelligence for autonomous rover missions on Mars.

Comment

ARES – Autonomous Rover Evaluation System

Inspiration

Space exploration has always fascinated me, but the engineering challenges behind it are what truly captivate me. The primary bottleneck in Mars exploration is the communication delay. It takes anywhere from 4 to 24 minutes for a signal to travel between Earth and Mars. If a rover encounters a cliff or a dust trap, it cannot wait for a human operator to say "STOP." It needs to think for itself.

I was inspired to build ARES to bridge the gap between Real-Time Stream Processing and Generative AI. I wanted to answer the question: Can I use a Large Language Model (LLM) not just as a chatbot, but as a real-time sensory cortex for a robot?

What it does

ARES is a mission control dashboard that simulates an autonomous rover pipeline. It ingests raw imagery from Mars, processes it through an AI "brain" to detect hazards, and streams telemetry back to Earth in real-time.

The system acts as a specialized geologist and safety officer, analyzing every frame to calculate:

  • Hazard Score (\(H\)): A quantitative risk assessment \(0-10\).
  • Scientific Value (\(S\)): A score determining geological interest \(0-10\).

Based on these metrics, ARES autonomously decides the rover's mode: SAFE (Proceed), REVIEW (Human intervention required), or CRITICAL (Emergency Stop).

How I built it

I designed ARES using an Event-Driven Architecture (EDA) to handle the asynchronous nature of space communications.

  1. Data Ingestion (The Uplink): A Python producer polls the NASA Mars Rover API to simulate a live feed. Images are pushed to an Apache Kafka topic (rover_uplink), keyed by the unique rover_id.

  2. The AI Cortex (Google Gemini): A consumer service reads these images and sends them to Google Gemini. I engineered a specific system prompt that forces the model to act as a rover navigation system:

    • Input: Raw Image.
    • Constraint: Ignore rover hardware (wheels, drills) when scoring science.
    • Output: Structured JSON containing scores and a terrain analysis summary.

  3. Real-Time Analytics (Apache Flink): To track safety trends over time, I used Apache Flink to process the stream. I implemented a Tumbling Window of 1 minute to calculate the moving average of the hazard score: $$ \bar{H}{t} = \frac{1}{N} \sum{i=1}^{N} H_i $$ This allows the system to detect if the rover is entering a progressively more dangerous region, even if individual frames seem safe.

  4. The Dashboard (React + FastAPI): The frontend is a sci-fi inspired "Mission Control" interface built with React and Tailwind CSS. It connects to the backend via FastAPI WebSockets, utilizing recharts for live telemetry plotting.

Challenges I ran into

  • The "Race Condition" of Distributed State: One of the hardest bugs involved the "History Review" feature. The image stream (Uplink) and the data stream (Telemetry) arrived via WebSocket at slightly different milliseconds. This caused history logs to sometimes save a placeholder URL instead of the real photo. I solved this by synchronizing the streams in the backend, ensuring the telemetry packet carried the image source Img_src as a payload.
  • Prompt Engineering for Hardware Bias: Initially, the AI assigned high "Scientific Value" scores to images dominated by the rover's own wheels or drill bit. It "saw" complex metal structures and thought they were scientifically interesting. I fixed this by implementing Negative Constraints in the system prompt, explicitly instructing the model that "Rover hardware has a scientific value of 0."
  • Stream Distribution in Flink: Our Flink SQL queries were initially failing to aggregate data correctly. I discovered that our Kafka Producer was sending null keys. Refactoring the producer to explicitly hash the rover_id as the message key solved the partitioning issue.

Accomplishments that I am proud of

  • Full End-to-End Pipeline: I successfully built a system where data flows from a NASA API -> Kafka -> Gemini AI -> Flink -> FastAPI -> React without interruption.
  • Time Travel Debugging: I implemented a "DVR-like" feature where clicking any data point on the live graph instantly restores the image, AI diagnostics, and system state from that specific moment in the past.
  • Resilient UI: I created a dashboard that handles high-frequency data updates without UI stuttering or layout shifting, maintaining a professional "Mission Control" aesthetic.

What I learned

Building ARES taught us that Generative AI is a viable component in streaming pipelines. It is not just for generating text; it can be used as a nondeterministic sensor that turns unstructured data (pixels) into structured telemetry (JSON).

I also gained a deep appreciation for State Management. Handling a continuous WebSocket stream while allowing a user to review history required a complex use of React Refs and immutable state updates to prevent memory leaks.

What's next for ARES – Autonomous Risk Evaluation System

  • Command Downlink: Implementing a feature to send commands from the dashboard back to Kafka to "Resume" a stopped rover.
  • Multi-Rover Support: Scaling the dashboard to handle multiple rover_id streams simultaneously (e.g., monitoring Perseverance and Curiosity at the same time).
  • Deployment: Dockerizing the entire bridge and frontend for easy deployment on cloud infrastructure.

Built With

Share this project:

Updates

Submission history