AstroAI

Meet Atlas, your AI co-pilot for your telescope. Powered by Gemini, Atlas helps you to plan your session, identify objects in the sky, and narrate the history of the universe as you observe.

Comment

AstroAI: Your AI Copilot for the Night Sky

Inspiration

It all started in my backyard in Neuquén, Argentina. Two things I love are computer science and astronomy. I've been an amateur astronomer since I bought a telescope at 15.

One night, while struggling to identify an object in the eyepiece and thinking about what I could do for the hackathon, the lightbulb finally went on: What if I could give my telescope an AI brain?

Stargazing is magical, but it has a steep learning curve. You see "stars," but you don't always know what you are seeing. Existing tools like Stellarium are great but can be complex for beginners or hard to use in skies with a lot of stars. I realized that a multimodal AI agent could bridge that gap, analyzing what I see in real-time and narrating the cosmos like a personal Carl Sagan.

I also wanted to solve the "analysis paralysis" of planning an observation session. Instead of spending hours checking star charts, I wanted an agent that could curate a perfect night of observing in minutes. That’s how AstroAI was born: from a desire to make the universe understandable and deeply inspiring for everyone, and to make a tool that every amateur astronomer would love to have.

What it does

AstroAI brings astronomy closer to people through Atlas, an intelligent agent powered by Gemini 3.

  • Real-Time Multimodal Analysis: Users can point their camera (or upload an image from a telescope) at the sky. Atlas doesn't just identify objects; it performs a deep analysis, combining visual data with astronomical coordinates to explain exactly what you are looking at—be it a nebula, a galaxy, or a cluster—with inspiring, high-quality audio narration.
  • Smart Observation Planning: Atlas acts as an expert guide. By asking about your equipment, location, and experience level, it generates a tailored observation plan, suggesting the best targets for your specific night and gear.
  • Conversational Astronomy: Beyond tasks, Atlas is a knowledgeable companion ready to debate astrophysical theories, explain phenomena, or simply chat about the wonders of the universe, always with an educational and engaging tone.

How I built it

Backend - The Atlas Agent

Atlas is a multi-agent system built with the Google Agent Development Kit (ADK), communicating via the Agent-to-Agent (A2A) protocol.

  • The Root Agent: Handles user interaction, orchestrating requests to specialized sub-agents. It features tools for web search and image plate-solving.
  • The Planning Agent: specialized in astronomical calculations. It uses tools to determine visibility, search astronomical catalogs, and map natural language locations (e.g., "Neuquén") to precise coordinates.

The "Grounding" Pipeline for Image Analysis

The image analysis process combines Gemini’s vision capabilities with traditional astronomical methods to "ground" the agent and maximize identification accuracy. The specific workflow is:

  1. Astrometric Calibration: First, we perform plate-solving on a remote server to generate a precise equatorial coordinate map of the uploaded image. To avoid the high waiting times of public servers like Astrometry.net, a deployment of the Astrometry software has been made on a private server hosted on Google Cloud.

  2. Star Identification (Deterministic): We use contrast detection algorithms to locate the brightest point sources (stars). Using their calculated equatorial coordinates, we query the SIMBAD catalog to positively identify these specific stars.

  3. Deep Sky Structure Analysis (Generative): Since traditional contrast detection often fails with faint, diffuse objects like nebulae or galaxies, we send the image—along with the plate-solving data and the list of identified stars—to Gemini 3 Pro. We task the model with identifying the main astronomical structure in the frame, using the provided context to accurately distinguish complex objects.

  4. Narrative Synthesis: Finally, all identified objects and their rich astronomical metadata (magnitude, distance, angular size, spectral type, B-V color index, etc.) are fed into Gemini 3 Flash. The model synthesizes this data into a cohesive, inspiring story designed to evoke curiosity in the user. This text is then processed by Gemini TTS, allowing users to listen to the narration without taking their eyes off the telescope.

Frontend - The AstroAI App

Built with Flutter, allowing us to deploy both a high-performance mobile app (crucial for controlling camera features needed for astrophotography, such as exposure and ISO) and a web version for easy accessibility (ideal for the demo, as it doesn't require installation).

  • A2A and Chat Integration: Flutter's integration with the Agent-to-Agent protocol simplified the connection with the backend agents. It also provides ready to use LLM chat widgets, what simplified the development process.
  • Modular Experience: Capture, results, chat, and logbook are distinct flows, making the app easy to extend while staying cohesive.
  • Dark‑First Design: The UI is intentionally optimized for night use, reinforcing immersion while preserving night vision.
  • UI Design: I have used Google Stitch to craft a space-themed interface that feels modern yet functional in low-light environments.

Development flow - The AI power

For the implementation of both the front-end and back-end models, I have made heavy usage of the Antigravity IDE, primarily using the Gemini 3 Pro and Claude Code Opus 4.5 models.

The project followed the spec-driven-development methodology, leaving all project specifications in the pertinent AGENTS.md files.

Challenges I ran into

Teaching Gemini to identify stellar objects Initially, I tried to get Gemini to identify deep-sky objects using only the image. It was a struggle—the model often confused visually similar galaxies and globular clusters. I then tried attaching the raw equatorial coordinates, hoping that would be enough, but the results were still inconsistent. The breakthrough happened when I decided to "ground" the model with context it could actually relate to: the names of nearby stars (obtained via SIMBAD) and the specific constellation area. Once I fed Gemini this semantic context alongside the visual data, the accuracy became much greater.

The "No Telescope" Demo Problem I really wanted the judges to experience the thrill of controlling a live telescope, but the logistics were a nightmare. Remote telescopes are expensive, and leaving my own gear outside 24/7 wasn't feasible (plus, half the world is in daylight while I'm observing!). I briefly considered streaming a video from Stellarium, but using simulated images felt like it defeated the purpose of the project. Ultimately, I decided to integrate the Harvard MicroObservatory Image Directory. This allows anyone to test the analysis functionality with high-quality, real astronomical data, even if they don't have a telescope in their backyard.

Letting Atlas Take the Wheel My original vision was for Atlas to physically control the telescope using the LX200 protocol. Imagine the agent automatically executing observation plans or even hunting for transient events autonomously and participating in citizen science projects! However, reality hit hard: I couldn't motorize my mount with the necessary precision in time for the deadline, and renting a remote observatory was way out of my budget. I had to make the tough call to cut this feature so I could focus on polishing the core analysis experience, but it remains a top priority for the future.

Accomplishments that I'm proud of

Overall, I'm proud to have been able to implement the agent I wanted, going from the inspiration I had that night of observation to creating a tool that, as an amateur astronomer, is very useful to me, and I know could be very useful to other beginners as well.

  • Successful Integration: merging three distinct fields—Software Engineering, Amateur Astronomy, and Generative AI—was a big technical challenge. Seeing the system work as I had envisioned it was very satisfying.
  • Solving a Real Need: As an astronomer, I built the tool I wanted to use and that I believe other amateur astronomers could find useful as well. The result is not just a tech demo, but a genuinely useful assistant that enhances the observation experience.
  • Technical Execution: I integrated the Google ecosystem (ADK, A2A, Gemini 3, Flutter, Antigravity and Stitch) to build an end-to-end, ready to use application in a very short timeframe.

What I have learned

This project became kind of a practical crash course in the edge of AI and astronomy:

  • AI Ecosystem: I learned to use the Google ADK and understand the A2A protocol, which helped me orchestrate and deploy the Atlas agentic system.
  • Astro-Informatics: I dived deep into libraries like astropy and astroplan, and learned to programmatically interact with services like Astrometry.net and SIMBAD.
  • Prompt Engineering & Grounding: I learned that "more context is better." The difference between a hallucinating model and an expert astronomer agent lies entirely in how effectively you can ground the prompt with real-world data.

What's next for AstroAI

While the hackathon is over, my vision for AstroAI is just beginning:

  • Advanced Image Processing: I plan to improve the analysis pipeline to handle lower-quality images (common in amateur setups), correcting for issues like light pollution, trailing, or focus errors before they reach the AI.
  • Full Telescope Automation (GoTo): The dream of "Atlas, take me to the Orion Nebula" is still alive. I aim to implement the ASCOM/Indi protocols to allow Atlas to physically control telescope mounts, closing the loop between planning and observation.
  • Citizen Science Mode: This is the ultimate goal. I want to enable Atlas to operate autonomously over long periods, scanning specific sectors of the sky to detect variable stars or transient events. By leveraging Gemini's ability to detect anomalies in vast datasets, AstroAI could contribute to real scientific discoveries, turning every amateur telescope into a research station.

Built With

  • agent-to-agent-(a2a)-protocol
  • antigravity
  • astrometry
  • astroplan
  • astropy
  • claude-code-opus-4.5
  • fastapi
  • flutter
  • gemini-3-flash
  • gemini-3-pro
  • gemini-tts
  • google-agent-development-kit-(adk)
  • google-cloud
  • google-stitch
  • hive
  • python
  • simbad
  • sse
Share this project:

Updates