Astro Vision

• 

Inspiration

The vastness of space has always been the ultimate frontier, yet scientific discovery remains locked behind the doors of massive space agencies. We were inspired by the concept of Democratized Science (DeSci)—the idea that a student with a smartphone or a backyard astronomer could contribute to humanity’s knowledge of the universe. We wanted to create a "Digital Spaceship Cockpit" where AI doesn't just answer questions but acts as a persistent research partner, helping users find things that have changed in the cosmos over the last 30 years.

What it does

AstroVision is a multimodal, interactive 3D platform that bridges the gap between amateur observation and professional astrophysics.

AI Observation Lab: Users can upload or "auto-scan" images through their camera. The system solves the celestial coordinates (RA/Dec), fetches historical 1990s data from NASA archives, and performs a pixel-level comparison to detect "New Observations" like supernovae or asteroids.

AstroSage-8B Agent: A specialised AI model that provides expert scientific reasoning and remembers your past discoveries using Unibase long-term memory.

3D Particle Playground: A GPU-accelerated sandbox where text and images are manifested into 6,000 interactive stardust particles.

Hand-Sensor Interaction: Using MediaPipe, users physically "touch" and manipulate the universe. Your hands act as magnetic attractors, allowing you to poke, stretch, and expand the particle fields.

Agentic DAO: Integrated with BNB Chain and AEON, the platform features an autonomous economy where the AI facilitates micro-payments (x402) for high-latency scientific solves and rewards users for verified discoveries.

How we built it

We built AstroVision using a sophisticated "Multi-modal Pipeline":

3D Engine: Three.js and GLSL Shaders were used for high-performance rendering. We implemented GSAP for fluid particle morphing and custom shaders for the atmospheric glow of the floating 3D Earth.

The Brain: We integrated AstroSage-8B and BLIP via Hugging Face server-less inference.

Scientific Pipeline: We built a Node.js backend that interfaces with Astrometry.net for coordinate solving and NASA SkyView for historical imagery.

Computer Vision: MediaPipe Hands provides the 21-point tracking for gesture control. We engineered a "Stability Gater" to detect when an object is held still in front of the camera for auto-scanning.

Infrastructure: Jimp and Pixelmatch handle server-side image processing, while Unibase provides the persistent memory layer and AEON x402 facilitates AI-native payments.

Challenges we ran into

Building a bridge between scientific data and real-time visuals was no easy task:

C++ Build Bottlenecks: Initially, we struggled with node-canvas installation errors due to C++ compiler requirements. we overcame this by pivoting to Jimp, a 100% JavaScript image processor, ensuring the backend was portable and stable.

High Latency: Scientific solving can take up to 60 seconds. To keep the user experience "modern," we designed a HUD with "Scanning" animations and "Neural Sensor Logs" to provide feedback while the background processes ran.

GPU Performance: Animating 6,000 particles in JavaScript caused frame drops. we solved this by moving the entire morphing and magnetic interaction logic into GLSL Vertex Shaders, allowing for a buttery-smooth 60FPS.

Accomplishments that we're proud of

Fluid Morphing: Successfully creating a system where stardust physically flows from a 3D sphere into the literal letters of a user's prompt or the colors of an uploaded galaxy.

The "Aha!" Moment: Building a functional "Discovery" engine that can actually pinpoint a discrepancy between a user's photo and a 1990s NASA survey.

Tactile Interaction: Making the hand-tracking feel "physical"—when you reach toward the camera, the particles react with depth-aware repulsion.

What we learned

DeSci Incentive Models: We learned that decentralized science isn't just about data; it's about creating a persistent identity for researchers. Using Unibase, we saw how an AI that "remembers" your history becomes a much more effective tool.

GPU vs CPU: We gained deep experience in offloading heavy math to the GPU, realizing that for a truly "modern and simple" UI, shaders are non-negotiable.

Multimodal Routing: Handling Text, Image, and Live Video concurrently taught us how to build robust branching logic in React.

What's next for AstroVision

On-Chain Attestation: We plan to allow users to mint their discoveries as "Research NFTs" on the BNB Chain, creating a permanent record of their contribution to science.

Autonomous Agent Research: Expanding the AEON integration to allow AstroSage to "hire" other AI agents to perform deep-sky cross-referencing autonomously.

Mobile AR Cockpit: Bringing the HUD experience to mobile AR, allowing users to point their phones at the real night sky and see the "Discovery Overlay" in real-time.

Community Observatory: Using the DAO treasury to rent time on professional robotic telescopes, giving every AstroVision user access to "Physical Sensor" data from the world's most powerful lenses.

Built With

• astrometry
• astrosage
• css
• gemini
• goggleaistudio
• huggingface
• javascript
• kimi
• react