StarGazer AI

StarGazer AI leverages MongoDB, ExpressJS, and the Flickr API to fetch night sky images, analyzes them with GPT-4o to trace and label constellations, then renders annotated images for users.

Comment

StarGazer AI

Home

What Does StarGazer AI Do? 🚀

Search for Night Sky Images

Using advanced LLMs, StarGazer AI retrieves images of the night sky. With our image randomizer, each click delivers a unique view, ensuring a fresh and exciting experience with every use.

Constellation Detection with GPT

Powered by GPT4o’s advanced image analysis capabilities, our system identifies constellations and highlights their names and locations within the image.

What Inspired Us

The inspiration for StarGazer AI stems from a fascination with space and constellations. There’s something magical about staring at the night sky, riddled with shining stars, and recognizing discrete constellations within it. This wonder drove us to create a tool that blends technology with this timeless curiosity, enabling others to explore and learn about constellations in a new, interactive way.

What We Learned

This project taught us the importance of integrating various technologies into a cohesive pipeline. We learned how to:

  • Leverage APIs like Flickr for retrieving high-quality, filtered images.
  • Use GPT-4o to perform image analysis and extract structured data.
  • Build efficient backend systems using MongoDB and ExpressJS.
  • Apply coordinate-based tracing and image annotation techniques for labeling constellations.
  • Work under tight time constraints while maintaining a focus on functionality and user experience.

How We Built It

  1. Backend:

    • Built with MongoDB and ExpressJS, our backend integrates the Flickr API to fetch images of the night sky.

  2. AI Analysis:

    • Used GPT-4o to analyze the retrieved images, detecting constellations and outputting a dictionary of their names and ordered coordinates for tracing.

  3. Image Annotation:

    • Developed a script to trace constellations by drawing lines between coordinate points and labeling them directly on the image.

  4. Front-End Rendering:

    • Rendered the annotated image in an interactive interface, allowing users to view the traced constellations with their labeled names.

Challenges We Faced

  • Time Constraint:
    With only 12 hours to complete the project, finding a dataset and training a custom model was not feasible. To meet the deadline, we relied on GPT-4o for constellation recognition.

  • Image/Data Acquisition:
    Using Perplexity wasn’t an option, and accessing the Google Search API wasn't possible. We settled on using Flickr, but finding high-quality night sky images with identifiable constellations was difficult.

  • Backend-Frontend Coordination:
    Integrating the pipeline seamlessly across different technologies (Flickr API, GPT-4o, and Javascript API) required careful handling of data flow and performance optimization.

Process

1. Obtaining the Perplexity API Token

  • Purpose: To access Perplexity's capabilities for retrieving high-quality night sky images.
  • Step:
    • Acquire an API token from Perplexity's developer portal.
    • Integrate the token into the project to authenticate requests for image retrieval.

2. Writing the API in JavaScript

  • Purpose: To interact with Perplexity and fetch tailored night sky images.
  • Steps:
    • Random Night Sky Image Selection:
    • Use the API to prompt Perplexity for a night sky image.
    • Include parameters to ensure the image:
      • Contains a minimum of x constellations.
      • Fits within the dimension constraints of 500x500 px to 2000x2000 px.
    • Randomization Logic:
    • Introduce randomness in the selection process for a fresh experience every time.
    • Image Retrieval:
    • Fetch the selected image and ensure it meets the required criteria.
    • Pass the image to the next stage of the pipeline for further processing.

3. Constellation Detection with GPT-4o

  • Purpose: To analyze the retrieved image and identify constellations.
  • Steps:
    • Image Submission: Provide the image to GPT-4o for processing.
    • Constellation Analysis:
    • GPT-4o locates visible constellations in the image.
    • Outputs a dictionary where:
      • Keys: Names of the identified constellations.
      • Values: Coordinate lists representing the constellation points.
    • Coordinate List Details:
    • Points are ordered sequentially to facilitate tracing lines between adjacent points.
    • Repeated coordinates are included to handle constellations that require retracing certain points.

4. Python Script for Image Annotation

  • Purpose: To trace the constellations and label them on the image.
  • Steps:
    • Input Parsing: Accept the dictionary output from GPT-4o.
    • Tracing Algorithm: Iterate through each constellation in the dictionary and draw lines between adjacent coordinates in the list to trace the constellation.
    • Labeling: Overlay the name of each constellation near its traced structure on the image.
    • Output Generation: Save the annotated image for display on the front end.

5. Front-End Image Display

  • Purpose: To provide users with a visually enriched and interactive experience.
  • Steps:
    • Image Display: Render the annotated image showing traced constellations and their names.
    • User Interaction: Allow users to explore the image, zoom in, and understand the traced constellations.
    • Explanation Section: Provide details about the identified constellations, including names and points of interest.

How It Works

StarGazer AI operates through a structured pipeline, integrating APIs and advanced machine learning models to deliver an interactive experience. First, the backend, built using MongoDB and ExpressJS, uses Flickr API to fetch high-resolution night sky images that meet our given constraints: the presence of at least 3 constellations and dimensions between 500x500 px and 2000x2000 px. This backend includes a randomizer for image selection to ensure unique results. The retrieved image is then passed to GPT-4o, which performs a detailed analysis, detecting constellations and generating a structured output in the form of a dictionary. The dictionary’s keys represent constellation names, while the values are ordered coordinate lists defining how to trace each constellation, including repeated points to accommodate retracing where necessary. Our script then consumes this output to draw lines between the coordinates and annotate the image with the constellation names. The final annotated image is then rendered on the front end for the user, providing an interactive display of the identified constellations. The display is complete with traced patterns and labeled names, creating a seamless blend of computational analysis and visual storytelling.

Future Improvements

Given the 12-hour constraint from concept to completion, we focused on building a functional pipeline leveraging existing APIs and models. However, we have ambitious plans to expand StarGazer AI's capabilities. In future iterations, we aim to implement an image upload feature, allowing users to upload their own night sky photographs for constellation detection. This would make the tool more interactive and user-centric. Additionally, we plan to train our own CNN-based constellation recognition model, tailored specifically to enhance accuracy and performance. This custom model would allow greater flexibility in handling diverse image types and environmental conditions, paving the way for StarGazer AI to become a robust and comprehensive platform for night sky exploration.

Built With

Share this project:

Updates