Lucky Stars

Inspiration

Humanity has always sought to navigate time, space, and stories through the stars, seeking answers about our place in the universe. Throughout our lives, we look to the ever-present, constantly shifting stars and search for familiar patterns. What if there was a way to map your memories as constellations formed from the stars that were present in that time and space?

Whether you're a traveler recording your adventures, a group of friends making a unique memory book, or just a lover of stars, we built Lucky Stars to offer a new way to explore the moments that mean the most.

What it does

Lucky Stars allows you to upload pictures and convert them into beautiful constellations that are mapped onto the real stars present at the time and place specified. You can navigate through time and space using a timeline and a 3D globe feature, which let you visit the memories stored in each instance. You can also give each image a name and short description and then view this information, the original photo, and the brightest stars in the constellation created from it. You can interact with the full star map in both a real view and a simplified view and click on various astronomical bodies to find out more about them.

How we built it

We built off of the open-source system Aladin-Lite which provided the basis of the star map. We used the Hipparcos star catalog to get information about the visible stars at each point in time and space and to map the constellations generated onto real stars. For turning images into constellations, we use a multi-model process:

Semantic segmentation: SegFormer
Identify subject: RMBG-1.4 to auto-identify or SlimSAM to click to cut out
Identify silhouette points: Moore neighborhood walk to outline subject and sample points. Visvalingam–Whyatt ranks points using triangular area.
Identify interior points: Shi–Tomasi corner detection for interior features
Feature identification: OWL-ViT zero shot model to identify critical interior features, e.g. eyes
Outline: Canny edge detection
Star Selection: Prioritize semantic features then outline points, then interior corners.
Line Generation: Outline vertices form a loop, relative neighborhood graph algorithm connects remaining points.

We used three.js to create the 3D globe that can be used for positioning.

We used a React Framework to structure our project.

Finally, we made extensive use of Cursor to create this project.

Challenges we ran into

Our main challenge lay in fine tuning the segmentation models to focus on the correct features and be able to generate constellations that would be recognizable to the human eye. We had some issues with overlapping constellations as well and choosing how much to prioritize brighter/more well known stars over choosing the stars that could create the best picture. We also spent a lot of effort making the UI intuitive and providing a seamless user experience.

What we learned

We learned how to scope out a project with a very large-scale with many possibilities to explore into a manageable hackathon project. We also learned how to collaborate effectively with Cursor, knowing how to write clear prompts, guide overall structure, break down tasks, and keep ourselves in the loop. We also learned a lot about segmentation models and how we can layer many different object-detection models to provide a complex output. Finally, we learned about the Aladin-lite project and spent much time exploring the astronomical bodies we could find using it.