Sketchify

Inspiration

The inspiration for Sketchify came from the idea of seamlessly blending the physical world with digital art. Many artists struggle with transforming real-world objects or scenes into a precise drawing. We wanted to create an app that would make this process simple and intuitive by scanning real-world images and transforming them into clean, outline-style drawings with just a click.

What it does

Sketchify is an innovative app that allows users to scan real-world objects or scenes through Snap Spectacles and convert them into outline sketches. By leveraging AR techniques, the app detects the contours and details of the image, providing users with an editable sketch that they can use as a base for further creative work. It’s perfect for artists, designers, and anyone who wants to quickly turn their photos into drawings or use outlines for projects.

How we built it

We built Sketchify primarily in Snapchat Lens Studio. We used Python for the backend, leveraging libraries like OpenCV for image manipulation and contours detection, and FastAPI for creating the web application. The app's core functionality revolves around scanning the image, detecting edges using the Canny edge detector, and providing users with a downloadable or editable outline which is redisplayed in the Snapchat Spectacles environment.

Challenges we ran into

One of the main challenges we faced was working with Lens Studio for the first time. The platform presented a steep learning curve, as we had to familiarize ourselves with its tools and workflows quickly to integrate image processing effectively. Additionally, we initially experimented with a variety of techniques to generate the outlines, including AI tools like Gemini, but found that these methods didn’t always produce the desired results. After some trial and error, we realized that going back to the basics and utilizing the Canny edge detection algorithm provided the most accurate and reliable outlines for our app. This process taught us the importance of sticking to tried-and-true techniques when working with complex image data.

Accomplishments that we're proud of

We are particularly proud of building a reliable API from scratch that efficiently handles image uploads, processes them, and returns the desired outlines. This was a key component of the app's functionality, and ensuring it worked seamlessly was a major accomplishment. Additionally, we successfully navigated the Lens Studio environment to capture the user's current environment in real-time and send the image to our API endpoint for processing. This integration allowed us to bridge the gap between augmented reality and image processing, enabling a smooth user experience. These technical achievements were central to bringing our vision for Sketchify to life.

What we learned

Throughout this project, we spent a lot of time reading documentation to understand the various tools and libraries we were using, especially with Lens Studio. Diving deep into its features and capabilities taught us a lot about AR development and how to manipulate real-time camera feeds for image processing. We also gained hands-on experience with APIs, learning how to efficiently manage image uploads, processing, and responses. Overall, the project enhanced our problem-solving skills, especially in navigating new platforms and tools to integrate them into a cohesive app.

What's next for Sketchify

The next step for Sketchify is to enhance the outline generation algorithm to support more advanced features, such as the ability to customize the level of detail in the outlines or add color to the sketches. We also plan to improve the app’s performance by optimizing the backend image processing to handle even larger files. In the future, we may add features like a mobile version of the app, integration with design tools, and real-time collaborative sketching for users working together on a project.

Built With

• fastapi
• lens-studio
• python
• snapchat
• typescript