Art Styles

Inspiration

With the increasing popularity of augmented reality technology, the potential for educational and interactive experiences is vast. In this project, the goal was to bring the excitement of art styles to life by allowing users to detect and classify them through their camera.

What it does

This lens is an augmented reality experience that allows users to detect and classify 10 different art styles through their camera. It uses a deep learning multi-classification model to detect the art style and 10 deep learning style transfer models to apply the corresponding style to the camera feed in real-time and shows the user 16 famous paintings and famous artists of the detected art style. The lens enables users to explore and learn about different art styles in an interactive and engaging way.

How we built it

Data Collection and Pre-processing: A large dataset of images representing different art styles was collected and pre-processed to create a suitable input for the deep learning model.

Model Training: The multi-classification model was trained on the pre-processed data to detect the art style. The style transfer models were also trained on separate datasets to generate the corresponding style for each art style.

Integration with AR Technology: The trained models were integrated with Lens Studio, to allow users to detect and classify art styles through their camera.

User Interface Design: The user interface was designed to provide a seamless and intuitive experience for users to interact with the lens.

Testing and Deployment: The lens was thoroughly tested on various devices and platforms to ensure optimal performance and compatibility. The final version of the lens was then deployed on Snapchat's platform for users to access and use.

This process required a good understanding of deep learning concepts and techniques, as well as experience with AR technology and Lens Studio. The lens was built using programming languages such as JavaScript, Python and programming tools such as TensorFlow and Keras.

Challenges we ran into

Data imbalance: Art styles may not have an equal number of images, leading to an imbalance in the training data, which can negatively impact model performance.

Computational resources: Deep learning models can be computationally intensive and require large amounts of memory and processing power, which can be challenging for mobile devices.

User interface design: Designing a user-friendly interface that provides a seamless and intuitive experience for users was a challenge.

Testing and deployment: Thoroughly testing the lens on various devices and platforms to ensure optimal performance and compatibility was a challenge.

To overcome these challenges, various techniques were used, such as data augmentation to balance the training data, model optimization to reduce computational resources, and user testing to ensure a user-friendly interface.

Accomplishments that we're proud of

Positive impact on education: The lens has the potential to provide a fun and interactive way for users to learn about different art styles, which can have a positive impact on education.

Creation of a unique and innovative AR experience: The lens combines the power of deep learning and AR technology to create an engaging and educational experience for users.

Advancement in deep learning techniques: The lens showcases the potential of deep learning techniques, such as multi-classification and style transfer, in AR applications.

User-friendly interface: The lens provides a user-friendly and intuitive interface that allows users to easily interact with and explore different art styles.

What we learned

Deep learning techniques: The project provided hands-on experience in deep learning techniques, such as multi-classification and style transfer, and a deeper understanding of the potential of these techniques in AR applications.

AR technology: The lens development process taught me about the technical challenges and considerations involved in integrating deep learning models with AR technology.

User-centered design: The lens development process emphasized the importance of user-centered design in creating a seamless and engaging experience for users.

What's next for Art Styles

Expansion of art styles: The lens could be expanded to include a wider range of art styles, providing users with a more comprehensive educational experience.

Improved deep learning models: The deep learning models used in the lens could be further optimized and improved, leading to increased accuracy and performance.