Doraemon’s Dorayaki

cover
image-target (please put it horizontal)
Dorayaki image-target (please put it horizontal)

Inspiration

Let's "Explore the Outside World Together!" The term "The Outside World" refers to people living outside a specific place or belonging to a specific group, or things that are common in everyday life but not part of your own experiences, like a picnic! We believe that the "Outside World" is a place full of imagination and magic, and we aim to connect the real world with the imaginative outside world using AR.

What it does

We have used creative props inspired by the famous Japanese cartoon character Doraemon to design our story background and 3D models. By scanning the target image of Doraemon's favorite food, Dorayaki, AR players can trigger a 3D cartoon room with Doraemon and Nobita, bringing them into a real-life scenario. They can freely explore the "outside" world using their smartphones and search for exciting props within the room. They will find delicious 3D Dorayaki props in the room. When they tap on a Dorayaki, it will duplicate into a new one and pop back into the player's real world. Players can enjoy adorable 3D Dorayaki in their real surroundings.

How we built it

We first constructed the room model and Dorayaki model, then utilized the renderer version 1.1.0 of A-Frame from 8th Wall, along with the min.js files of A-Frame Extras library version 4.2.0 from cdn.8thwall.com and A-Frame Physics System library version 4.0.1. We implemented the main functional modules named 'physics-image-target' and 'tap-to-play-sound'. These components define the behavior of handling physics-based image targets and playing sounds upon tapping. We also utilized features such as light, shadow, position, animation, @keyframes fade-in, and more.

Challenges we ran into

Disappearing portal functionality model: We resolved this issue by placing the target model within an area resembling a "box" composed of six walls.
Model disappearing during image tracking: We addressed this problem by using the project template from A-Frame: Image Targets Flyer.
Combining portal and image tracking functionalities failed. Issues with inconsistent texture mapping, face distortion, and misalignment during scan localization.

Accomplishments that we're proud of

Generating duplicate Dorayaki upon tapping an object, with randomized initial velocity in terms of size and direction.
Beautifully designed models.
Combining the image of a well-known IP character, leveraging the features of AR to create an engaging and unique concept.
Perfect sound effects.

What we learned

Throughout the project development, we gained knowledge about JavaScript features, the flexibility of using A-Frame modules, and mastered the development techniques of WebAR.

What's next for Doraemon’s Dorayaki

Enriching the interactive flow, such as implementing image tracking through scanning cards, creating portals by tapping on empty space, teleporting into the room, adding flashlight effect using the phone, and more.

The appearance of the main Dorayaki character and related sound effects.
Generating more Dorayaki in the real world by continuously tapping on any Dorayaki.
Introducing the appearance of a magnifying glass and related sound effects.
Using a magnifying glass in hand to enlarge the Dorayaki.
The appearance of a bamboo dragonfly and related sound effects.
Using a bamboo dragonfly in hand to fix the Dorayaki in the air.
Realistic effects of Dorayaki flying without gravity.
The appearance of a drawer with related sound effects, combined with VFX effects.
Emitting light and shrinking the portal when approaching the drawer, combined with VFX effects.