[MAR][C(offe)++]OdynAR

💡 Inspiration We wanted to bring posters to life using AR — imagine pointing your phone at a simple print and seeing it turn into an interactive digital portal. With the rise of hybrid experiences in events, museums, and education, we aimed to make static materials immersive and meaningful.

⚙️ What it does Our app detects a physical image marker (like a poster or flyer) and overlays a digital AR poster on top of it. The user can tap the poster to open a fullscreen version, creating a smooth transition between the physical and digital world. It’s intuitive, fun, and opens the door for creative storytelling through augmented reality.

🛠️ How we built it We used:

Swift and SwiftUI for the UI RealityKit and ARKit for image tracking and 3D rendering Xcode for development and testing The app runs on iOS and tracks a custom image from the user’s environment. When the marker is detected, a 3D plane is placed on top with a texture of the poster. Tapping the poster brings up a fullscreen overlay for a more detailed view.

🧱 Challenges we ran into Image marker setup in ARKit is surprisingly picky — we had to make sure our marker was big enough and had the correct physical dimensions. Getting textures to load dynamically and apply them in RealityKit was tricky. Making sure gesture recognizers and ARView didn’t conflict took some debugging wizardry. SwiftUI and UIKit interoperability for overlay handling added complexity.

🏆 Accomplishments that we're proud of Fully working AR experience with marker detection and real-time overlay Clean and responsive UI with tap interactions A working demo within a single day 🎉 Our first time using RealityKit, and we managed to integrate a complete end-to-end AR experience!

📚 What we learned How to work with ARKit and RealityKit for image tracking The difference between ModelEntity, AnchorEntity, and how to work with transforms UIKit and SwiftUI integration in an AR project Practical AR workflow and how to debug physical-world problems with digital solutions

🚀 What’s next for MAR[C(offe)++]OdynAR Support for multiple markers and interactive poster sets Add video, 3D models, or animations to posters Publish the app as an educational or promotional tool for events Explore markerless AR or LiDAR-based enhancements Add analytics to track poster interactions