💡 Inspiration The journey of a software engineer is often a solitary one, filled with late-night debugging and complex documentation. As an older daughter and an engineer, I realized that while the internet is full of information, it lacks human connection. I was inspired by the idea of bringing a mentor out of the screen and into the real world—a guide who doesn't just provide answers but reacts with personality and empathy. My goal was to turn the cold, static experience of a chatbot into a "neat and clean" holographic presence that feels alive in your room.
✨ What it does ~ SoulLink AR is an emotionally intelligent Augmented Reality mentor designed to assist developers in real-time. ~Interactive Presence: Using ARCore, a professional 3D mentor is projected into the user's physical space. ~Voice-First Design: Users can speak naturally to the mentor using integrated Android voice recognition, moving beyond tedious typing. ~Dynamic Emotions: The mentor isn't just a puppet; she has a "soul." She smiles when she helps you, looks pensive while "thinking," and expresses genuine disappointment (Sad motion) if the server is unable to deliver an answer. ~AI-Powered Brain: Driven by the Gemini Flash API, she provides expert-level guidance on complex coding problems.
🛠️ How we built it The project was developed using a robust full-stack AR architecture: • Engine: Built in Unity 6 using the ARCore plugin for stable world-tracking. • Avatar System: Custom professional avatars were designed via the Ready Player Me hub and integrated through their specialized Unity SDK. • Intelligence: We utilized UnityWebRequest to communicate with the Gemini 2.0 & 1.5 Flash models. • Logic: C# scripts were used to bridge the gap between AI text and physical character behavior. For example, the probability of a successful response $P(S)$ vs. a server error $P(E)$ determines the animation state:
• Animations: Professional motions were retargeted using a Humanoid Rig to ensure smooth, realistic gestures.
⚠️ Challenges we ran into The path to a "neat and clean" project was paved with technical obstacles: • The "Shitty" API Era: We faced frequent 503 Service Unavailable errors from the Gemini servers. Instead of letting the app feel broken, we turned this into a feature by creating a "Sad" state where the mentor apologizes for her "busy brain," maintaining the user's immersion. • Memory Walls: During the Android build phase, we repeatedly hit LLVM out of memory errors. We solved this by optimizing the IL2CPP build configuration to "Faster (smaller) builds" and manually managing the Bee library artifacts. • Body Distortion: Early animations caused "messy" body twisting. This required deep-diving into Unity's Rig settings to correctly map every bone to the Humanoid standard.
🏆 Accomplishments that we're proud of • Emotional Resilience: Creating a system that handles technical failure with emotional intelligence rather than a simple "Error 404" text. • Neat & Clean UI: Designing a transparent, modern interface that complements the AR experience without cluttering the user's view. • Audio Integration: Successfully implementing a voice-recognition trigger that makes the interaction feel like a real conversation.
📚 What we learned This project was a masterclass in Technical Problem Solving: • AR Optimization: Learned how to balance high-fidelity 3D models with the performance constraints of mobile hardware. • API Architecture: Mastered the use of JSON parsing and asynchronous coroutines to keep the app responsive while waiting for AI data. • User Experience: Discovered that a mentor’s "personality"—her smile and her audible greeting—is just as important as the accuracy of her code.
🚀 What's next for SoulLink AR The bridge is just beginning. Future updates will include: • Persistence: Allowing the mentor to remember past conversations using local data storage. • Shared AR: A mode where multiple developers can see and interact with the same mentor during a team sprint. • Haptic Feedback: Using phone vibrations to mimic the "tap" of a mentor when the user makes a logic error in their code.
Log in or sign up for Devpost to join the conversation.