Visibot

Adding multiple functions to create order of operations
Creating your first function
After a while, your visual script can get large. Time to look into alternative methods.
Fail UX
Using if loops to keep your script and functions clean and easy to read

Inspiration

I have been an XR developer throughout my career, specializing in educational and training applications for clients across various industries. My passion is finding innovative ways to help users learn new skills. While volunteering for a kids' game show, I was tasked with developing a robot programming challenge, where kids used visual scripting to program a Lego-based robot.

I was inspired by the concept of using visual scripting to teach programming and wanted to make this experience accessible to a broader audience. However, the high cost of the robots and the lengthy process of uploading code to them were significant obstacles. I recognized the potential of using XR technology to overcome these challenges, making such educational activities more affordable and efficient. This realization motivated me to explore how XR could be used to replicate and enhance the learning experience.

What it does

The Visibot app is designed for users interested in coding, offering a fun and interactive way to learn. The app presents a series of challenges that teach basic coding concepts, requiring users to apply their knowledge through visual scripts to help Visibot complete each task.

Although the app is heavily gamified, its primary goal is to equip users with foundational programming skills. This approach prepares them for using advanced engines like Unity and Unreal, providing a clearer understanding of how these platforms function. Consequently, I submitted the app in the hobbies and skills category.

As users progress through the challenges, Visibot gains intelligence and unlocks new functions and methods for subsequent tasks. Initially, users learn the basic order of operations in a script similar to a start function with entry and exit points. They then advance to more complex concepts, such as if loops, which streamline their scripts and enhance readability.

How we built it

Using the Unity Engine, I aimed to leverage Meta's new building blocks for this challenge to see how much I could utilize them without needing to create my own basic functions. My goal was to design the challenges to be as object-oriented as possible, ensuring that everything the challenge required was part of the prefab.

I also wanted to create an app that relied entirely on hand tracking, which was new for me since I had previously depended heavily on controllers for more complex UX options. This shift required me to adapt to the nuances and limitations of hand tracking technology.

Challenges we ran into

The building blocks saved a lot of time but were limited in the mechanics they could quickly add to the app. I wanted to maintain ray-based interactions as the primary method for interacting with the UI because physical button presses often resulted in misclicks. However, the building blocks didn't support UI-based raycasting, so I had to set everything up by examining the demo scenes.

Additionally, there were no building blocks for interacting with the world mesh to place my anchor, so I had to implement that functionality myself.

As a solo programmer, I relied on free assets for some content, including the robot model, audio, and the shader used for loading the challenge.

Accomplishments that we're proud of

Creating the mechanics for the visual scripting was a real head scratcher for me. I had to write code that allowed the user to make code for the robot. I had to make sure that the functions behaved exactly as the user wanted and also in the correct order. There were some functions I had to dumb down as I was making my life a bit too difficult with the amount of time I had to work on this. But when it was finally done and I could send my visual script to the robot and see it do what I told it to do, I was over the moon

What we learned

I haven't made any applications for Meta's passthrough AR so being able to quickly set up the basics and learn what kind of UI/UX works and what just doesn't compared to VR was a great learning experience for me.

What's next for Visibot

This application was made during weekends but I would love to continue to work on this. I have so many more challenges I would love to create that use the cutting edge AR technology that could allow the robot to exit the constraints of the challenge environment and allow the robot to interact with the users surroundings instead.

I would love to create challenges that could teach more complex programming such as setting up basic AI to command multiple Visibots, using enumerators, using voice commands, or creating custom controller setups to give you control of Visibot.

All of these are heavily based on my free time and financials as I would be working on this on weekends and using my own money.