My Pet Goose

My Pet Goose

Inspiration

Globally, at least 2.2 billion people live with near or distance vision impairment. We wanted to explore how robotics and conversational AI could help make independent mobility more accessible.

Traditional mobility tools can help users detect obstacles, but we wanted to create something that could provide more support. We asked ourselves:

What if a mobility aid could guide you, speak with you, understand where you want to go, and read your surroundings aloud?

That idea became My Pet Goose, a conversational robotic guide designed to help lead visually impaired users while providing information about the environment around them.

What it does

MyPetGoose combines robotics, computer vision, navigation, obstacle detection, and conversational AI into one system.

A user can speak naturally to the robot and give commands such as:

“Take me to Room 204.”

MyPetGoose can understand the request, begin guiding the user, provide spoken directions, detect obstacles in its path, and stop when it identifies a possible hazard.

The camera can also recognize text in the environment to read room numbers, door labels, and other visible text aloud.

Our goal was to make the system simple to use. The user does not need a joystick, complicated controls, or a screen to interact with the robot.

How we built it

We divided MyPetGoose into two main systems: an Android-based software system and an ESP32-based robotics system.

The Android application acts as the main interface and handles:

Conversational AI
Text-to-speech
Camera-based text recognition
GPS Navigation
Communication with the robot

The physical robot uses an ESP32 microcontroller, 2 motor drivers, 4 drive motors, and a Samsung S21 mounted at the head. The ESP32 receives movement commands from the phone and controls the motors.

Challenges we faced

One of our biggest challenges was connecting the software and hardware into one reliable system.

Many of the individual components could work on their own, but connecting voice recognition, AI responses, motor control, obstacle avoidance, GPS navigation and computer vision created several integration challenges.

What we learned

Building MyPetGoose taught us how several areas of engineering and computer science can work together in one project.

Throughout the project, we worked with:

Android development
Embedded C++
PWM Motor control
Computer vision
Speech recognition
Conversational AI
Automatic Navigation
Hardware and software integration

One of the biggest lessons we learned was that building individual features is only part of the challenge. The harder part is connecting those features and making them work together reliably in a physical system.

What's next

With more time, we would improve MyPetGoose's understanding of its surroundings using phone-based depth sensing. We would also improve autonomous path planning, and develop more accurate indoor navigation.

We would also like to test MyPetGoose with visually impaired users and use their feedback to improve how the robot communicates and guides people.

MyPetGoose started with a simple question:

What if a robotic mobility aid could do more than detect obstacles? What if it could guide you, communicate with you, and help you understand the environment around you?