Blimpy

Inspiration

We first got the idea for Blimpy after seeing several papers and research projects about cute floating companion robots online. https://www.cnet.com/tech/floating-robots-safe-friendly-human-interaction/ https://arxiv.org/pdf/2504.01293 https://www.youtube.com/watch?v=mDCtz_BN-vc https://www.youtube.com/watch?v=oHikvpqlAEk

What it does

Blimpy is a floating helium balloon robot designed to detect and follow a person, maintain its position and altitude, and navigate to fixed positions in a room. Blimpy can also take spoken commands and converse with the people around it. For example, "Hey Blimpy, follow me!" or "Blimpy, can you help me out with my math homework?"

How we built it

We used a 1.1 m latex balloon, two ESP32 microcontrollers, motors, propellers, and some string.

There are two main parts: the balloon itself and the chassis, or gondola. The helium in the balloon provides lift. The chassis holds the breadboard, wiring, propellers, battery, weight and motors.

The hardware

We used four reversible motors that drive propellers, with each motor controlling a different part of Blimpy's movement. Two motors were mounted at the rear for forward and backward movement. Running them at different power levels also allowed Blimpy to turn. A third motor faced sideways to correct lateral drift, since even small air currents could push Blimpy around. The fourth motor was mounted vertically at the bottom of the chassis to control altitude. Because the balloon was slightly buoyant, this motor sometimes needed to push the gondola down instead of lifting it up.

All four motors needed to run in both directions. Once Blimpy was moving, it did not stop immediately when we cut the power and could continue moving for several metres. To stop it, we had to apply thrust in the opposite direction.

Power

We originally powered the system with a single 3.7 V battery. After changing the design to use two ESP32 boards, this did not seem to provide enough power for the whole system. We therefore added a battery holder with four 1.5 V cells.

During normal operation, we measured a little over 1 V across each motor. The completed gondola weighed approximately 800 g, although this was only an estimate because we did not have a suitable scale.

The software

The two ESP32 boards handled the motors, while a laptop made the higher-level decisions. The laptop connected to them through Bluetooth and sent velocity commands twenty times a second.

We made a physics simulator with mass, quadratic drag, thrust, gusts, drift, camera latency and packet loss so that we could experiment with different setups and control strategies.

The laptop runs YOLO on its camera to detect people. We also set up a mat of four AprilTags to give Blimpy a fixed reference for positioning.

For voice interaction, we run Qwen3.5-Omni in real time to listen to the microphone, look through the camera, talk with the user, and issue commands. We also run a local Qwen model through Ollama to decide when the Omni model needs to process the conversation.

Challenges we ran into

Controlling a soft body

We had trouble making Blimpy go where we wanted it to. Unlike a rigid drone, Blimpy has a large balloon with a gondola suspended underneath it by string. The balloon and gondola can swing and rotate relative to each other, so changing direction often causes the whole system to oscillate. This made it difficult to turn or change position precisely.

Voice in a noisy room

When talking to Blimpy, it was difficult for our models to differentiate between speakers and understand commands. Tuning the audio processing took a large amount of our time.

Wiring

The hardest bug of the weekend looked like a software problem for most of a day.

At first, we soldered all the components directly together. This worked for the first few tests, but it soon became difficult to debug. Whenever a connection failed, we had to desolder part of the circuit before we could test that component separately. We eventually moved everything onto a breadboard so that we could disconnect and test one subsystem at a time.

Our first design used one ESP32-C3 to control two H-bridge boards, with two motors connected to each board. During testing, two of the motors would not start on their own. However, they sometimes started if another motor was already running. We were using the same code and sending the same commands, but the results changed depending on which other motors were active.

We checked the pins with a multimeter, rewired the circuit several times, and asked our mentors for help. The supply voltages and control signals seemed correct, and all four motors worked when we tested them individually.

Eventually, we found that the problem was caused by the ground wiring. All four motors were returning current through the same path on the breadboard. Each motor driver reads its input signals relative to its own ground pin, but the motor current was shifting that reference enough to affect how the signals were interpreted. We rewired the circuit so that each driver board had its own return path to the battery negative. This solved the problem at the time, although a similar issue appeared again later as we added more wiring.

Two boards

We later replaced the original H-bridge boards with motor driver boards provided by the hardware lab. These boards were more reliable, but they also required more control pins than one ESP32-C3 could provide. Because of this, we divided the system between two ESP32 boards, with I2C.

After making this change, the master stopped advertising over Bluetooth, so we could no longer connect to it. We first thought this was another power problem, because BLE communication can draw current in short spikes. After more testing, we found that the problem was actually in the code.

Flying Blimpy

Testing on the bench only showed us that the motors could turn. The system behaved very differently after we inflated the 1.1 m balloon and suspended the gondola underneath it. Blimpy often drifted or rotated in directions we did not expect. Its response also changed depending on air currents and how level the gondola was hanging. The larger envelope gave us more lift, but it also created more drag because there was more surface area moving through the air.

After mounting the motors, we did not know exactly which direction each one would spin or how much thrust it would produce. We had to test them one at a time and manually record their direction and approximate strength. Even with the same command, the motors did not produce exactly the same result. This was probably due to differences between the motors, how far each propeller was pushed onto its shaft, and the angle at which each motor was mounted.

We calibrated the motors manually and divided the overall problem into smaller tasks. We started with the vertical motor and adjusted it until Blimpy could move both up and down. We then worked on forward movement, followed by turning and finally travelling over longer distances.

In each round of testing, we changed the power sent to one or two motors, observed the result, and recorded what happened. We repeated this process until Blimpy could move roughly in the direction we wanted. The control was still not very precise, but working on one movement at a time helped us identify which motor or parameter needed to be adjusted.

Accomplishments that we're proud of

Tracing an intermittent motor fault to a shifting ground reference, and fixing it with proper star grounding
Getting a suspended balloon and gondola to follow commands through manual calibration of each motor
Connecting a local language model and a cloud model to handle different parts of the voice interaction
Making Blimpy capable of moving autonomously and following instructions
Creating a really cute project!

What we learned

One of the main things we learned was to test hardware in smaller sections. Several problems initially looked like software bugs but were actually caused by wiring or power. Moving to a breadboard made it much easier to disconnect parts and test them separately.

We also learned that the simulator could only take us so far. Blimpy behaved differently once the gondola was attached to the inflated balloon, so we had to test frequently and adjust the design based on what we observed. The team abandoned our initial plan early and continued changing the design as new problems appeared.

What's next for Blimpy

We want to add more features that an assistant should have, such as reminders and the ability to answer questions about what it sees. We also want to improve the movement control so that Blimpy can hold its position and follow a person more precisely. Finally, we want to improve the audio processing so that Blimpy can understand commands in a noisy room with several people talking.