Inspiration
Our inspiration came from Alfred Pennyworth, Batman’s butler—the person who keeps everything running behind the scenes. We wanted to bring a little of that dependable, helpful presence into a household robot.
A.L.F.R.E.D pairs two robotic arms and a screen face with Mini Alfred, a small mobile companion that acts as its “third arm.” Together, they are our first step toward a robotic butler that can help with everyday tasks around the house.
What it does
A.L.F.R.E.D is a prototype platform for household assistance. The main robot has two robotic arms mounted on a wooden panel, a screen face, and a table that provides a consistent workspace for manipulation tasks.
Mini Alfred is its mobile companion. It accepts movement commands over Wi-Fi, uses an IMU to control turns, and has ultrasonic sensors facing forward and downward to detect nearby obstacles and changes in surface height. Its role is to extend Alfred’s reach beyond the workstation, with small transport and delivery tasks as our next step.
The foundation supports two complementary capabilities: manipulating objects at a fixed workspace and moving a separate helper between locations.
How we built it
The main robot uses two SO-101 robotic arms, cameras, and an NVIDIA Jetson Orin Nano Super as its onboard controller. Our software combines Python, PyTorch, Hugging Face LeRobot, and a custom inverse kinematics library.
We built around two approaches to arm motion:
- Action Chunking Transformer (ACT): an imitation-learning approach that learns from demonstrations and predicts sequences of joint actions.
- Inverse kinematics (IK): a geometric approach that converts desired gripper positions and orientations into joint configurations.
ACT provides a path toward learning manipulation behaviors, while IK gives us explicit control over planned movements. Our custom NumPy implementation uses the robot’s URDF model, coordinate transformations, joint limits, and trajectory timing to generate arm motion.
We configured CUDA-enabled PyTorch and benchmarked ACT inference on the Jetson. This lets us explore learned control on embedded hardware while keeping geometric planning and motor control in separate software layers.
Mini Alfred uses an Arduino Nano ESP32, two DC motors, an L298N motor driver, an MPU IMU, and two HC-SR04-style ultrasonic sensors. A Python client sends commands over Wi-Fi using UDP. Its firmware handles relative turns, experimental IMU-based distance estimation, motor-speed ramping, and sensor-triggered movement cancellation. Sensor acquisition runs in a background task so echo timeouts do not block the main control loop.
Challenges we ran into
Our biggest challenge was matching our ambition to the hardware and the hackathon deadline.
Originally, Mini Alfred was supposed to carry the entire robot as a mobile base. Once we mounted the arms and upper structure, the weight and high center of gravity caused instability. Shortening the structure improved stability, but the loaded base still struggled to turn.
With limited time remaining, we scoped down from a mobile manipulation robot to a stationary workstation with a separate mobile helper. That meant rebuilding the physical arrangement and revising how the robots would divide responsibilities.
We also worked through servo calibration, camera bandwidth limits, CUDA and PyTorch compatibility, IMU noise, turning overshoot, and ultrasonic timing issues. A sensor working in a standalone example did not automatically mean it would work inside a system handling Wi-Fi, motor control, and IMU sampling simultaneously.
Accomplishments that we're proud of
We are proud of turning a mechanical limitation into a workable two-robot architecture within the hackathon.
We calibrated the robotic arms, established a CUDA-enabled learning environment, benchmarked ACT on the Jetson, and built a custom IK and trajectory-planning pipeline with simulation and software checks.
For Mini Alfred, we developed a wireless control interface, tuned rotation behavior, and added obstacle and surface-distance guards. We also built diagnostics that make sensor failures easier to distinguish from calibration problems. Together, these pieces give us a foundation for testing household skills and coordinating a stationary robot with a mobile companion.
What we learned
We learned that robotics depends on software, mechanics, electronics, and timing working together. A valid trajectory cannot compensate for an unstable chassis, and a successful model benchmark does not establish that a complete physical task will work.
We also learned where learned and geometric control complement each other. ACT offers a way to capture demonstrated behavior, while IK provides explicit spatial control. Choosing the right approach for each skill matters.
Most of all, we learned to test physical assumptions early. Payload, balance, calibration, and stopping distance need attention before adding more features. Reducing scope gave us a clearer architecture and more time to make its individual parts reliable.
What's next for A.L.F.R.E.D
Our next milestone is a repeatable household workflow: Alfred performs a tabletop task, then Mini Alfred transports the result.
We plan to collect more demonstrations, train and evaluate task-specific ACT policies, validate IK trajectories on the physical arms, and improve coordination between the two robots. Mini Alfred also needs wheel encoders and better position feedback for reliable delivery; accelerometer-based distance estimates alone are not precise enough.
Longer term, we want to add voice interaction, object recognition, docking, and broader edge detection. We will expand the task library as each behavior becomes reliable on the hardware.
