As a passionate team of Mobile Robotics, ROS and Human Robot Interfaces, we built a teleprescense robot that merge our passions and as well is a valuable application for new technologies in the future. Our idea was shaped in the ideal that this will not only be cool, but also be useful for different scenarios where robot could assist humans in almost any imaginable way.

What it does

The user enters a web page where he registers and it shows in given video frames if there is a known face in front of the Kinect Camera. If there is a recognizable face the user can point at the face (with Leap Motion Technology) and the bb8 robot will approach to that specific person. This is supposing that the user is in one place and the camera and the robot in a completely different one.

How we built it

Our project was developed with high-technology. First, the whole application is implemented using Robotic Operating System (ROS), a variety of nodes that put together several technologies like a Kinect Camera, Leap Motion, a Sphero BB8 Robot and Web Services.

The Kinect Node is developed with OpenCV for Face Detection, this can learn some faces based in photos and detect if someone is present or not in the camera, with help of this web application the person can know who is in the room.

The Leap Motion is based in the SDK that the company provides; however, the interfacing with the web server as a IOT part was a real challenge.

The Sphero BB8 node was also a tremendous challenge because we needed to dive deeper in the specification sheets even down to hexadecimals instructions, the hacking of these was really complicated, but after testing we got the measurements and the functions to make it work properly.

Finally the web server is a complete application where a face database is implemented as well as the Leap Motion Control.

All the project is based on materials that were provided to us here at HackMty, we did not bring any hardware; we wanted to make the best possible with what we got, so this was certainly a challenge we were proud to face.

We learned a variety of new skills that made the whole project possible. That said, the project could only be completed with the unconditional support of each member of the team.

Must of the challenges we encountered, were due to the fact that we had no idea what the hackathon was gonna be about. This, however challenged us to step out of our comfort zone and to be creative with the material at hand.

We are very proud that most of the objectives were achieved. Given a bit more time, we could have done a bit more testing/debugging which would have allowed us to make a more professional delivery.

What we learned

We learned that it is really important to state the idea before start working, to research in an efficient and organized way and as well to take in account any of the details that a project could involve, like communications, interfaces, among a lot more.

What's next for BB8 Streamer

The hardware is not ours, but if possible it will be amazing to keep developing this project, however, he have hardware that we can implement for new project like Human Robot Interfaces.

Share this project: