To get the opportunity for children to play with robots without any extra software of hardware. Just Lego bricks. But also they get the opportunity to learn about programming while having fun!
What it does
EV3 Robots from Lego can be remote controlled from a PC with a webcam. Our program is able to detect hand gestures using Google's TensorFlow through the webcam and convert them into EV3 commands in order to move the robots. Furthermore, robots can be also controlled by a sequence of Lego bricks, our program detects the color of the bricks and moves according to the color sequence.
How we built it
We planned together what we wanted to achieve, and then we split up the team in order to address: TensorFlow and hand gesture detection. Robot building and its movement. Brick colors detection. Then, together we merged these three in order to get the final product, adding the needed controllers, connections and helper classes. Finally we built a box in order to have a controlled environment for the light provided to the webcam and for detecting the proper brick color with the most accuracy.
Challenges we ran into
TensorFlow not having enough data. Color Isolation. Mindstorm EV3 robots wouldn't move under certain conditions.
Accomplishments that we're proud of
We are proud to have met all our milestones in the given time. We are also proud of our UI for controlling the EV3s. But, most importantly we are proud of our team cooperation, we got it all working together, thanks to a friendly working environment and some great team members.
What we learned
We learned about Python, how to use some new IDEs, TensorFlow, Webcam control and color isolation, connection to the robots.
What's next for Brick-It-On!
A lot more!