Fabric toys with openwrt
Shapes to generate the toy
Profile picture of the toy
Picture without legs
Toy with raspberry inside
Custom head with 2 microphones
Custom head with 2 microphones
Everything started a couple of days ago, watching how my friends and my girlfriend's friends start to be engaged and get married. I started to think about how our kids or well the future kids of all the millennial and future generations will play, what kind of toys they will have.
That's when I thought about watson and the future of kids, with machine learning telling stories and searching for information when kids ask, our future kids might be a little more intelligent than us and of course with less screens or monitors laying around a house. We might have more creative and imaginative children.
What it does
The watson toy detects when a person speaks, how does it do it?, well it has a microphone as a sound sensor to detect when the user speaks, then it activates a pin on the Raspberry board and start to make a record. After that it sends the recording to watson to generate the Speech to text, then a second process is run to determine if that dialog is within the allowed questions a kid should ask to watson, and finally it gets the response. Fortunately the watson toy can easily translate between several languages, so we can switch between English and spanish for example.
How we built it
Well the whole toy was build from scratch, we started looking for ways to have the toy to be very familiar and enjoyable, something you might consider funny or curious. So the fabric toys were the perfect solution for somethings kids could enjoy carrying around. The fabric we selected was a very funny like if the toy went to school. Initially I though of using an openWRT board but to be honest the watson SDK for nodejs is a little big for the small space, the space can be expanded using an SD card but I didn't want to mess with the filesystem, so we switch to a raspberry board, although a lot of boards can be used, for instance the beagle bone, the dragon board and other boards in between, thanks to the nodejs SDK from IBM.
About the hardware we used: 2 MIcrophones (1 used to recognize ambient noise, 1 to recognize voice) 1 USB sound card PNP 1 set of speakers 1 USB WIFI 1 Portable battery
Unfortunately the raspberry 1 and 2 doesn't have WIFI integrated so that's the reason why we used the usb wifi, but other boards can just be used directly reducing tons of space. We decided to use nodejs because it is fast and easy to see real life results, now one of the microphones is connected to PIN4 in the raspberry such as in the image attached. One of the core libraries I used was the GPIO from nodejs, this library can recognize PIN interrupts and of course trigger the sound record.
Challenges we ran into
The biggest challenge is to recognize the sound, because any sound activates the recording, but thanks to the raspberry pinouts and external interrupts we are able to make the application recognize when a conversation is taking place. That is so important because the toy must save the state of the conversation and that was one of the biggest challenges, and the microphones perform this task perfectly
Accomplishments that we're proud of
We have an interesting list of accomplishments, we know there are very super interesting projects for this technology but one of them for us is to let kids play with this technology. For this we would like to share with you that taking in consideration the fabric and selecting the correct one is one big milestone. Then designing the body to be able to contain and fit a whole raspberry board was the second milestone. About the software, we followed all the procedures to integrate voice recognition, text to speech, and translation APIs to work as a whole. Finally integrating everything and putting together a little MVP is one cute accomplishment for us, and hopefully our future kids.
What we learned
The IBM APIs are something interesting to study and learn, we learned a lot from making a toy from scratch, we learned that watson can be used as an introductory technology for kids. We also learned how machines help us raise our future children and ourselves in productive ways.
What's next for Intelligent fabric toy
The next step is no doubt to continue finish and assambling several toys in different shapes and voices, which can easily be done with watson. Then the next step is to introduce the toy to children in different ages, this might help us realize how to make changes and improve our actual toys. Finally we might offer them for sale if we manage to create better and improved versions of this toy.