Lilybot

Inspiration

I use AI a lot. It helps me organize ideas, learn and introduce new concepts and even give me quizes to test my skills within a topic. However it does consume a lot of energy, is cloud based whereby it doesn't run on your own device which means not so private. moreover it depends on internet connection so once your offline u can't access it anymore. Thus i decided to produce my own AI that solve all these problems. Also as someone who likes drawing and is studying about flowers, i was inspired to make the bot look like a flower of my favorites, a "Lily" flower. I sketched a lily printed it and got some clay (which i have no previous experience with at all), to make my lilybot container more naturalistic and give it a natural vibe.

What it does

lilybot is a desk assistant, whether you just wanna chat randomly or want to ask about something without constantly reaching to your phone, lilybot is right on your desk just say the wake word "lily" and go ahead and ask. It can also work offline because it is built based on EDGE AI. it fully runs on your device. ensuring extra privacy aswell.

How we built it

lilybot was built with small hardware, the main hardware which is the raspberry pi zero 2 W only has 512MB. This project proved that we don't need some big hardware to make an AI assistant exist. When it comes to it's code, it is coded with python. i was based on 4 main files. speaker.py, microphone.py, brain.py and finally main.py. The main file was essentially made to act like a manager so not all modules run at once which would lead to a freeze because the Pi's RAM wouldn't handle it. i have a detailed explanation in my github, if you are interested feel free to visit it. as for the body of the bot, i decided to build it using air-dry clay, which i had no previous experience with. and for the external layer, the front side, i sketched a lily flower to make it match the vibe.

Challenges we ran into

i ran into a lot of challenges, whether they were hardware challenges, coding and debugging challenges, or even when building the body with the clay. i had no previous experience with hardware, and this was my first time building an AI assistant. i discovered and learned a lot of things throughout the project. i messed wires, burned the microphone when soldering, smelt burning plastic from the amplifier, had to debug for hours constantly for some days, had to discover bugs on my own because apparently even when reaching for AI assistance to help me debug, they would sometimes give hypothesis that are just leading to more mess rather than less. After i finished and passed all these challenges i finally built the body with the clay, i checked on it once it was dry, after it was dry i lifted it to check it and edit it, it then felt apart, it was all broken, so i had to make an other one.

Accomplishments that we're proud of

i managed to solve all the challenges i faced, thus i am proud of them all. but the first and most important accomplishment is that i was able to start this project without any previous experience with almost nothing. one of the other accomplishment i was proud of, is when i finished everything and eventually tried to test the bot, it would listen and generate answer, but not speak the answer. i had to check the code again and again, and ask AI again and again and finally found out that the main problem is that i forgot to get a voice model, seems too funny to be a bug but it is what actually happened, the thing is that i found the bug that was messing with the whole output on my own. an other accomplishment would be that iam the one to design the whole layouts for the presentation including the video editing.

What we learned

the lilybot project made me learn various things, from soft skills to dealing with hardware. this project depended on resilience, patience, critical thinking and problem solving. it is a marathon, so each day makes a difference, whether you would work that day or count it as a rest an move on. when you rest you default mode network works on the project and find patterns, which then helps you solve problems more efficiently and prevent burnout. i learned to deal with hardware, even if not much because its my first time, but i can confidently say that i am way better with hardware then i was before this project. when it comes to coding, i learned more concepts in python, which i didn't before. i learned that even when you end up the day with some bugs and errors in the terminal, it still can be called a day, because even if you don't see it, you have definitely improved in some way.

What's next for Lilybot

i am thinking to add some LED that changes color whenever the bot is doing something, for example if its thinking, the light would be blue and so on.. but for now we gonna stop here.

Built With

• edge-ai
• python
• raspberry-pi