We drew inspiration for this project while we were searching for an interesting topic and we all expressed interest in computer vision and artificial intelligence and stumbled upon a similar project and thought it would be interesting to give it a try our selves, despite having no prior knowledge in any of these areas.

What it does

The program starts by identifying the users hands and then it will identify the shape of the hand

How we built it

We looked at many guides and different ideas and methods that could be used to accomplish the same task before settling on what we were going to do. We used opencv to first identify the hand and get the main points of the hand, from there we used mlpack to run the points through a knn function to sort the current position of the hand into one of the possible configurations.

Challenges we ran into

The first challenge we ran into was a lack of C++ knowledge as some of our members were not very familiar with the language, this caused a lot of confusion and slowed us down considerably. The next problem we ran into was a difficulty installing libraries to Visual Studio. For example we tried to use the Shark library before the mlpack library but we could not get it to work.

Accomplishments that we are proud of

We are proud that we were able to learn so much from this event, especially the opencv and mlpack libraries that we learned.

What we learned

We learned a lot about c++ in general as some of us were not very well versed in the language. We also learned considerable amounts of opencv as we had to do much more advanced operations than we had previously experienced. Finally we learned a lot about machine learning and how the different algorithms work.

What's next for Finger Detection and Tracking using opencv and mlpack in C++

The next step for our project would be to identify sign language.

Share this project: