ApolLens

Inspiration

The Inspiration behind ApolLens came from a group member that works with tech repairs, with the clunkiness behind repairing computers and laptops being a massive hassle, with information being hard to discern with google searches and clunky manuals, and the need for offline access, thats quicker and easier than what exist currently.

What it does

ApolLens' built-in computer vision scans a live camera feed, and through it large collection of image-training, it tries to detect what its looking at, and send that information to the local LLM running on the app, which finds the best fit solution for you, and guides you step-by-step on troubleshooting the faulty system.

How we built it

We built it to run through Flutter, with the computer vision model being built after Flutter and integrated into the backend, with the learning module being built and trained at the same time, and the UI design being built last, after all the other systems were finished. With the help of AI, we implemented the local LLM.

Challenges we ran into

Due to the limited time we had due to the ambition of the size of the project, we were only able to our train our computer vision model on a few parts, with further parts requiring even more time which we did not have due to the very small window of time we were given for this hackathon. In addition, implementing Elevenlabs was clunky and inefficient, as the time it took to call the LLM and the Elevenlabs voice AI took too long, and was unviable.

Accomplishments that we're proud of

We are proud of our computer vision model, and machine learn portion of the code, and the training of the parts, which come back with fairly accurate results.

What we learned

We learned how to integrate a computer vision model to work side-by-side a LLM, as well as working independently of each other. We also learned how to train the computer vision model to properly detect components in a live camera feed.

What's next for ApolLens

If the ApolLens project continues, what's next is the development of being able to detect more parts of a laptop, than the limited amount of parts it can accurately detect now, and most likely detect more than just laptop internals, with the next logical step being training the app on detecting computer parts, which are quite similar to laptop parts.