Utilizing custom vision models on mobile devices is a key practical use of AI. Such systems can help a visually-impaired person to understand the physical environment, a factory laborer to record the entries, or an environmental scientist to note the observations. In this work, I share the code and tutorial for developing high-performance/small-size custom models and using these models on Android devices.
Implementation
We used the mobilenet-v3 pre-trained model as the base architecture for the custom classification task. We modified PyTorch's official Transfer Learning tutorial to train this custom model. Our modification includes a live data collection step, integrating the mobilenet-v3, and optimizing it for mobile devices. Then we updated PyTorch's official Android demo app with the latest build, deploy our model and define new tasks to test the model on the device.
Challenges
In the Android app development process, we faced compatibility issues. Switching from <1.8 to a higher version requires to use of Lite Model Interpreter, which is not actually implemented in the Android demo. We noted all the challenges and add our solutions on README documents.
Learnings and Findings
Checking PyTorch mobile and exploring possibilities was an item on my TODO list. I encountered this hackathon only one week before the deadline, and I decided to participate to complete this TODO list item by exploring what PyTorch Mobile is capable of. I am impressed by the ease of use of the framework. About an hour, I was testing the first iteration of the prototype which is not mostly possible for an end-to-end application...
Next Steps
I really liked working with PyTorch mobile and I planned two next steps for this project:
- I will also add custom NLP model examples to the project,
- I will develop an on-Android-device transfer learning application.
Log in or sign up for Devpost to join the conversation.