SqueezeNet was originally an optimization of AlexNet, achieving the same accuracy with 50x fewer parameters. We plan to take it one step further by re-implementing SqueezeNet with multiple further optimizations including but not limited to network pruning, quantization, weight sharing, huffman coding, and segfast. The goal of this is to allow even lower-spec machines to run image classification models with lower power consumption. Our biggest challenges so far have been making sure that we implement our preprocessing and model with our size and runtime benchmarks in mind. For example, in our preprocessing, we struggled to format our data so that we could dynamically preprocess batches of images as we had to consider the size of the database. In the first place, it was difficult to understand the file formats of the data and open them in the correct format. For example, ImageNet uses the .npz file format which requires the entire dataset to be loaded in if using np.load(). Dynamic batching simply added another layer of complexity, in terms of determining which data formats to use and how we should be reading in the data into TensorFlow friendly formats. Although the Keras ImageDataGenerator has a very convenient one line format, after some testing as well as research online about benchmark speed, we realized that using tf.data.Dataset is much more scalable. We ultimately decided that it would be more appropriate to write scalable code, particularly when working with a dataset such as ImageNet. Despite all these difficulties, we were able to successfully write working preprocessing code with transformations that can be batched and streamed in real time as the dataset is needed as well as begin to write the layer architecture and decide on hyperparameters. The code can still be optimized more depending on the use case that we decide on, but the structure and flow of the preprocessing is well defined and good enough for progress on the main project content. Although it is still unclear how much work we have left in the project just from the work we have done so far, it is clear that there will be many more unforeseen nuances and difficulties in our project as we continue on with SqueezeNet and Deep Compression optimizations. We hope to maintain our plan of being flexible with our goals to accommodate these difficulties in the future.
Log in or sign up for Devpost to join the conversation.