Determined driven model hyper parameter search

• 

  Image classification loss, accuracy graph

• 

  Hyperparameter search Image classification

• 

  Running bert adverserial qa

• 

  Completed bert adverserial qa

• 

  list of experiments

Inspiration

The primary aim of the project is to learn creating model experiments, hyper parameter search and distributed training on the determined ai platform.

The project includes 2 models with several experiments to successfully deploy on the determined ai platform.

One project is a fine tuned efficientnet image classifier and re-training a question/answer transformer.

What it does & How its built

Image classification

The EfficientNetB0 model is the base model and it is fine tuned for Stanford dogs database using transfer learning where only the top layers are retrained.

The model is written in keras in the determined-ai platform template and deployed for distributed training.

After training experiment, an adaptive hyper parameter search experiment was created from that checkpoint to identify correct learning rate and drop out.

dataset: https://www.tensorflow.org/datasets/catalog/stanford_dogs

Question & Answer Transformer

A pytorch experiment using an existing question and answer transformer example was developed training on 2 slots for distributed training.

The transformer was fine tuned for Adverserial QA database from huggingface.

dataset: https://huggingface.co/datasets/adversarial_qa

Challenges we ran into

The main challenge was adapting model code to templates for the determined platform to be able to run experiments. Other challenges include ensuring the model works with the pytorch and tensorflow/keras versions of determined-ai

What's next for Determined driven model hyper parameter search

The next approach is to try image to text and text to image transformers. I had tried keras-nlp but due to conflicts between libraries was unable to execute the experiments

Built With

• determined
• keras
• python
• pytoch
• tensorflow