Project Checkin 2
Introduction:
This project is to examine the effectiveness of SOTA meta-learning algorithms on top of pre-trained language model (i.e. BERT) on natural language understanding (NLU) tasks (i.e. GLUE dataset) and further on multimodal tasks (i.e. question answering, image captioning)
Challenges:
The biggest challenge so far for us has been resource management and debugging performance issues. Our program is exhausting huge amounts of memory and disc space. Because we are using a pretrained model, we don't have great insight into what is going on under the hood and how it can be managed better.
Insights:
Unfortunately we are yet to run our program on the full data set. While our code runs and produces results on a subset of the data, as mentioned above it crashes on the full data set. There are no viable point of references for the expected performance on a subset of data so we are not able to make any assessments regarding the performance of the model at this time.
Plan:
The absolute priority is to fix the performance issues since the next step is to run tons of experiments and we can not do it without a fully functioning code. The original plan was to test and compare 4 different implementations (MAML, REPTILE, LEOPARD, SMLMT), in light of the current setbacks we have decided to drop LEOPARD and SMLMT and focus on the former two
Log in or sign up for Devpost to join the conversation.