I am interested in switching the input word embedding of the dynamic memory network (see here for an implementation) from Glove to BERT. However, I am new to NLP. I see from this github page that Glove has pre-trained word vectors, and we only need to download these pre-trained vectors to embed the input sentences for the network. Is there a similar thing for BERT as well? I saw from posts online that we need to train BERT on our own dataset and thus achieve the contextualized word embeddings. I want to train the network above using the same babi dataset but with BERT instead of Glove. Does anyone have some advice on how to approach or can point me to some useful links? Thanks a lot!!

  • 1
    I'm voting to close this question as off-topic because it is about implementation details of NLP software. It'd probably be better answered on [stackoverflow.se], [crossvalidated.se], or Data Science
    – Mitch
    Mar 6 '20 at 20:19