Transformer NLP

Demos

Here are a few info:

In Natural Language Processing:

Roberta distilation
Roberta Distilation classification
Knowledge distilation from large language model
Initializing a model with different embedding size and layer from large model through Teacher-Student pretraining

Name		Name	Last commit message	Last commit date
Latest commit History 23 Commits
README.md		README.md
debertav3-mlm.ipynb		debertav3-mlm.ipynb
gpt2-text-generation.ipynb		gpt2-text-generation.ipynb
robert6l-seq2seq.ipynb		robert6l-seq2seq.ipynb
roberta-base-6L-distill-pytorch.ipynb		roberta-base-6L-distill-pytorch.ipynb
roberta-base-from-scratch-pytorch.ipynb		roberta-base-from-scratch-pytorch.ipynb
seq2seq-transformer-translation-final.ipynb		seq2seq-transformer-translation-final.ipynb
transformer-distilation-pre.ipynb		transformer-distilation-pre.ipynb
transformer-pytorch-native-knowledge-distillation.ipynb		transformer-pytorch-native-knowledge-distillation.ipynb
transformer-pytorch-native.ipynb		transformer-pytorch-native.ipynb
update-gpt2-scratch-implementation-text-gener.ipynb		update-gpt2-scratch-implementation-text-gener.ipynb
utilizing-transformer-representations-efficiently.ipynb		utilizing-transformer-representations-efficiently.ipynb