Skip to content
/ exploring-gpt2-language-model Public

This repository contains notebooks and pipelines exploring internal working of GPT2, visualization of its embeddings, its capabilities for different NLP tasks - text completion, classification, reading comprehension etc.

1 star 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

Taaniya/exploring-gpt2-language-model

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
Explore_gpt2_for_QA.ipynb
Explore_gpt2_for_QA.ipynb
 
 
Exploring_GPT2.ipynb
Exploring_GPT2.ipynb
 
 
README.md
README.md
 
 
Text_classification_with_gpt2.ipynb
Text_classification_with_gpt2.ipynb
 
 
Visualizing_gpt2_token_embeddings.ipynb
Visualizing_gpt2_token_embeddings.ipynb
 
 
fine_tuning_DistilGPT2.ipynb
fine_tuning_DistilGPT2.ipynb
 
 
finetune_gpt2_clm_pytorch.py
finetune_gpt2_clm_pytorch.py
 
 

Repository files navigation

exploring-gpt2-language-model

Finetune Transformer decoder only models like GPT2 with causal language model objective

python finetune_gpt2_clm_pytorch.py \
	--model-name gpt2 \
	--dataset-path train.txt \
	--batch-size 8 \
	--epoch 4 \
	--model-version-no 1 \
	--model-dir ./

About

This repository contains notebooks and pipelines exploring internal working of GPT2, visualization of its embeddings, its capabilities for different NLP tasks - text completion, classification, reading comprehension etc.

Resources

Readme
Activity

Stars

1 star

Watchers

3 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Languages