Important

• Model files are available in Google Drive
• After cloning this repository download these folders and files from the above link: best-84.11, checkpoint-600, checkpoint-7776, final-roberta-classifier, and RoBERTa-base-PM-M3-Voc-distill-align-hf

Model files explanation

• best-84.11 / final-roberta-classifier: Best RoBERTa with classification head checkpoint
• checkpoint-600: Best BART checkpoint trained via CONFIT with attempt to add length as a loss
• checkpoint-7776: Best BART checkpoint trained via CONFIT
• RoBERTa-base-PM-M3-Voc-distill-align-hf: From bio-lm

Run descriptions

Task A - Run 1:

• Classification: RoBERTa with classification head (best-84.11 / final-roberta-classifier)
• Summaries: Trained BART via CONFIT (checkpoint-7776)

Task A - Run 2:

• Classification: Fine-tuned OpenAI davinci
• Summaries: Trained BART via CONFIT (checkpoint-7776)

Task A - Run 3:

• Classification: RoBERTa with classification head (best-84.11 / final-roberta-classifier)
• Summaries: Trained BART via CONFIT (checkpoint-600) with dynamic max length

Task B - Run 1:

• Fine-tuned OpenAI davinci (temperature = 0.2)

Task B - Run 2:

• Fine-tuned OpenAI davinci (temperature = 0.0)