Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changing data folder #9

Open
taghreed34 opened this issue Aug 7, 2022 · 1 comment
Open

Changing data folder #9

taghreed34 opened this issue Aug 7, 2022 · 1 comment

Comments

@taghreed34
Copy link

taghreed34 commented Aug 7, 2022

Hi, sorry for asking about something that is explained in the repository, but it took me two days to deal with changing data directory and I didn't manage to do it.

I want to test NER model on my own file, it's the same format as conll2003.
I changed the data folder in the config file named "xlmr-large-pretuned-tuned-wiki-full-first_10epoch_1batch_4accumulate_0.000005lr_10000lrrate_en_monolingual_crf_fast_norelearn_sentbatch_sentloss_withdev_finetune_saving_amz_doc_wiki_v3_ner20" to the folder named: "/content/EN-English_conll_rank_eos_doc_full_wiki_v3_test" that contains the testing file with the name: "testb.txt".
I also put the the same folder path as a target directory in the command line to run without the testing file name. So what do I miss that causes the following issue: Nonetype object has no attribute exists?
This error happens because of data path type assertion.
And is there anything else needs to be added or changed?

MFVI:
hexa_rank: 150
hexa_std: 1
iterations: 3
normalize_weight: true
quad_rank: 150
quad_std: 1
tag_dim: 150
use_hexalinear: false
use_quadrilinear: false
use_second_order: false
use_third_order: false
window_size: 1
ModelFinetuner:
distill_mode: false
sentence_level_batch: true
anneal_factor: 2
ast:
Corpus: SEMEVAL16-TR:SEMEVAL16-ES:SEMEVAL16-NL:SEMEVAL16-EN:SEMEVAL16-RU
atis:
Corpus: ATIS-EN:ATIS-TR:ATIS-HI
chunk:
Corpus: CONLL_03:CONLL_03_GERMAN
embeddings:
TransformerWordEmbeddings-0:
fine_tune: true
layers: '-1'
model: resources/taggers/xlmr-large-pretuned-tuned-wiki-first_3epoch_1batch_4accumulate_0.000005lr_10000lrrate_multi_monolingual_crf_fast_norelearn_sentbatch_sentloss_withdev_finetune_saving_amz_doc_wiki_v3_10upsample_addmix_ner23/xlm-roberta-large
pooling_operation: first
interpolation: 0.5
is_teacher_list: true
model:
FastSequenceTagger:
crf_attention: false
dropout: 0.0
hidden_size: 256
locked_dropout: 0.0
remove_x: true
sentence_loss: true
use_cnn: false
use_crf: true
use_rnn: false
word_dropout: 0.1
model_name: xlmr-large-pretuned-tuned-wiki-full-first_10epoch_1batch_4accumulate_0.000005lr_10000lrrate_en_monolingual_crf_fast_norelearn_sentbatch_sentloss_withdev_finetune_saving_amz_doc_wiki_v3_ner20
ner:
ColumnCorpus-EN-EnglishDOC:
column_format:
0: text
1: pos
2: upos
3: ner
comment_symbol: '# id'
data_folder: /content/EN-English_conll_rank_eos_doc_full_wiki_v3_test
tag_to_bioes: ner
Corpus: ColumnCorpus-EN-EnglishDOC
professors:
config/single-de-ner.yaml: CONLL_03_GERMAN
config/single-en-ner.yaml: CONLL_03
config/single-es-ner.yaml: CONLL_03_SPANISH
config/single-nl-ner.yaml: CONLL_03_DUTCH
tag_dictionary: resources/taggers/EN-English_x.pkl
teachers:
? config_gen/multi-bert_flair_word_char_charcnn_300epoch_32batch_0.1lr_256hidden_de_monolingual_crf_sentloss_10patience_fast_sentbatch_relearn_fasttext_freeze_nodev_panx_ner45.yaml
: PANX-DE
? config_gen/multi-bert_flair_word_char_charcnn_300epoch_32batch_0.1lr_256hidden_es_monolingual_crf_sentloss_10patience_fast_sentbatch_relearn_fasttext_freeze_nodev_panx_ner45.yaml
: PANX-ES
? config_gen/multi-bert_flair_word_char_charcnn_300epoch_32batch_0.1lr_256hidden_nl_monolingual_crf_sentloss_10patience_fast_sentbatch_relearn_fasttext_freeze_nodev_panx_ner44.yaml
: PANX-NL
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_de_monolingual_crf_sentloss_10patience_baseline_fast_nodev_ner12.yaml
: CONLL_03_GERMAN
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_en_monolingual_crf_sentloss_10patience_baseline_fast_nodev_ner11.yaml
: CONLL_03
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_en_monolingual_crf_sentloss_10patience_baseline_fast_nodev_panx_ner8.yaml
: PANX-EN
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_es_monolingual_crf_sentloss_10patience_baseline_fast_nodev_ner12.yaml
: CONLL_03_SPANISH
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_eu_monolingual_crf_sentloss_10patience_baseline_fast_nodev_panx_ner8.yaml
: PANX-EU
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_fa_monolingual_crf_sentloss_10patience_baseline_fast_nodev_panx_ner8.yaml
: PANX-FA
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_fr_monolingual_crf_sentloss_10patience_baseline_fast_nodev_panx_ner8.yaml
: PANX-FR
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_he_monolingual_crf_sentloss_10patience_baseline_fast_nodev_panx_ner6.yaml
: PANX-HE
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_id_monolingual_crf_sentloss_10patience_baseline_fast_nodev_panx_ner8.yaml
: PANX-ID
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_nl_monolingual_crf_sentloss_10patience_baseline_fast_nodev_ner11.yaml
: CONLL_03_DUTCH
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_sl_monolingual_crf_sentloss_10patience_baseline_fast_nodev_panx_ner7.yaml
: PANX-SL
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_256hidden_ta_monolingual_crf_sentloss_10patience_baseline_fast_nodev_panx_ner6.yaml
: PANX-TA
target_dir: resources/taggers/
targets: ner
teacher_annealing: false
train:
embeddings_storage_mode: none
fine_tune_mode: true
gradient_accumulation_steps: 4
learning_rate: 5.0e-06
lr_rate: 10000
max_epochs: 10
mini_batch_size: 1
monitor_test: false
one_by_one: true
save_finetuned_embedding: true
select_model_by_macro: true
train_with_dev: true
true_reshuffle: false
use_warmup: false
trainer: ModelFinetuner
upos:
Corpus: UD_GERMAN:UD_ENGLISH:UD_FRENCH:UD_ITALIAN:UD_DUTCH:UD_SPANISH:UD_PORTUGUESE:UD_JAPANESE
UD_GERMAN:
train_config: config/
professors:
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_de_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_GERMAN
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_en_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos0.yaml
: UD_ENGLISH
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_es_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos0.yaml
: UD_SPANISH
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_fr_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_FRENCH
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_it_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_ITALIAN
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_ja_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_JAPANESE
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_nl_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_DUTCH
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_pt_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_PORTUGUESE
tag_dictionary: resources/taggers/pos_tags.pkl
teachers:
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_de_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_GERMAN
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_en_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos0.yaml
: UD_ENGLISH
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_es_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos0.yaml
: UD_SPANISH
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_fr_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_FRENCH
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_it_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_ITALIAN
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_ja_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_JAPANESE
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_nl_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_DUTCH
? config_gen/multi_bert_origflair_300epoch_2000batch_1lr_400hidden_pt_monolingual_crf_sentloss_10patience_baseline_fast_nodev_upos1.yaml
: UD_PORTUGUESE

@wangxinyu0922
Copy link
Member

Please provide the error lines for your running.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants