Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to reproduce results on JF17K #3

Open
wjj0122 opened this issue Jun 11, 2023 · 1 comment
Open

Unable to reproduce results on JF17K #3

wjj0122 opened this issue Jun 11, 2023 · 1 comment

Comments

@wjj0122
Copy link

wjj0122 commented Jun 11, 2023

Hello! I use the default parameters in Readme.md to run experiments on JF17K, but the final result of entity MRR has a gap with your paper, could you please tell me the reason?
The parameters are:
)KQ1_(EYXC%5 9Q_T$ )SBI
The results are:
image

@handsomelys
Copy link

I have the same problem:
image
I use the script in the README to run the experiment. However, the experimental results seem to be a bit different from what is reported in the paper...
I'm wondering if my hyperparameters aren't set to optimal? Here's my list of hyperparameters (follow the README):
python -u ./src/run.py --dataset "jf17k" --device "0" --vocab_size 29148 --vocab_file "./data/jf17k/vocab.txt" --train_file "./data/jf17k/train.json" --test_file "./data/jf17k/test.json" --ground_truth_file "./data/jf17k/all.json" --num_workers 1 --num_relations 501 --max_seq_len 11 --max_arity 6 --hidden_dim 256 --global_layers 2 --global_dropout 0.9 --global_activation "elu" --global_heads 4 --local_layers 12 --local_dropout 0.35 --local_heads 4 --decoder_activation "gelu" --batch_size 1024 --lr 5e-4 --weight_deca 0.002 --entity_soft 0.9 --relation_soft 0.9 --hyperedge_dropout 0.85 --epoch 300 --warmup_proportion 0.05

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants