About the results of JF17K dataset

[zhiweihu1103]

Would you mind providing the running script for JF17K? According to the default parameters you provided, I can only get 10% MRR results.

export LOG_PATH=./logs/jf17k.out
export CUDA=4
export VOCAB_SIZE=29148
export DATASET=jf17k
export VOCAB_FILE=./data/jf17k/vocab.txt
export TRAIN_FILE=./data/jf17k/train.json
export TEST_FILE=./data/jf17k/test.json
export GROUND_TRUTH_FILE=./data/jf17k/all.json
export NUM_RELATIONS=321


nohup python -u ./src/run.py --name "TEST" --dataset $DATASET --device $CUDA --vocab_size $VOCAB_SIZE --vocab_file $VOCAB_FILE \
--train_file $TRAIN_FILE --test_file $TEST_FILE --ground_truth_file $GROUND_TRUTH_FILE --num_workers 1 --num_relations $NUM_RELATIONS \
> $LOG_PATH 2>&1 &

The corresponding log file config:

05/14/2023 22:50:42  -----------  Configuration Arguments -----------
05/14/2023 22:50:42  batch_size: 512
05/14/2023 22:50:42  ckpt_save_dir: ckpts
05/14/2023 22:50:42  dataset: jf17k
05/14/2023 22:50:42  decoder_activation: gelu
05/14/2023 22:50:42  device: 4
05/14/2023 22:50:42  entity_soft: 0.8
05/14/2023 22:50:42  epoch: 100
05/14/2023 22:50:42  global_activation: elu
05/14/2023 22:50:42  global_dropout: 0.1
05/14/2023 22:50:42  global_heads: 4
05/14/2023 22:50:42  global_layers: 2
05/14/2023 22:50:42  ground_truth_file: ./data/jf17k/all.json
05/14/2023 22:50:42  hidden_dim: 256
05/14/2023 22:50:42  hyperedge_dropout: 0.0
05/14/2023 22:50:42  local_dropout: 0.1
05/14/2023 22:50:42  local_heads: 4
05/14/2023 22:50:42  local_layers: 12
05/14/2023 22:50:42  lr: 0.0005
05/14/2023 22:50:42  max_arity: 32
05/14/2023 22:50:42  max_seq_len: 63
05/14/2023 22:50:42  name: TEST
05/14/2023 22:50:42  num_entities: 28825
05/14/2023 22:50:42  num_relations: 321
05/14/2023 22:50:42  num_workers: 1
05/14/2023 22:50:42  relation_soft: 0.9
05/14/2023 22:50:42  remove_mask: False
05/14/2023 22:50:42  test_file: ./data/jf17k/test.json
05/14/2023 22:50:42  train_file: ./data/jf17k/train.json
05/14/2023 22:50:42  use_edge: True
05/14/2023 22:50:42  use_global: True
05/14/2023 22:50:42  use_node: False
05/14/2023 22:50:42  vocab_file: ./data/jf17k/vocab.txt
05/14/2023 22:50:42  vocab_size: 29148
05/14/2023 22:50:42  warmup_proportion: 0.1
05/14/2023 22:50:42  weight_decay: 0.01

[LHRLAB]

OK, updated.

[zhiweihu1103]

I suggest you update the paper, because the Table 1 and Table 4 are different from your new hyperparameters.

[LHRLAB]

OK.