bad performance

[lichaoaaron]

Hi, after I run the code on Cora based on README.md, I got an accuracy of 85%, which is much lower than 89% as reported in the paper. Can you tell me how can I improve the performance?

[hzhaoaf]

@lichaoaaron Hi, Can you give more details about the experimental setups, including the OS, GPU info, versions of torch and pyg, etc. We need more information to check the reason.

[lichaoaaron]

Hi, thank you! The information is as follows:
GPU : TITAN RTX
CUDA Version: 10.2
python 3.7.13
pytorch 1.10.0
torch-cluster 1.6.0
torch-scatter 2.0.9
torch_geometric 2.0.4
python train_search.py --data Cora --fix_last True --epochs 20
python fine_tune.py --data Cora --fix_last True --hyper_epoch 50 --arch_filename exp_res/cora.txt

Result:
times:0, valid_Acc:89.8523941040039, test_acc:86.29629516601562
times:1, valid_Acc:89.8523941040039, test_acc:86.29629516601562
times:2, valid_Acc:89.8523941040039, test_acc:86.66666412353516
times:3, valid_Acc:90.03689575195312, test_acc:86.29629516601562
times:4, valid_Acc:89.8523941040039, test_acc:86.29629516601562

test_results_5_times:86.3704+-0.1481
finsh tunining 5 archs, saved in tuned_res/Cora_res_20220418-154824_res-20220418-154626-eps0.0-reg0.0005.pkl

[hzhaoaf]

@lichaoaaron Got it. We are conducting some experiments to check the robustness of our code. Will let you know asap.

[a912289748]

@lichaoaaron Got it. We are conducting some experiments to check the robustness of our code. Will let you know asap.

torch _sparse version