Statement of the performance correction in FB15K-237-IMG

[zxlzr]

Dear all,

We found a severe bug in the data processing code for FB15K-237-IMG, which leads to an unfair performance comparison. We are very sorry for that.

The bug was caused by a data processing function that wrongly feeds the images of head entities similar to the images of tail entities (those images should be filtered for a fair comparison). Due to poor version control, we mistakenly used the buggy version to obtain the performance.

The bug has been fixed now. We have updated the performance in arxiv and released the checkpoints (The model trained with/without the severe bug) for reproducibility.

The correct fully-supervised performance (without carefully hyper-parameter tuning) is hit@1 0.256 hit@3 0.367 hit@10 0.504 MR 221, which is still better than previous multimodal baselines and does not affect the paper's conclusion.

Sincerely,
All authors