[Bug]: reproducing bumblebee result on gpt2-base model is very slow

[Bug]: reproducing bumblebee result on gpt2-base model is very slow #3322