Question for input

[kimsekeun]

1. for pre-training FM, how many frames are used ? (150fr? for each batch)?

2. for downstream task, I see that you used 224 224 with 1 frame for segmentation. Then how can it be possible to process Video ViT for taking 1 frame input as it is pre-trained on input size of 150 fr?

Thank you.

[Kyfafyd]

Hi @kimsekeun thanks for your interest!

1. We use 16 frames from each video to pre-train FM.
2. We feed an entire video at once for segmentation training. Our model can tackle any length of video input.