question about backward_tracking

[XiaoyuShi97]

https://github.com/16lemoing/dot/blob/e32c6f7de12342460371f2efc4789bd79c4a39a3/dot/models/shelf/cotracker2_utils/predictor.py#L145-L153

Hi, I am a bit confused about backward_tracking. First, back_tracking means computing the trajectory starting from the last frame to the first N-1 frames? If so, does backward_tracking benefit (get higher accuracy) from forward_tracking? And do the variables "tracks and visibilities" in L153 overwrite those in L145? Thanks!

[nikitakaraevv]

Hi @XiaoyuShi97, CoTracker is an online algorithm, and can only track forward. That's why there's such a thing as backward tracking, where we simply run tracking forward on a reversed video.

The variables "tracks and visibilities" in L153 overwrite those in L145 only for the frames for which the model didn't predict any tracks. For example, if a point was queried at a frame number 10, forward tracking will predict tracks for frames 10,11,12,..., while backward tracking will add predictions for 9,8,7,...