Multiple bounding box regressor to avoid id switch

[FawadAbbas12]

Hello i am continuing on your work by crating an ensemble of MixFormer and conventional kalman filter to handle cases like

• longtime occlusion in which object goes out of view but keeps same trajectory (may lead to template degradation)
• multiple instance of object with same features exist in close proximity with each other
• and edge cases like these

currently i am using only one EKF instance which tracks the object and mixformer's output is considered as model update which in turn predicts error/jump for tracked object.
This process is giving some reliable results but is prune to large error.
One way around is to maintain multiple track-lets for all objects which match the template in given search space thus we can match the tracklet of out selected object with EKF's prediction.

So is there any way to regress multiple bounding boxes for given template as we also have the option to have multiple templates. i.e get bounding box for all objects which match with the given template.

this video shows the said problem

[FawadAbbas12]

incase google drive link is not accessible here is github link for video
https://github.com/MCG-NJU/MixFormer/assets/36898352/54d701e0-0d93-4417-a1a2-f27bd7736d71

[yutaocui]

Hi, thanks for your attention. It's quite a good question since the occlusion and similar objects are the two main obstacles of MixFormer. If you want to regress multiple bounding boxes, there is a simple solution: substituting the corner head with center-based head or query-based head. So that you can obtains multiple objects bounding boxes with high confidence score.

[yutaocui]

The mentioned query-based head is like DETR, which predict a bounding box and a confidence score. But you should re-train the model in tracking dataset or your own dataset.

[FawadAbbas12]

Thank you for sharing key insight on implementing the bbox-regressor i will try to implement it and if i can get reasonable results would like to create a pull request.
As of dataset i will be training on need for speed sampled down to 30 and 10 FPS along with GOT.