PyTorch implementation of "Emerging Properties in Self-Supervised Vision Transformers"
Student ( |
Teacher ( |
|
---|---|---|
Before (mismatch) |
![]() |
![]() |
After (matched) |
![]() |
![]() |
Epoch | Student ( |
Teacher ( |
---|---|---|
0 | ![]() |
![]() |
1 | ![]() |
![]() |
30 | ![]() |
![]() |
150 | ![]() |
![]() |
300 | ![]() |
![]() |
- PyTorch 2.0.1
[1] Mathilde Caron, et al. "Emerging Properties in Self-Supervised Vision Transformers." Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.