New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Define slide-level evaluation protocol #475

Open

roman807 opened this issue May 28, 2024 · 1 comment · Fixed by #511

Assignees

Collaborator

roman807 commented May 28, 2024

No description provided.

roman807 self-assigned this

roman807 mentioned this issue

459 add camelyon16 slide level task #476

Merged

roman807 mentioned this issue

475 define slide level evaluation protocol #511

Merged

roman807 linked a pull request

that will close this issue

475 define slide level evaluation protocol #511

Merged

Collaborator Author

roman807 commented Jun 7, 2024

Following findings with experiments with three checkpoints of different FM sizes on Camelyon16 + Panda:

AdamW converges, SGD does not
LR=1e-3 ideal (lower doesn't reach optimal performance, takes much longer to run)
Number of steps 12,500: works well for large dataset (Panda 10k slides), is too many for small datasets (Camelyon 400 slides) -> cap for max n epochs necessary
Patience 10% is sufficient, min of 8 epochs needed
Batch size: 32 works as well or better than 16, 64 and bettern than >= 128

--> Proposed protocol:

Number of steps: 12,500 (max: 100 epochs)
Patience: 10% of epochs (min 8 epochs)
Batch size: 32
LR: 1e-3
Optimizer: AdamW

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment