Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Examples: can remove cross-validation from outlier detection #24

Open
jwmueller opened this issue Sep 16, 2022 · 0 comments
Open

Examples: can remove cross-validation from outlier detection #24

jwmueller opened this issue Sep 16, 2022 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@jwmueller
Copy link
Member

jwmueller commented Sep 16, 2022

This example: https://github.com/cleanlab/examples/tree/master/outlier_detection_cifar10
could become much more straightforward without cross-validation now that we've seen it doesn't help too much in the train/test OOD settings.

But to truly assess the utility of cross-validation, we should first include an additional benchmark setting where we evaluate OOD detection on a training dataset with some outliers included (during training not just during testing). Cross-validation may still help in this setting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
Development

No branches or pull requests

1 participant