Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardising matrix arrays (adopt Array API spec) #941

Open
KennethEnevoldsen opened this issue Jun 17, 2024 · 0 comments
Open

Standardising matrix arrays (adopt Array API spec) #941

KennethEnevoldsen opened this issue Jun 17, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@KennethEnevoldsen
Copy link
Contributor

Currently MTEB does not consistently use a ground truth for vector manipulation. A solution would be to adopt the Array API spec. A potentially simpler solution is to use either torch or numpy.

  • The benefit of using numpy is that it is very stable and rarely leads to OOM and related GPU issues.
  • The benefit of using PyTorch is that it potentially allows evaluation using GPUs (currently, sklearn does not support this, but might in the future, when they adopt they API spec)

partly related to the discussion in: #883

This is not a pressing concern atm.

@KennethEnevoldsen KennethEnevoldsen added the enhancement New feature or request label Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant