Allow passing batch_size via CLI

[Muennighoff]

I think it'd make sense to allow passing the batch_sizekwarg when using the cli argument mteb run ...? Some missing results are due to OOMs (#1014)

[isaac-chung]

Yeah. And within the tasks / evaluators, batch_size could be passed along as a kwarg?

[KennethEnevoldsen]

That has been the approach so far. I would actually probably prefer having encode_kwargs as an argument for the MTEB(model, encode_kwargs={"batch_size": 16}). Batch size seem very specific. This also allow passing e.g. length normalization.

We can still add the batch_size argument to the CLI

[Muennighoff]

Yeah. And within the tasks / evaluators, batch_size could be passed along as a kwarg?

As @KennethEnevoldsen said you can already pass it in evaluation.run like e.g. here: https://github.com/ContextualAI/gritlm/blob/0cc9aeab83b90f2e22bcdd2b084d51507c624d95/evaluation/eval_mteb.py#L1206

Maybe it would help having that in the docs / one of the README examples so people know 🤔

[KennethEnevoldsen]

Yea. I was actually hoping to transition away from using the batch_size arguments (essentially it is a model argument, but it is passed on to the task). I also don't think it is consistently implemented for all tasks types.

[KennethEnevoldsen]

I am more than happy to implement the encode_kwargs if people think it is a good idea.