Why no open source llama/orca/platypus submissions?

[aipocdtc]

Hey guys, I notice that there are no open source llama/orca or other pure decoder submissions (ok there's one, but it seems quite incomplete and the spelling makes me sus: https://huggingface.co/Shimin/LLaMA-embeeding).

Why is this?

Word on the street is that decoder only models make bad embeddings in general, but has anyone tested this assumption with any rigor? I see that ada-002 is up there in the top 10, which means they can't be awful.

[Muennighoff]

Apart from ada-002 & the one you mentioned, the SGPT models are the only decoder-only models. They are a bit outdated now, so they are no longer at the top of the leaderboard (still quite good on the retrieval ranking though).

I will try to find some time to create an updated version of them using Llama 2 & the latest techniques 👍

Edit: GritLM is the updated version I created with several collaborators, check it here: https://github.com/ContextualAI/gritlm