SynapseAI v1.16.0
The codebase is validated with SynapseAI 1.16.0 and optimum-habana 1.12.0.
Tested configurations
- LLama2 7B BF16 / FP8 on 1xGaudi2
- LLama2 70B BF16 / FP8 on 8xGaudi2
- Falcon 180B BF16 / FP8 on 8xGaudi2
- Mistral 7B BF16 / FP8 on 1xGaudi2
- Mixtral 8x7B BF16 / FP8 on 1xGaudi2
Highlights
- Add support for grammar feature
- Add support for Habana Flash Attention
Full Changelog: v2.0.0...v2.0.1