Runing llama2.c on a microcontroller

[mahdi259]

Hi
Thanks for your valuable effort.
Is it possible to compile llama2.c project to be run on a microcontroller without operating system?

[trholding]

It is theoretically possible, but almost all uC boards lack the flash to store the models and have insufficient ram. You'd need to design a uC board with sufficient RAM/PSRAM that has a decent uC such as ESP32. With some changes to llama2.c it could be built for baremetal using the toolchain (compiler) for your uC. You should use the 8bit quantized llama2.c if you plan to do so to save resources.

This project comes close: https://github.com/maxbbraun/llama4micro