projectNala
A pure C++ LLM inference server optimized for Armv9-A cloud instances. Leverages KleidiAI for maximum performance, exposing a lightweight, OpenAI-compatible REST API for GGUF models.
A pure C++ LLM inference server optimized for Armv9-A cloud instances. Leverages KleidiAI for maximum performance, exposing a lightweight, OpenAI-compatible REST API for GGUF models.
Log in or sign up for Devpost to join the conversation.