LITTLETai: right-sizing llama.cpp for big.LITTLE phones
llama.cpp defaults to every core on Android - the worst choice on big.LITTLE. LITTLETai reads your CPU layout and picks the right flags. 2.2-11x faster decode, up to 11x less energy. No patch.
llama.cpp defaults to every core on Android - the worst choice on big.LITTLE. LITTLETai reads your CPU layout and picks the right flags. 2.2-11x faster decode, up to 11x less energy. No patch.
Log in or sign up for Devpost to join the conversation.