Launches · Project
llama.cpp
llama.cpp is a C and C++ runtime for running language and multimodal models on local hardware.
+ FollowBuilder facts
- Weights. The quantization tool converts higher-precision GGUF weights to lower-precision GGUF files, reducing size with a possible accuracy cost. Source ↗
- Hardware. The runtime supports CPU execution and accelerator backends including Metal, CUDA and Vulkan. Source ↗
- Serving. The local server supports continuous batching and OpenAI-compatible chat, Responses and embedding endpoints. Source ↗
- Access. The MIT-licensed runtime uses separately obtained compatible model weights and can run on infrastructure you control. Source ↗
Sources checked
Publisher update ·
llama.cpp 1.0.0 ↗Makes the NextN speculative decoding API public in llama.h and adds an OpenAI-compatible speech endpoint.
Change timeline
0 storiesNo verified releases in this timeline yet.