Launches · Project

llama.cpp

llama.cpp is a C and C++ runtime for running language and multimodal models on local hardware.

+ Follow

Builder facts

  • Weights. The quantization tool converts higher-precision GGUF weights to lower-precision GGUF files, reducing size with a possible accuracy cost. Source ↗
  • Hardware. The runtime supports CPU execution and accelerator backends including Metal, CUDA and Vulkan. Source ↗
  • Serving. The local server supports continuous batching and OpenAI-compatible chat, Responses and embedding endpoints. Source ↗
  • Access. The MIT-licensed runtime uses separately obtained compatible model weights and can run on infrastructure you control. Source ↗

Sources checked

Publisher update ·

llama.cpp 1.0.0 ↗

Makes the NextN speculative decoding API public in llama.h and adds an OpenAI-compatible speech endpoint.

Change timeline

0 stories

No verified releases in this timeline yet.