llama.cpp
C/C++ inference engine that powers most local LLM runners. Tagged releases.
Release history
llama.cpp b10644
Adds nanbeige4.2-3B model support
llama.cpp b10636
Ui: disable the npm UI build (LLAMA BUILD UI=OFF)
llama.cpp b10625
Scopes Qwen3-coder chat workarounds
llama.cpp 0.3.0
Llama.cpp 0.3.0 adds dots3-note multimodal support, tensor-split for DeepSeek 4, multi-token prediction for GLM-4.5-Air, and WebP image decoding
llama.cpp b10610
Metal flash-attention tuning: 53 new f16 instantiations, per-device dispatch tables
llama.cpp b10603
GLM-4.5-Air MTP support added to llama.cpp for improved model inference efficiency
llama.cpp b10588
Fixes clang LTO build issue
llama.cpp b10568
CI switches to LLVM OpenMP on Windows; Drops non-redist debug build
llama.cpp 0.2.0
Llama.cpp 0.2.0 syncs ggml to 0.21.0, adds backend optimizations for CUDA/Metal/SYCL/OpenCL, and improves quantization kernel performance across hardware
llama.cpp b10505
Adds dedup-cache-models preset option to server
llama.cpp b10486
Fixes LFM2 image tiling threshold; Cross-platform refactor
llama.cpp 0.1.2
Llama.cpp synced GGML 0.20.2, added integer tokenizer scores, improved CUDA performance on DGX, and fixed xcframework builds
llama.cpp b10470
CI; Explicitly pushes release tag before creating GitHub release
llama.cpp b10452
Removes some ggml concat calls; Model-layer refactor
llama.cpp b10447
Adds Kimi-K3 text-model support with hybrid KDA and MLA attention
llama.cpp b10434
Llama.cpp now passes reasoning_effort parameter through chat templates, enabling fine-grained control over model reasoning depth in inference
llama.cpp b10419
OpenVINO backend gains Qwen3.5 support, memory optimizations for GPU inference, and fixes for stateful RoPE accuracy and recurrent state handling
llama.cpp b10369
Build b10375 tightens bare function parsing for Qwen models
llama.cpp b10359
Switches ROCm target to 7.14, the first production release using TheRock build system
llama.cpp b10344
Added MTP (Multi-Token Prediction) support for Nemotron models, enabling faster inference for this architecture
llama.cpp b10329
Corrects server info endpoint to report the isolate working directory when a tools runtime is configured
llama.cpp b10326
TTS timing now accounts for full vocoder pass instead of single trailing window
llama.cpp b10290
AMD ROCm CI support for gfx1151; No inference changes
llama.cpp b10289
Fixed directory traversal vulnerabilities in file search, improved path handling on Windows with UTF-8 conversion, and added cache expiration for UI picker searches
llama.cpp b10271
Adds a per-conversation working directory with file picker; Agents can now treat path-like queries as directory navigation
llama.cpp b10236
Implements DSv4 Lightning Indexer and f16 Lightning Indexer for Metal; Affects 128-dimensional 64-head inputs on Apple silicon
llama.cpp b10232
WebGPU backend adds f16 repeat support; MacOS KleidiAI build disabled
llama.cpp b10219
Llama.cpp now preserves reasoning content in chat history, enabling multi-turn conversations to leverage previous model reasoning with --reasoning-preserve flag
llama.cpp b10212
Adds driver-version check for Intel GPUs on Windows to avoid crashes; No action unless running that config
llama.cpp b10195
Tests alternative convolution layout; Extends layout checks for conv2d kernel
llama.cpp b10182
Adds NextN and MTP speculative decoding for GLM-5.2; GLM DSA support
llama.cpp b10173
Added Laguna-S-2.1 model support to llama.cpp for local inference
llama.cpp b10153
MiMo-V2.5 RVQ-based audio input support added
llama.cpp b10105
Refactors mlock, mmap, and direct I/O into unified load-mode argument
llama.cpp b10099
CUDA NVFP4 W4A4 activation quantization improvements, fused per-channel amax and quantization kernels
llama.cpp b10090
Adds depthwise conv2d kernel for WebGPU backend, ported from Vulkan
llama.cpp b10078
Vulkan queue refactoring enables driver-side synchronization on supported hardware, reducing host-side mutex contention for GPU inference
llama.cpp b10067
K/V cache rotation fix for DFlash quantization workflows
llama.cpp b10064
OpenCL kernel gemm_moe_q6_k_f32_ns now loaded from binary kernel library. Binaries for macOS, iOS, Linux, and Windows published
llama.cpp b10051
MacOS, iOS, and Linux build artifacts now included in release notes
llama.cpp b10032
Fixes OpenCL buffer creation for CL 2.x; MacOS arm64 and Linux CPU builds included
llama.cpp b10012
Adds contiguity checks for inner tensor dimensions
llama.cpp b9992
Corrects server endpoint to honour per-request reasoning-budget tokens in chat completions
llama.cpp b9977
OpenCL int8 dp4 dense and MoE GEMM optimization for Adreno GPUs
llama.cpp b9959
Llama.cpp synced ggml changes, updating core ML infrastructure for model inference
llama.cpp b9952
Llama.cpp optimizes DeepSeek V4 inference with f16 KQ masks, removes unnecessary attention bias, and eliminates redundant cache repeats for faster execution
llama.cpp b9945
Hexagon: tiles wide rows in unary ops to avoid VTCM overflow, replaces divs with fastdiv, adds VTCM layout and host-computed offsets
llama.cpp b9910
Fuses MMVQ post-scale and batch-size-1 path for NVFP4
llama.cpp b9890
Adds Metal col2im 1d op support for f32, f16, bf16
llama.cpp b9878
Fixes stale tensor-split metadata for GQA attention on draft models
llama.cpp b9873
Adds guard to prevent dereferencing unallocated K/V rotation buffers; Defensive fix for edge-case crashes
llama.cpp b9866
Trims messages sent to StepFun parser, fixes long reasoning loops
llama.cpp b9860
Adds llama_model_ftype_name function to expose quantization type as string
llama.cpp b9857
Uses Hugging Face primary split as model path; Fixes issue 25181
llama.cpp b9849
Registers attention normalization tensor for Qwen3; Fixes input assignment in layer loop
llama.cpp b9839
Adds DeepSeek V4 conversion support including pro model, rope fixes, and chat template
llama.cpp b9835
Fixes UI stop and reasoning skip in single-model mode
llama.cpp b9825
Vulkan backend fix for zero-input step operator handling in llama.cpp inference
llama.cpp b9821
Adds support for version, licenses, and help flags
llama.cpp b9788
Fixes quantizing MoE models with mtp
llama.cpp b9776
Adds LFM2.5-ColBERT-350M and LFM2.5-Embedding-350M support; No impact unless loading those weights
llama.cpp b9767
Improves WebGPU mat-vec path for small-batch decoding
llama.cpp b9760
Relevant if running the server component and permitting file edits, appends at end of file no longer trigger heap overflow
llama.cpp b9752
Server batch construction refactored for more careful slot handling and improved abort logic
llama.cpp b9739
WebGPU adapter toggles for F16 on Vulkan and NVIDIA. Relevant only if running those backends
llama.cpp b9732
Server context-shift fix for non-bound n_discard value. Upgrade if running server mode with ctx shifting
llama.cpp b9694
Fixed Windows x64 OpenVINO release build link to resolve CI pipeline failures
llama.cpp b9690
Optional USM system allocations for buffers over 1GB if the device supports it
llama.cpp b9670
Adds Vulkan support for gated delta net with S v=16
llama.cpp b9642
CUDA operators note F32/F16 restriction on GGML OP REPEAT; KleidiAI macOS build remains disabled
llama.cpp b9628
Adds Cohere2MoE vocab for TINY AYA. No impact unless loading those weights
llama.cpp b9611
Llama.cpp refactored build dependencies to reduce coupling between fit and llama-ext headers
llama.cpp b9594
Refactors normalizer flags into options struct and adds strip-accents support. No model or inference changes
llama.cpp b9589
CUDA: Fix ssm_scan_f32 data-races missing syncthreads calls to prevent shared memory corruption
llama.cpp b9585
Operators running llama.cpp with plamo2 models should upgrade. Fixes an attention key-value length regression introduced in a prior build
llama.cpp b9568
Fixes im2col 1D case for audio models. Relevant only if running audio inference
llama.cpp b9553
Sampler name matching now case-insensitive and accepts alternative names like top-k alongside canonical top_k format
llama.cpp b9538
Qwen3.5 video support added; No impact unless loading those weights
llama.cpp b9533
Build system fix resolves compilation failures in llama.cpp, enabling reliable local LLM inference on consumer hardware
llama.cpp b9498
WebGPU FlashAttention refactored with standardized quantization support, relevant if running WebGPU backend
llama.cpp b9486
Adds support for qwen3 SSM architectures
llama.cpp b9473
SWA checkpoint optimization reduces KV cache storage masked cells, improving memory efficiency for inference
llama.cpp b9444
Adds tokenizer support for jina-embeddings-v2-base-zh. No impact unless loading those specific embedding weights
llama.cpp b9434
Fixed tensor parallelism granularity for Qwen 3.5/3.6 models across multiple GPUs
llama.cpp b9414
Adds DeepSeekOCR 2 support with multi-tile dynamic resolution
llama.cpp b9368
Mesa users on Intel BMG should see a 4.8 percent tg128 improvement for Qwen3.5-9B BF16 workloads
llama.cpp b9334
CUDA users should upgrade for the PDL sync fix; MacOS KleidiAI artifacts now shipping
llama.cpp b9305
Fixes UI build and adds macOS KleidiAI artifacts. No impact unless building the UI or deploying on Apple Silicon with KleidiAI
llama.cpp b9297
Adds NVFP4 MTP scale tensors and links Qwen3.5 MTP tensors. No action unless running Qwen3.5 or NVFP4 hardware
llama.cpp b9289
SYCL backend now supports gated_delta_net operations with K 1 for improved inference performance on Intel GPUs
llama.cpp b9258
Merges HunyuanOCR into HunyuanVL vision path and restores bilinear sampler precision for OCR tasks
llama.cpp b9245
RDNA3 GPU kernels for Q6_K quantized inference now use tuned warp configurations for better performance
llama.cpp b9222
Adds Hexagon TRI op support; Matters only if targeting Qualcomm HTP acceleration
llama.cpp b9216
Llama.cpp UI refactored models store, MCP service, and debug logging behind environment flags to reduce noise and improve performance
llama.cpp b9198
Vulkan build now checks for SPIRV-Headers availability upfront to catch configuration issues before compilation starts
llama.cpp b9174
Moves web UI output directory and renames CMake variables. Existing build scripts may need path adjustments
llama.cpp b9156
Adds Qwen3.5 tokenizer support. No impact unless loading Qwen3.5 weights
llama.cpp b9129
Llama.cpp ZenDNN backend now adaptively falls back to CPU for small batch sizes, improving efficiency via runtime control