llama.cpp
C/C++ inference engine that powers most local LLM runners. Tagged releases.
Release history
llama.cpp b11808
Llama.cpp adds optional FC input scale loading for improved inference flexibility in quantized models
llama.cpp b11803
Added support for Flint model (FlintForCausalLM) to enable inference of this architecture in llama.cpp
llama.cpp b11748
SYCL backend warning cleanup only; No functional change
llama.cpp b11747
Fixes CUDA flash-attention race conditions; Relevant for anyone running GPU inference who's seen instability
llama.cpp b11717
Hexagon quantized inference now supports hadamard operations and flat/tiled row access for Q1_0, Q2_0 quantizations, reducing packet overhead
llama.cpp b11716
Hexagon backend now supports Q1_0 and Q2_0 quantization formats with optimized tiling and vector operations for faster inference
llama.cpp b11680
Hexagon tensor repacking reworked to support tiled and flat quantized weights, improving vectorized operations and memory efficiency on Qualcomm hardware
llama.cpp b11640
Vulkan backend gets L2-friendly workgroup scheduling for matrix multiplication; Relevant for Vulkan GPU inference performance
llama.cpp b11608
WebGPU fix for duplicate expert IDs in mixture-of-experts matmul; Relevant only to WebGPU backend users
llama.cpp b11580
Extends Hexagon DMA scalar broadcasting for binary ops; Relevant only for Hexagon-accelerated builds
llama.cpp b11577
Vulkan backend tweaks for CM2 quant block sizing; Relevant only if running Vulkan-accelerated builds
llama.cpp b11559
OpenCL GPU decode optimizations for Gemma-4 and improved DK64/DK128 performance with grouped query attention configurations
llama.cpp b11552
Fixes a server bug where busy slots ran prompt cache updates before deferring; Relevant for multi-slot server deployments
llama.cpp b11539
SYCL backend gets Q5_K reorder-layout MMVQ and fused GLU; Relevant mainly for Intel GPU inference setups
llama.cpp b11514
Removes duplicate SYCL block-size defines; Minor backend cleanup, no functional change for most users
llama.cpp b11459
WebGPU backend now works on WASI without requiring Dawn native features, enabling GPU acceleration for llama.cpp in WebAssembly environments
llama.cpp b11456
Fixes a k-pool scatter data race affecting shared sequences; Relevant for multi-session inference setups
llama.cpp b11433
CUDA kernel change tiles the lightning indexer for low-head-count cases; Relevant for custom inference tuning, not a default-path change
llama.cpp 0.6.0
Adds extended batch API for mixed token/embedding inputs, supports GLM-5.3-Flash and decision models, enables MTP speculative decoding and Metal flash attention optimizations
llama.cpp b11398
CI/build-pipeline fix for tag pushing; No functional change for end users
llama.cpp b11379
Fixes a soft-max-back output bug triggered aliasing; Affects custom ggml backprop usage
llama.cpp b11335
Fixes CUDA MMVQ routing for Volta (sm70) GPUs, relevant only to that hardware generation
llama.cpp b11311
Replaces alloca() with std::vector in ggml-opencl
llama.cpp b11270
Optimizes hexagon concat op. Reduces packets in gather/transpose hot loop. Uses fastdiv to replace software divide. Optimizes DMA-HVX pipeline
llama.cpp b11246
Fixes Muse Glimmer ignoring response format json schema when using --jinja
llama.cpp b11240
Llama.cpp now supports vision, audio, and video inputs for the /v1/embeddings endpoint using OpenAI-compatible multimodal content arrays
llama.cpp b11222
Server now supports RANK pooling batch splitting for causal LLM rerankers
llama.cpp b11205
Added CUDA support for Nemotron 3 Puzzle state size 96 SSM scan operations
llama.cpp b11191
Fixes unicode path handling on Windows; Directory creation now creates last directory when no trailing separator is present
llama.cpp b11172
Rejects MUL_MAT_ID when src1 precision is F32 on Hexagon
llama.cpp b11149
HIP optimization for IQ2 and IQ3 using SWAR bit manipulation for vcmpne4 and vsub4
llama.cpp 0.5.0
CUDA convolution acceleration, Metal MoE/SSM fusion, multi-address server binding, expanded model support including HunyuanOCR, and improved tensor-parallel handling for fused QKV architectures
llama.cpp b11114
New HMX-optimized GATED DELTA NET with pipelining
llama.cpp b11090
Fixed CUDA SM_70 tile compilation error load_ldmatrix shape handling for Volta architecture compatibility
llama.cpp b11060
CUDA path: enables sparse flash attention for Qwen4 models
llama.cpp b11057
Llama.cpp added specialized Ling 3.0 parser handling tool calls and reasoning blocks correctly in streaming chat templates
llama.cpp b11045
Added OpenCL support for bin kernel flash attention f32 f16
llama.cpp b11026
Skips gate-up exps if TENSOR_SKIP is set; Required for qwen35moe when MTP tensors are fused but not loaded
llama.cpp b11009
Fixed fused QKV attention layer calculations for Gemma-4 and Qwen-3.5, enabling correct tensor parallelism splits when embedding dimensions don't match head counts
llama.cpp b10988
OpenCL MoE expert matmul selection size for speculative decoding; Gated prebuilt q4_0 MoE GEMM on routing count
llama.cpp b10970
Flash-attention on CDNA MFMA devices now uses fp32 accumulators
llama.cpp 0.4.1
Llama.cpp 0.4.1 adds Maple/Hy/Spark2.5 model support, fixes MoE/normalization bugs, refactors JSON schema handling, and updates ggml to v0.24.0
llama.cpp b10947
Refactored JSON schema handling to use common internal representation; Updated json-schema-to-grammar
llama.cpp b10932
Updates ggml-webgpu to a recent Dawn version
llama.cpp b10907
Fixed KV cache allocation for MTP context in DeepSeek2, GLM4MoE, and Cohere2MoE model architectures
llama.cpp b10899
Vulkan small M matrix optimizations for qwen: optimize m=1 mul mat A/B, allow split k with small M
llama.cpp b10877
Converts FILL to 2D workgroup distribution to avoid exceeding maxComputeWorkGroupCount on Intel GPUs
llama.cpp b10864
Reverts HIP build prop.integrated change
llama.cpp b10850
Fixed L2_NORM batch array initialization to resolve GCC compiler warnings in aarch64 Release builds
llama.cpp b10828
Fixes GDN normalization to use rsqrt instead of max for gated delta net q/k, matching flash-linear-attention spec
llama.cpp b10818
SYCL backend fixes and Kronecker FWHT support restored
llama.cpp 0.4.0
Adds Qwen3.8-Flash-Next and Nemotron-3-Puzzle support, on-demand tensor reading, per-slot context limits, video input, GGML 0.23.0 with sparse attention and RDMA
llama.cpp b10813
Build 10794 refactors SYCL to use global var for GGML SYCL ENABLE MKL FA
llama.cpp b10791
Fixes idefics3 preprocessing in mtmd module
llama.cpp b10754
Fixes out-of-bounds reads in Adreno OpenCL image kernels; Clamps q4 K decode GEMV fetch row on padded grids
llama.cpp b10750
Adds recurrent-state rollback support for MTP speculative decoding in Qwen4 experimental models
llama.cpp b10726
Adds Vulkan top-k radix sort shader for Qwen 3.8 Flash Next
llama.cpp b10689
Allows passing alloc dependencies in graph optimize; Adds alloc dependency tests
llama.cpp b10687
Optimized Adreno GPU matrix multiplication kernels, improving prefill performance% on X2E and 9% on A7X generations through better GEMM routing
llama.cpp b10678
Reduces number of graph splits for Qwen4Exp model
llama.cpp b10662
Adds nanbeige4.2-3B model support
llama.cpp b10636
Ui: disable the npm UI build (LLAMA BUILD UI=OFF)
llama.cpp b10625
Scopes Qwen3-coder chat workarounds
llama.cpp 0.3.0
Llama.cpp 0.3.0 adds dots3-note multimodal support, tensor-split for DeepSeek 4, multi-token prediction for GLM-4.5-Air, and WebP image decoding
llama.cpp b10610
Metal flash-attention tuning: 53 new f16 instantiations, per-device dispatch tables
llama.cpp b10603
GLM-4.5-Air MTP support added to llama.cpp for improved model inference efficiency
llama.cpp b10588
Fixes clang LTO build issue
llama.cpp b10568
CI switches to LLVM OpenMP on Windows; Drops non-redist debug build
llama.cpp 0.2.0
Llama.cpp 0.2.0 syncs ggml to 0.21.0, adds backend optimizations for CUDA/Metal/SYCL/OpenCL, and improves quantization kernel performance across hardware
llama.cpp b10505
Adds dedup-cache-models preset option to server
llama.cpp b10486
Fixes LFM2 image tiling threshold; Cross-platform refactor
llama.cpp 0.1.2
Llama.cpp synced GGML 0.20.2, added integer tokenizer scores, improved CUDA performance on DGX, and fixed xcframework builds
llama.cpp b10470
CI; Explicitly pushes release tag before creating GitHub release
llama.cpp b10452
Removes some ggml concat calls; Model-layer refactor
llama.cpp b10447
Adds Kimi-K3 text-model support with hybrid KDA and MLA attention
llama.cpp b10434
Llama.cpp now passes reasoning_effort parameter through chat templates, enabling fine-grained control over model reasoning depth in inference
llama.cpp b10419
OpenVINO backend gains Qwen3.5 support, memory optimizations for GPU inference, and fixes for stateful RoPE accuracy and recurrent state handling
llama.cpp b10369
Build b10375 tightens bare function parsing for Qwen models
llama.cpp b10359
Switches ROCm target to 7.14, the first production release using TheRock build system
llama.cpp b10344
Added MTP (Multi-Token Prediction) support for Nemotron models, enabling faster inference for this architecture
llama.cpp b10329
Corrects server info endpoint to report the isolate working directory when a tools runtime is configured
llama.cpp b10326
TTS timing now accounts for full vocoder pass instead of single trailing window
llama.cpp b10290
AMD ROCm CI support for gfx1151; No inference changes
llama.cpp b10289
Fixed directory traversal vulnerabilities in file search, improved path handling on Windows with UTF-8 conversion, and added cache expiration for UI picker searches
llama.cpp b10271
Adds a per-conversation working directory with file picker; Agents can now treat path-like queries as directory navigation
llama.cpp b10236
Implements DSv4 Lightning Indexer and f16 Lightning Indexer for Metal; Affects 128-dimensional 64-head inputs on Apple silicon
llama.cpp b10232
WebGPU backend adds f16 repeat support; MacOS KleidiAI build disabled
llama.cpp b10219
Llama.cpp now preserves reasoning content in chat history, enabling multi-turn conversations to leverage previous model reasoning with --reasoning-preserve flag
llama.cpp b10212
Adds driver-version check for Intel GPUs on Windows to avoid crashes; No action unless running that config
llama.cpp b10195
Tests alternative convolution layout; Extends layout checks for conv2d kernel
llama.cpp b10182
Adds NextN and MTP speculative decoding for GLM-5.2; GLM DSA support
llama.cpp b10173
Added Laguna-S-2.1 model support to llama.cpp for local inference
llama.cpp b10153
MiMo-V2.5 RVQ-based audio input support added
llama.cpp b10105
Refactors mlock, mmap, and direct I/O into unified load-mode argument
llama.cpp b10099
CUDA NVFP4 W4A4 activation quantization improvements, fused per-channel amax and quantization kernels
llama.cpp b10090
Adds depthwise conv2d kernel for WebGPU backend, ported from Vulkan
llama.cpp b10078
Vulkan queue refactoring enables driver-side synchronization on supported hardware, reducing host-side mutex contention for GPU inference
llama.cpp b10067
K/V cache rotation fix for DFlash quantization workflows
llama.cpp b10064
OpenCL kernel gemm_moe_q6_k_f32_ns now loaded from binary kernel library. Binaries for macOS, iOS, Linux, and Windows published
llama.cpp b10051
MacOS, iOS, and Linux build artifacts now included in release notes
llama.cpp b10032
Fixes OpenCL buffer creation for CL 2.x; MacOS arm64 and Linux CPU builds included
llama.cpp b10012
Adds contiguity checks for inner tensor dimensions
llama.cpp b9992
Corrects server endpoint to honour per-request reasoning-budget tokens in chat completions
llama.cpp b9977
OpenCL int8 dp4 dense and MoE GEMM optimization for Adreno GPUs
llama.cpp b9959
Llama.cpp synced ggml changes, updating core ML infrastructure for model inference
llama.cpp b9952
Llama.cpp optimizes DeepSeek V4 inference with f16 KQ masks, removes unnecessary attention bias, and eliminates redundant cache repeats for faster execution
llama.cpp b9945
Hexagon: tiles wide rows in unary ops to avoid VTCM overflow, replaces divs with fastdiv, adds VTCM layout and host-computed offsets
llama.cpp b9910
Fuses MMVQ post-scale and batch-size-1 path for NVFP4
llama.cpp b9890
Adds Metal col2im 1d op support for f32, f16, bf16
llama.cpp b9878
Fixes stale tensor-split metadata for GQA attention on draft models
llama.cpp b9873
Adds guard to prevent dereferencing unallocated K/V rotation buffers; Defensive fix for edge-case crashes
llama.cpp b9866
Trims messages sent to StepFun parser, fixes long reasoning loops
llama.cpp b9860
Adds llama_model_ftype_name function to expose quantization type as string
llama.cpp b9857
Uses Hugging Face primary split as model path; Fixes issue 25181
llama.cpp b9849
Registers attention normalization tensor for Qwen3; Fixes input assignment in layer loop
llama.cpp b9839
Adds DeepSeek V4 conversion support including pro model, rope fixes, and chat template
llama.cpp b9835
Fixes UI stop and reasoning skip in single-model mode
llama.cpp b9825
Vulkan backend fix for zero-input step operator handling in llama.cpp inference
llama.cpp b9821
Adds support for version, licenses, and help flags
llama.cpp b9788
Fixes quantizing MoE models with mtp
llama.cpp b9776
Adds LFM2.5-ColBERT-350M and LFM2.5-Embedding-350M support; No impact unless loading those weights
llama.cpp b9767
Improves WebGPU mat-vec path for small-batch decoding
llama.cpp b9760
Relevant if running the server component and permitting file edits, appends at end of file no longer trigger heap overflow
llama.cpp b9752
Server batch construction refactored for more careful slot handling and improved abort logic
llama.cpp b9739
WebGPU adapter toggles for F16 on Vulkan and NVIDIA. Relevant only if running those backends
llama.cpp b9732
Server context-shift fix for non-bound n_discard value. Upgrade if running server mode with ctx shifting
llama.cpp b9694
Fixed Windows x64 OpenVINO release build link to resolve CI pipeline failures
llama.cpp b9690
Optional USM system allocations for buffers over 1GB if the device supports it
llama.cpp b9670
Adds Vulkan support for gated delta net with S v=16
llama.cpp b9642
CUDA operators note F32/F16 restriction on GGML OP REPEAT; KleidiAI macOS build remains disabled
llama.cpp b9628
Adds Cohere2MoE vocab for TINY AYA. No impact unless loading those weights
llama.cpp b9611
Llama.cpp refactored build dependencies to reduce coupling between fit and llama-ext headers
llama.cpp b9594
Refactors normalizer flags into options struct and adds strip-accents support. No model or inference changes
llama.cpp b9589
CUDA: Fix ssm_scan_f32 data-races missing syncthreads calls to prevent shared memory corruption
llama.cpp b9585
Operators running llama.cpp with plamo2 models should upgrade. Fixes an attention key-value length regression introduced in a prior build
llama.cpp b9568
Fixes im2col 1D case for audio models. Relevant only if running audio inference
llama.cpp b9553
Sampler name matching now case-insensitive and accepts alternative names like top-k alongside canonical top_k format
llama.cpp b9538
Qwen3.5 video support added; No impact unless loading those weights
llama.cpp b9533
Build system fix resolves compilation failures in llama.cpp, enabling reliable local LLM inference on consumer hardware
llama.cpp b9498
WebGPU FlashAttention refactored with standardized quantization support, relevant if running WebGPU backend
llama.cpp b9486
Adds support for qwen3 SSM architectures
llama.cpp b9473
SWA checkpoint optimization reduces KV cache storage masked cells, improving memory efficiency for inference
llama.cpp b9444
Adds tokenizer support for jina-embeddings-v2-base-zh. No impact unless loading those specific embedding weights
llama.cpp b9434
Fixed tensor parallelism granularity for Qwen 3.5/3.6 models across multiple GPUs
llama.cpp b9414
Adds DeepSeekOCR 2 support with multi-tile dynamic resolution
llama.cpp b9368
Mesa users on Intel BMG should see a 4.8 percent tg128 improvement for Qwen3.5-9B BF16 workloads
llama.cpp b9334
CUDA users should upgrade for the PDL sync fix; MacOS KleidiAI artifacts now shipping
llama.cpp b9305
Fixes UI build and adds macOS KleidiAI artifacts. No impact unless building the UI or deploying on Apple Silicon with KleidiAI
llama.cpp b9297
Adds NVFP4 MTP scale tensors and links Qwen3.5 MTP tensors. No action unless running Qwen3.5 or NVFP4 hardware
llama.cpp b9289
SYCL backend now supports gated_delta_net operations with K 1 for improved inference performance on Intel GPUs
llama.cpp b9258
Merges HunyuanOCR into HunyuanVL vision path and restores bilinear sampler precision for OCR tasks
llama.cpp b9245
RDNA3 GPU kernels for Q6_K quantized inference now use tuned warp configurations for better performance
llama.cpp b9222
Adds Hexagon TRI op support; Matters only if targeting Qualcomm HTP acceleration
llama.cpp b9216
Llama.cpp UI refactored models store, MCP service, and debug logging behind environment flags to reduce noise and improve performance
llama.cpp b9198
Vulkan build now checks for SPIRV-Headers availability upfront to catch configuration issues before compilation starts
llama.cpp b9174
Moves web UI output directory and renames CMake variables. Existing build scripts may need path adjustments
llama.cpp b9156
Adds Qwen3.5 tokenizer support. No impact unless loading Qwen3.5 weights
llama.cpp b9129
Llama.cpp ZenDNN backend now adaptively falls back to CPU for small batch sizes, improving efficiency via runtime control