llama.cpp 0.4.1

Overview

llama.cpp 0.4.1 adds Maple 20B-A1B, Tencent Hy 4, and Spark2.5 support, improves JSON schema handling, chat parsing, logging, and server child-process management, and updates ggml to v0.24.0.

API changes

• Changed llama_sampler_chain_n() to return int32_t instead of int (#28631).
• Added server_subproc and waiter for router child-process monitoring in server-common.h (#28555).

New models

• Added Maple 20B-A1B ternary MoE architecture (CPU) (#27000).
• Added Tencent Hy 4 (hy_v4) preview architecture support (#28127).
• Added Spark2.5 support (#27868).

Core changes

• Added Kimi-K3 recurrent-state rollback support (#28466).
• Fixed MTP context KV cache allocation for DeepSeek2, GLM-MoE, and related models (#28630).
• Fixed GDN normalization from max to rsqrt for affected Qwen/Kimi/GLM models (#28068).
• Fixed Granite parameter counts and Granite3 MoE unknown parameter counts (#28643, #28632).
• Fixed MIMO2 SWA pattern loading and incorrect get_key_or_arr uses (#28865, #28868).
• Added --fuse-qkv conversion flag to fuse Q/K/V tensors during HF-to-GGUF conversion (#22780).
• Wrote explicit recurrent_layers metadata for Qwen3-Next/Qwen3.5 conversion (#28208).
• Refactored JSON schema handling with a common_schema internal representation (#28736).
• Split specialized chat parsers into common/parsers (#27764).
• Added structured JSONL logging via --log-jsonl and LOG_JSON (#28437, #28586).
• Removed deprecated --mmap/--mlock/--direct-io args in favor of --load-mode (#28334).
• Made mmproj and draft devices default to the global --device selection (#28390).
• Disabled lazy tensor loading by default on iGPUs (#28326).
• Fixed speculative decoding after multimodal input and DFlash mtmd chunk decode (#28715, #28587).
• Fixed Jinja dot-property integer literals and null in membership (#28817, #28620).
• Improved grammar insert/move handling and max repetition threshold (#26885, #28469).
• Improved qwen3-coder complex-type parsing and typed content detection (#28742, #28511).
• Added PCH and unity build support to improve build times (#28091).
• Updated cpp-httplib to 0.56.0 (#28787).

Multi-modality changes

• Propagated video hash to bitmap IDs ({hash}+{frame}) for cache identification (#28601).
• Fixed DFlash speculative decoding failures on mtmd chunks (#28587).

Server changes

• Refactored server child-process handling into a single monitor thread (#28555).
• Fixed LRU hang on multiple requests for the same model (#28539).
• Allow model downloads when --models-max is already reached (#28530).
• Fixed context checkpoint eviction on prompts shorter than checkpoint_min_step (#28302).
• Fixed speculative decoding after image input (#28715).
• Frame router child state commands as whole lines (#28747).
• Enabled --reasoning-preserve by default (#28437).

UI changes

• Improved chat message rendering performance with lazy mounting and compositor-friendly animations (#28460).
• Added UI asset cache for faster builds (#28802).
• Embedded UI assets directly with CMake, removing the build-time helper (#28445).
• Fixed MCP image attachments not displayed in tool blocks (#28089).
• Export conversations from the database instead of the cached store (#27432).

ggml changes

• Bumped ggml from v0.23.0 to v0.24.0 (release).

Version 0.24.0 focuses on expanding backend coverage and robustness, with a new precision-control API, major Vulkan/SYCL/Hexagon/OpenCL work, and numerous correctness and performance fixes across CPU, CUDA, Metal, and other backends.

Nightly build: b10964

More info

• Releases and versioning of ggml-org projects

Changelog since v0.4.0

b29c606e2 llama.cpp : bump version to 0.4.1 (#28900) d9e03f107 sync : ggml eeea73161 ggml : bump version to 0.24.0 (ggml/1627) bbdd9f246 tests : add fusion baseline README and broaden fusion CI triggers (#28893) 97e4ca735 models : fix incorrect uses of get_key_or_arr (#28868) 1aca1f9fc models : fix mimo2 swa pattern load (#28865) be2c6d7d1 tests(s390x): add non-vxe build to tests (#28776) 3d10bcd19 llama: add Maple 20B-A1B ternary MoE architecture (CPU) (#27000) 21f6b0d22 sycl: rfc: Use radix select for top_k (#28670) 2f539596c ggml-cpu : disable PCH and fix CACHE_LINE_SIZE ambiguity to fix heap corruption (#28882) 89fe24240 ci : trigger self-hosted CI on changes to ci/run.sh (#28859) 15d8f2d59 ci : remove gg_sum summary logic (#28857) 661643e43 sycl : fix oneDNN scratchpad breaking the pool free order (#28704) 093a2f86c common : move llama_n_rs_seq to before llama_decode (#28749) ad6c66839 ggml-cuda: fallback to F32 on device without BF16 hardware acceleration (#28846) 7a16a6ce3 grammar : coalesce find + insert into a single insert and adjust move/copy mechanics (#26885) 5f436dddb tests : exclude HY_V4 from WebGPU test-llama-archs tests (#28855) e49d2c276 models : guard the expert FFN size fallback in nemotron-h against a zero divisor (#28779) 697805298 ggml-cpu(s390x): guard VXE-only repack helpers (#28775) 243a3082d tests : fix typo in test-quant-type-selection for nemotron 3 nano (#28835) b6b003d2c sycl : Fix get mem error (#28227) c95f8e47b ci : run editorconfig and code-style checks on ubuntu-slim (#28854) bc52a12b3 pi : prefer PI_MODEL_NAME env var for model disclosure (#28853) 4a8993735 tests : reduce FA test sizes (#28842) 37b3a9e0c ci : remove leftover command (#28839) 002a12ad2 ci : cap test-backend-ops parallel jobs at 2 and add a 3600s timeout (#28833) f1e44dcc1 vulkan: workaround NV queuesubmit driver bug (#28830) 56b9eb280 opencl: apply the noshuffle row-alignment rule to q4_K, q5_K and q8_0, not just q6_K (#28575) 790cf51aa chat : improve parsing of complex types in qwen3-coder (#28742) 8e330954a common: add LOG_JSON macro to log structured data (#28586) acecd5603 common : implement common_schema internal representation for JSON schemas (#28736) ae9afff8d jinja : support dot property integer literals (#28817) 737e0980f cmake: leave the timestamp out of precompiled headers on clang (#28816) 3057bb66c ui : add cache (#28802) 56381e407 server : allow model downloads at model limit fix issue #26809 (#28530) c8edceb06 ggml-cuda: hip add specific config table for AMD GCN (#27841) e192abb40 server : add missing headers (#28795) 718f7b417 vendor : update cpp-httplib to 0.56.0 (#28787) 2a3005c23 syscl : Handle (fail gracefully) unsupported tq1_0 quants (#28681) f3a33dff2 rpc : fix linking when compiling with BUILD_SHARED_LIBS=OFF (#28492) c069aa7f5 server: frame the router child state command as a whole line (#28747) 8a56aedd6 opencl: fix several bugs where the backend aborts (#27630) 07fc97716 opencl: add bin kernel kernel_gemm_noshuffle_q4_k_f32_32b_trans_ila_a8_bin (#28677) 3f5e94d7c webgpu: align tensor bindings to the type block size (#28382) eafe15a5e hexagon: support for multi-device model split (aka row-split) (#28589) d3146f2b5 ggml-webgpu: Update to a recent version of Dawn (#28683) 82d6bb284 server: refactor subproc handling (#28555) 8ea290247 cmake : skip PCH for llama-server PCH when using MSVC (#28763) b78a39a2f ci : run test-backend-ops as a dedicated ci/run.sh test (#28740) 982937a33 tests: extend test-quantize-fns to test nrc=2 (i8mm) kernels (#16234) 8172e6577 tests: tolerate a shared pool abort in test_completion_unified (#28759) 43f3dda62 ggml: skip 0-sized ids tensor when offloading selected experts (#28739) 5bda51bfb metal : skip the empty half of the mul_mm_id token tile (#28301) 3bcfeb700 cmake : add PCH and unity build to improve build times (#28091) 1dfe94e04 common : fix typo in speculative.cpp comment [no ci] (#28750) a2878d30d metal : single-source fusion table + fusion debug rework (#28164) aac810230 metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 (#28692) 5cdd3d1da model : fix MTP context kv cache allocation for deepseek2, glm4moe, ... (#28630) b0dcb8192 server: fix speculation after an image (#28715) 16378d93f CUDA/HIP: Flash Attention tuning (gfx1201) (#28102) 451b89bae ci : key cache to sanitizer matrix (#28708) 481c65f09 vulkan: fix data race and OOB access in argsort(large) (#28705) df03399b8 opencl: add A8 Q4_0 mm binary kernel support (#28268) 28ff09582 vulkan: use CPU writes in ggml_backend_vk_cpy_tensor_async if the context is idle (#28618) 50182a53f vulkan: use add_alloc_dep to enable topk_moe fusion for prefill (#28422) 6788edb4f vulkan: small M matrix optimizations for qwen (#28457) 52d426865 ci : add self-hosted-gpu-cuda and server-sanitize to hf-jobs (#28693) 18c17b4d6 ci : Update WoA CUDA 13.4 release to use 13.4.1 GA redistributables (#28687) fa6769818 spec: fix failed to decode mtmd chunk with DFlash (#28587) 41fc7584f scripts : use sed instead of grep for version parsing [no ci] (#28700) d344123fe models: clean up some dead switch branches in old models (#28669) c32d1dabe tests : increase tolerance for Add fusion tests (#28691) e5a8d439c tests : drop SYCL special-casing in test-backend-ops.cpp (#28688) 3ff67eb43 vulkan: fall back to shared-memory reduction for dmmv on PowerVR (#28341) 8c322d5bc convert : expand Nemotron H conversion fix (#28689) 311d4211b memory : avoid allocating V cache for indexer (it's not used) (#28330) 72797e891 vulkan : add command-buffer debug labels for GPU profilers (#28101) 4ea6d1bb6 ggml-cpu(s390x): add repack support for q4_0 (#28667) f1b6fbf35 ggml-cpu(s390x): add Q1_0 vector intrinsic support (#28606) d7e86430a model: fix all granite family parameter counts (#28643) 434ddbbc0 ci: fix sanitizer tests (#28583) 91f6a6cf3 vulkan: use spec constant for matrix matrix multiplication A-type (#25773) 6d9c82ea2 hexagon: rope updates (#28628) 22397c31a vulkan: Convert FILL to distribute workgroups in 2D to avoid exceeding maxComputeWorkGroupCount (#28592) 4b98ab805 py : lower numpy to 2.2.6 (#28654) 9cf3bf256 py : bump numpy to 2.4.6 (#28649) 4850c7727 llama : use int32_t for llama_sampler_chain_n return type (#28631) d4abd573f CUDA: size routed MoE MMQ N-tiles from typical expert width on RDNA3 (#28552) 5a4d0feca CUDA: replace GGML_FA_ALL_QUANTS with GGML_FA_QUANTS, more control over what is compiled (#28079) 14a9d09f7 args: officially deprecate --mmap|mlock|dio (#28334) e2d2c0d6a model: fix granite3 moe unknown parameter count (#28632) 6de9cdb26 mtmd: propagate video ID to bitmap (#28601) b31b71f3a jinja: treat a null left operand of in as a plain lookup (#28620) df750f76b vulkan: add dedicated iq4_xs mat-vec shader (#28426) 1945e0920 vulkan: add f16 B-type matmul pipelines and warp tile size tuning for Intel coopmat1 (#27471) 30b6a755e tests : use less threads for data initialization (#28325) 304665fe7 Add IQ type handling for MoE (#28476) f3f1a8f27 llama: disable lazy tensor loading by default on iGPUs (#28326) 9113cc188 ggml : fix msvc+clang ggml_vld1q_u32 (#28284) d4389a4dd Revert "ggml-cuda : restore prop.integrated on HIP builds (#24233)" (#28604) 5d806aa25 server : apply checkpoint min-step eviction only when the checkpoint list is full (#28302) 88ada91c1 metal : fix idle threads in mul_mv_iq3_xxs for ne00 < 1024 (#28086) 415e909d8 spec: single device drafter should create meta backend wrapper (#28390) 03fa73cb2 ci : disable npm gha cache (#28600) 1744c6bde ci : add PYTEST_WORKERS=1 to fix server-self-hosted job (#28603) ca86fb222 llama : add missing headers (#28566) 64e9bceb2 vulkan : fuse UNARY(GELU|SIGMOID|SILU|SOFTPLUS) + MUL (#27220) f014bfef8 Fix Vulkan-Hpp handle usage on 32-bit targets. (#22892) 895c045fd chat : split specialized parsers into common/parsers (#27764)