llama.cpp 0.5.0

Overview

This release focuses on backend performance and correctness, broader model coverage, and more robust server/router operation. It adds HRM-Text (DFM Mimir 1B) support, MiMo-V2.6 and HunyuanOCR conversion support, ggml 0.25.0 backend improvements, multi-address HTTP binding, image outputs from function calls, and several chat parser/UI fixes.

Highlights

• Accelerate CUDA conv2d with implicit GEMM (#29135)
• Add Metal MoE and SSM_CONV fusion optimizations (#28948)
• Allow the server to bind to multiple addresses (#28690)

API changes

• Add llama_adapter_lora_init_from_file_ptr() for loading LoRA from an open FILE (#28993)
• Document llama_model_load_from_file_ptr() as reading from the current position and requiring aligned mmap (#28993)
• Add LLAMA_VOCAB_TYPE_TEST dummy tokenizer (#29084)
• Add input_image support to server function-call outputs (#22575)
• Allow --host to accept comma-separated TCP addresses and UNIX sockets (#28690)

New models

• Add HRM-Text / DFM Mimir 1B support (#27625)
• Add MiMo-V2.6 conversion support (#29257)
• Add DFlash support for HunyuanOCR (#28890)
• Extend Nemotron MTP and Nemotron-H model handling (#29018, #28989)
• Add Qwen4Exp hyper-connection ops and sparse flash attention (#28901, #28770)
• Add --fuse-qkv support for Muse Glimmer (#29203)

Core changes

• Add graph input/input-tensor diagnostics during scheduler reserve (#26625)
• Enable CUDA graphs for MTP drafting (#28549)
• Fix tensor-parallel split state/granularity for fused QKV models (#28965)
• Fix Mamba time-step projection input contiguity (#28832)
• Write the SWA pattern in the model saver and round-trip 15 more architectures (#29042)
• Add environment variables for temperature, top-p, min-p and penalties (#27380)
• Reduce the sampler backend probe size (#29285)
• Add Ling 3.0, DeepSeek V3.2/V4, qwen3-coder, Muse Glimmer and Gemma 4 parser fixes (#28682, #29008, #28869, #29242, #29115)
• Improve JSON Schema and PEG handling (#28518, #29127, #29161)
• Add ufakzeka pre-tokenizer and llama-bench --version (#29033, #28971)

Multi-modality changes

• Add sanity checks for mtmd layer indices, SAM layer counts, resize targets and graph allocation (#29276, #28149)
• Fix SigLIP bucket buffer overrun for tall/wide images (#29276)

Server changes

• Fix router eviction races and child process lifecycle handling (#29217)
• Do not pass log file or API key file to router-spawned children (#29212, #28938)
• Improve startup and model-source logging (#29125)
• Update vendored cpp-httplib to 0.57.1 (#29239)

UI changes

• Accept WEBM video files (#28622)
• Add close button to UI toasts (#28246)
• Fix mobile breakpoint and content overflow issues, including horizontal table scrolling (#29108)
• Restore the reasoning menu in single-model desktop mode (#27985)
• Stop re-probing a disabled /tools endpoint on every message (#28646)

ggml changes

• Updated ggml to v0.25.0 (release)
• The release expands hyper-connection, flash-attention, and fused MoE/SSM support across backends, with robustness, quantization, data-layout, and RPC/meta improvements.
• API changes include gated ggml_dsv4_hc_pre_gated(), optional ggml_dsv4_hc_post() comb, and RPC protocol major v7.

Nightly build: b11146

More info

• Releases and versioning of ggml-org projects

Changelog since v0.4.1

7fe450e19 llama.cpp : bump version to 0.5.0 (#29333) 177cd8cc7 sync : ggml e4e2f6232 ggml : bump version to 0.25.1 (ggml/1637) 66fba63af CUDA: add a reserve to avoid spurious warning on older GCC builds (#29317) bddf8263c common : keep HF cache dir as path, expose UTF-8 only for logs (#29320) 957538960 metal: add the missing f32 x bf16 mul_mv variants (#28741) dc9879cf6 CUDA: enable sparse-fa for dsv4 prefill (again) (#29298) 42916d83f server: fix token counting API crash on sleep (#29309) 4e416ee73 jinja : parse unary +/- before variables (#29244) ee3ecce05 metal : key the fa-vec tuned table by family instead of SKU (#29075) 057494f93 server: accept OpenAI video_url content type and data: video URIs (#27921) bcbc936a8 server: Dedup the draft HF model via dedup-cache-models (#27934) 26758d38f ci : fix build-cmake runner target (#29299) 18f9f7bef model-conversion : add causal-compare-logits recipe (#29305) 633733d0a model : support Gemma4 DSpark draft backbone (#29226) 86b2daa73 ci : run python (jinja) test (#29302) 183d2a04c make-release : update summary prompt 45062d405 sync : ggml 503549c5f ggml : bump version to 0.25.0 (ggml/1635) e97545d91 sycl : fix compile warnings b1ff4ca23 vulkan: add IQ4_XS MMQ/MMV matmul kernels (#28415) 94256114c ggml-meta: resolve multi buffer views (#29266) 1a679828f cuda: top-k MoE should always fire (#28432) 384a534ce sycl : support new UT case for mul_mat_hadamard fp16 (#29218) 5e48b3100 sycl: extend MMVQ GLU fusion, add rms_norm+scale and ssm_conv+silu fusions (#28931) 4d7d7703f sycl : support op get_rows_back, only support fp32/fp16 (#25266) 08b1d2aea vulkan: hide internal symbols to prevent duplicate-dlopen state destruction (#29139) 441df11f6 sampler: reduce the size of the probe (#29285) e6ab7c1a4 hex-dma: introduce direct-mapped DMA cache that is better suited for HVX FA mask handling (#29282) f46bc30cb HIP : optimize IQ2/IQ3 (__vsub4 __vcmpne4) using SWAR (#27962) 709fe755d jinja : fix dangling reference warning in for_statement (#29279) d5f66492e opencl: add bin kernel kernel_gemm_noshuffle_q4_k_q8_1_dp4a_ila_a8_bin (#29056) 991991118 server: fix router eviction races with the existing queue (#29217) bbf99b1b3 server: do not pass log file to children (#29212) 4098fdc92 server: support input_image in function_call_output (#20663) (#22575) 4ceb17191 vulkan: add Intel Xe flash attention optimization kernels (2/3, Xe-LPG Plus/Xe2/Xe3) (#24406) 73c941b11 mtmd: add various sanity checks (#29276) 0f8a414b7 metal : gate mul_mm_id src1 rescale behind ggml_prec (#29029) f95b0d953 ggml : IQ1_M build prefix sums once per block (#28706) c350a40bb Performance tune for gemma4-26b-a4b flash attention shape. (#28450) 9b421fa94 ui : Accept WEBM video files (#28622) 348f853b7 jinja: use const for statement::execute and ::visit (#29271) 217f81c26 server: Add support for binding to multiple addresses (#28690) 828fdf282 spec : support DFlash for HunyuanOCR (#28890) bfd73a876 convert: add MiMo-V2.6 support (#29257) a60f9aead cmake : allow repeated find_package calls for llama (#29228) 7ab4ee7ba chat : Fix Muse Glimmer tool-call first parser error (#29242) 0ee9435b8 ci : publish snapdragon builds in release workflow (#29007) 8cfc315a8 Add close button to UI toasts (#28246) ec5a12b85 opencl: add A8 Q4_0 non-MoE dp4a binary kernel (#29055) c550d2f60 ci : update Level Zero SDK to v1.33.1 and enable the L0/oneDNN CMake flags in the SYCL job (#29230) 58367713a hexagon: new HMX-optimized GATED_DELTA_NET (#29199) ff0dbb975 vendor : update cpp-httplib to 0.57.1 (#29239) fb34fc262 metal : fix mask bounds in flash attention block pre-pass (#29220) c641dfa83 test-save-load-state : compare logits with NMSE and feed expected tokens (#29238) 965506136 llama-context : report graph inputs and input tensors during sched reserve (#26625) b1c2863e2 cuda: fix sm_70 tile compilation error (#29224) f4e276a20 ggml-cuda : convert contiguous tensors four elements at a time (#29155) e6cef8152 cuda : accelerate conv2d with implicit GEMM (#29135) c21284cdf ggml : fix dimension and stride truncation in ggml_permute (#29227) 6f41ac59e vendor : update cpp-httplib to 0.57.0 (#29214) ec91ab5ad docker : bump cuda to 13.4.1 (#29207) bb3c853c3 sycl : support gated DSV4_HC_PRE and optional HC_POST comb matrix (#29132) af911149c sycl : pinned memory use right device context instead of 0 (#28895) 1884824fd CUDA: Follow up of #25635, refactoring FA shared smem swizzle (#28536) 161755f29 test-llama-archs : make tensor data stdev configurable and improve help (#29133) 1d72b05d3 tests/test-backend-ops : allow regex entries in the -o filter (#29204) 542e9202d ci : refactor build-self-hosted into backend-specific workflows (#28991) e0dff5847 args: add env vars for temperature, top-p, min-p and penalties (#27380) 982a3329a server : do not forward --api-key-file to router-spawned child instances (#28938) 711f60bee tests : remove stale comment (#29140) 335b21fcb ggml-metal : simplify fusion pattern op list declaration (#29206) 26394b4e6 json: Fixed json enum handling (#28518) 1aa2954bd sycl : coalesce MKL-FA softmax loads instead of one work-item per row (#28918) 8034c1d1f ggml-cpu: ARM Repack kernels for Q1_0 (#23492) 6ad1af560 ci : Upgrade CUDA to 13.4 for Ubuntu CUDA Release Builds (#29202) 0c3626ec0 hexagon: overhaul of buffer and DMA handling to support 64bit mappings + improvements (#29197) 68d9053af cuda : tune MMVQ to MMQ crossover for SM70 (Volta) (#28912) 8aa161b54 metal : fix deprecation warnings from macOS 27 SDK (#29136) 932a68e06 webgpu : add fused gdn + cpy (#28976) 62668d6b2 convert: enable --fuse-qkv for muse-glimmer (#29203) ce8caa6e6 CUDA: tune FA for Gemma 4 on Ampere or newer (#29152) a894dae93 metal : support arbitrary hc in dsv4_hc_pre (#29169) 3d82ef62d common/peg : handle invalid utf-8 sequences in the AST (#29161) 3cf03257f CUDA: enable sparse fa for qwen4 (#28770) b23efaa2e ui: Fix mobile breakpoint + content overflow issues (#29108) 426090367 fix(mamba) : make time-step projection input contiguous (#28832) 9a9f939b8 metal: add F16 input to the FWHT (#29094) f072b1037 chat : fix gemma4 required tool grammar (#29115) 59657a613 chat : add dedicated Ling 3.0 (Bailing V3) parser (#28682) e613ef2c8 hexagon: enable I32 GET_ROWS (#29116) 851cb34f2 hexagon: add support for GEGLU_QUICK (#29114) 7d4b92bb9 hexagon: enable support for TOP_K op (#29113) 1af554f8f server : improve startup log messages (#29125) eb1e1f495 json-schema : accept escaped hyphen in regex patterns (#29127) 5b59b83f4 metal : add MoE and SSM_CONV fusion optimizations (#28948) 60b06ab9a metal : fix FA support checks (#29122) efa28e950 test-llama-archs : generate dummy test vocab (#29084) 59fc5a1ca metal : support qwen4exp hc ops (#29000) b23701f77 cuda : fix CUB argsort corruption caused by in-place keys (#28389) 60081bb2b opencl: add support for bin kernel flash_attn_f32_f16_bin (#29046) 2b1847030 hexagon: add ROLL op support (#29105) 50631b3d2 hexagon: im2col update (#29103) 18a04f09c hexagon: HMX flash-attention head_dim padding (support DK=DV=72) (#26539) ec9281505 opencl: add bin kernel kernel_gemm_noshuffle_q6_k_f32_32b_trans_ila_a8_bin (#28678) 4fea119de ggml-cpu: add F16 input to the FWHT (#27779) 5b335f413 ggml : check for allocation failures to prevent crashes (#28149) 542348a35 Model-Saver: Write the SWA pattern, 15 more architectures roundtrip (#29042) d663dd3f3 ci: change ubuntu-latest to ubuntu-24.04 (#29079) 44be98f05 ggml-webgpu: fix supports_op condition for GET_ROWS (#28978) 911f6cdc8 ggml : handle graph buffer reservation failure (#26070) bbd488c42 vulkan: add IQ3_S MMQ matmul kernels (#28822) dc85f89c7 vocab : add ufakzeka pre-tokenizer (#29033) 8ed1a55ef cmake : fix build when GGML_CPU=OFF and GGML_CUDA=ON (#29026) bb11ebb68 gguf-py: fix Q8_1 block size in GGML_QUANT_SIZES (2+2+32) (#29036) f03cf3e9b ci : disable GHA cache for copilot (#29068) bdcbaaf6e ci : bump android-actions/setup-android to 4.0.4 (#29065) 5c53396b8 vulkan: raise the hoisted row-id limit for mul_mat_id from 256 to 512 experts (#28501)