LocalAI 4.9.0

LocalAI 4.9.0 Release!

LocalAI 4.9.0 is out!

Thirteen days and 146 pull requests, spent on the parts of LocalAI you touch every day rather than on new engines. Authentication is now deny-by-default, chat gained end-to-end context compression, models and backends each have one canonical page instead of three, and vllm-cpp grew a video modality serving MiniMax-H3 with a real audio track.

Highlights:

• Authentication is deny-by-default - every HTTP route requires credentials unless it appears in an explicit public registry. This closes a class of bypass in which unprefixed aliases such as /moderations, /models, /backends and /mcp/chat/completions fell outside the old protected-prefix list. Reported by Naor Yaacov.
• Chat context compression - opt-in per model, older complete turns are compressed through a LocalAI model before inference, preserving system prompts, the newest messages and whole tool-call units. Ratio and duration come back as response metadata and metrics.
• One page per resource - /app/models now owns Explore and Installed, /app/backends owns Catalog and Installed, and the nested Host view is gone. Old /app/manage bookmarks still work.
• MiniMax-H3 video generation - vllm-cpp opens a second engine handle for the H3 checkpoint set and renders video and audio jointly, so the MP4 arrives with a real AAC track. Ask for speech in the prompt and the model lip-syncs it.
• Qwen3-TTS on llama.cpp - text-to-speech on the full accelerator matrix already shipped for text generation (CUDA, ROCm, SYCL, Vulkan, Metal, L4T), using upstream's own GGUF conversion.
• KNN as a first-class router - similarity-weighted voting over a curated, persisted corpus of labelled prompts. No classifier model, and a prompt unlike anything labelled is treated as undecidable rather than guessed.
• Global admission control and live backend traces - process-wide HTTP admission bounds, in-flight backend operations are represented while they run, and the UI links straight to their logs.
• Reversible PII pseudonyms - masked values become request-scoped deterministic pseudonyms (EMAIL_001) and are restored if the backend echoes them, across JSON and SSE tokens split over writes.
• Parallel Hugging Face downloads - snapshot materialization runs up to N whole-file transfers at once, so a repository split into many shards stops spending its wall clock in per-file latency.
• Cold model loads are durable jobs - the per-model advisory lock no longer spans a multi-GB transfer, which had made a 35.7 GB load look permanently broken from the operator's seat while staging progressed normally underneath.
• vllm-cpp covers the cards you own - CUDA builds went from one or two architectures to eight on amd64 and five on arm64, picking up A100, L4, 4090, H100/H200, B200, Jetson Orin and Jetson Thor.

Plus a single shared WebRTC UDP port for Realtime, Metal actually enabled in the macOS Stable Diffusion and Parakeet builds, backend crash diagnostics at the default log level, and Portuguese (Brazil) and Indonesian UI translations.

Models and backends each get one canonical page, with Explore and Installed as views rather than separate destinations.

---

This release in numbers

| | | |---|---| | Pull requests merged | 146 | | Commits | 148 | | Files changed | 420 (+27,788 / -4,670) | | Development window | 13 days (2026-08-07 to 2026-08-20) | | Human contributors | 10, of whom 3 first-time | | Gallery entries | 1,622 to 1,707 (+85) |

Where the work landed:

| Area | Change | |---|---| | core/ | +17,193 / -4,187 across 279 files | | gallery/ | +3,808 / -62 | | backend/ | +3,298 / -177 across 60 files | | pkg/ | +1,180 / -113 across 31 files | | swagger/ | +1,095 / -5 | | docs/ | +952 / -98 across 23 files |

---

TL;DR

| Area | Summary | |------|---------| | Auth by default | Every method and path requires credentials unless listed in an explicit public registry. API instructions, Swagger GETs, the LocalAI well-known document, and the health, login, OAuth, SPA, asset, branding and node registration-token bootstrap flows stay public. Migration: with database auth or legacy API keys configured, /version and generated audio, image, video and 3D URLs now require credentials. Embedded deployments can add narrow prefixes through ApplicationConfig.PathWithoutAuth, and the legacy GET exemption flags remain as explicit compatibility overrides. | | Context compression | Opt-in per-model compression config. Older complete turns are compressed through a configured LocalAI model before inference, after PII filtering and Assistant/MCP prompt injection (including later MCP iterations). Leading system/developer prompts, the newest messages and complete tool-call/result units are preserved; tool schemas and completion headroom are accounted for with a conservative offline token bound. Metadata rides non-streaming responses and streaming usage trailers, with event, ratio and duration metrics exported. Disabled by default; cloud-proxy passthrough is rejected (translate mode works). | | Unified lifecycle UI | /app/models owns Explore and Installed with URL-backed search, state and selection; /app/backends owns Catalog and Installed while keeping variants, development builds and target-node scope. Explore offers capability-aware Open and Manage installation; destructive model controls stay in Installed. Operate Overview shows host capacity from its shared summary poller, and the nested Host destination is removed. /app/manage redirects while preserving legacy query state. No API change. | | Import form rebuild | The import page moves to page--medium with a work column and the format reference beside it rather than behind a closed chevron. The source field is the hero and carries its own Import button, which removes the aria-hidden submit that existed only because the real action sat outside the . Simple and Advanced modes are gone (about 80% the same surface); the real distinction, a source or YAML, is now two tabs. Also fixes two class bugs: a primary button with no className at all falling through to browser chrome, and class="btn btn-primary fas fa-save fa-upload" setting Font Awesome as the button's own font while two icons fought over one ::before. | | MiniMax-H3 video | vllm-cpp over vllm.cpp ABI v12. A second engine handle loads the H3 checkpoint set (the DiT is parameters.model, the text encoder and two VAEs are named in options:), and GenerateVideo renders video and audio jointly into an MP4 with a real AAC track. The DiT partition is declared, not detected: community quantizations strip the release metadata and the FL2VA and Ref2VA DiTs are byte-structurally identical, so checkPartitionConditioning refuses a reference-conditioned FL2VA request before the engine runs (it would otherwise render for hours and return a coloured lattice). ffmpeg comes from the host: libvllm composes the mux argv and spawns nothing. New gallery entry minimax-h3-fl2va-q4. | | Qwen3-TTS on llama-cpp | TTS through the llama-cpp backend on CUDA, ROCm, SYCL, Vulkan, Metal and L4T, using upstream's GGUF conversion. Implemented as a slot-based SERVER_TASK_TYPE_TTS task, which is the concurrency-safe integration given that server_context owns the llama_context and runs the slot scheduler on its own thread. Carries the still-draft upstream server hunks as patches/0002-add-server-task-type-tts.patch (delete on merge of ggml-org/llama.cpp#26603). Gallery: qwen3-tts-llamacpp and qwen3-tts-llamacpp-q4. The existing qwen3-tts-cpp backend is untouched and remains a separate path. | | KNN routing | classifier: knn routes by similarity-weighted voting over labelled example prompts, so no classifier model is needed and label knowledge lives in a corpus you seed and curate. Entries below knn.similarity_threshold cannot vote; when none clears it the router takes the fallback, and nearest_similarity is recorded on decisions and fallbacks alike. One JSONL file per router under /router-corpus is the source of truth, with the in-memory index rebuilt at classifier build time and entries re-embedded when the embedding model changed. Corpus input is API-only by design: POST /api/router/{name}/corpus, GET .../corpus/stats (label counts only, texts are never returned), DELETE .../corpus, admin-gated and exposed as MCP tools. | | Admission and traces | Process-wide HTTP admission control, bounding what was previously only per backend. Backend operations are represented while in flight, and running backend traces surface in the UI with immediate log links. | | PII pseudonyms | Opt-in pii.reverse_in_response. Masked request values become unique deterministic pseudonyms within the request (EMAIL_001, EMAIL_002) and are restored if the backend returns them, including SSE tokens split across response writes. Substitution maps are request-local and never persisted. Irreversible [REDACTED:...] remains the default. | | Parallel HF downloads | DownloadFilesWithConcurrency runs up to N whole-file transfers through an errgroup with SetLimit. Single files are never split, so .partial resume and per-file SHA verification are untouched, and the two non-artifact callers keep sequential ordering and fail-fast behaviour through a limit-of-1 wrapper. completedBytes became an atomic.Int64 (the race detector reported three races otherwise) and the caller's status callback stays serialized. | | Realtime WebRTC port | --web-rtc-udp-port / LOCALAI_WEBRTC_UDP_PORT reuses one Pion ICE UDP mux across Realtime calls, with bind failures surfaced through signaling and container/firewall setup documented. The follow-up fix keeps LOCALAI_WEBRTC_ICE_INTERFACES effective when a fixed port is set, which had been silently ignored: a wildcard mux made pion enumerate every interface itself, handing browsers unroutable 172.x candidates that dropped once ICE consent checks failed. | | Durable cold loads | The per-model advisory lock is a dedup decision measured in milliseconds, not a transfer's lifetime. Cold loads now run as durable jobs instead of holding it across backend install, multi-GB staging and checkpoint load, and WithLockCtx now defends against statement_timeout as well as lock_timeout (both abort the same blocking pg_advisory_lock, only the latter was overridden). | | vllm-cpp CUDA coverage | amd64 goes from 120a;121a to 80;86;89;90a;100a;103a;120a;121a, arm64 from 121a to 87;90a;100a;110;121a, split by where the silicon exists. An unlisted card did not run slower, it died at the first request with no kernel image is available for execution on the device, long after install reported success. The CUDA 13 guard now covers both branches, and Triton-AOT stays on. | | Two new languages | Portuguese (Brazil), a complete 14-namespace translation at full key parity with en/, and Indonesian for the admin, media and navigation surfaces. | | Models | 85 new gallery entries: Qwen3.8 (9B, 27B, Ridge and small variants), Gemma 4 agentic and Scotoma 2, DeepSeek V4 Pro 0813, Ling 3.0 Flash, Nemotron 3.5 Lightning 30B, Tess 4 27B, Ornith 1.0 and 1.5 9B, LFM2.5 230M and VL 1.6B, HunyuanOCR and OvisOCR2, Higgs Audio v3 TTS, MiniMax-H3 Ref2VA, plus vllm.cpp text-generation entries and a first Carbon genomics family. |

---

New Features & Major Enhancements

Authentication now denies by default

The previous classifier gated selected API-style paths by prefix. Anything whose path was not on that list was public, which meant unprefixed aliases (/mcp/chat/completions, /moderations, /models, /backends, /import-model) could bypass global authentication, and any newly registered route inherited the same weakness by default.

The middleware is now method-aware and denies by default: a route is public only if its method and path appear in an explicit public registry. What stays public is the set required to bootstrap and to be discoverable: API instructions, Swagger GET routes, the LocalAI well-known document, and the health, login, OAuth, SPA, asset, branding and node registration-token flows. Whole-router coverage is asserted in tests, so a new route cannot become public by omission.

Migration impact. When database authentication or legacy API keys are configured, /version and generated audio, image, video and 3D URLs now require credentials. Embedded deployments can still add narrow prefixes via ApplicationConfig.PathWithoutAuth, and the legacy GET exemption flags remain available as explicit compatibility overrides.

Thanks to Naor Yaacov for reporting this class of authentication bypass.

PRs: #11602

End-to-end context compression

A long conversation eventually stops fitting. Compression is opt-in per model, and when enabled it compresses older complete turns through a configured LocalAI model before inference rather than truncating them away.

What it will not touch: leading system and developer safety prompts, the newest messages, and complete tool-call/result units, which are kept whole so a compressed history never leaves a call without its result. It runs after PII filtering and after Assistant/MCP prompt injection, including on later MCP iterations, so what gets compressed is the prompt that would actually have been sent. Tool schemas and completion headroom are accounted for with a conservative offline token bound.

Compression metadata is exposed in non-streaming responses and in streaming usage trailers, and compression events, ratios and durations are exported as metrics. Cloud-proxy passthrough configurations reject compression because LocalAI cannot safely rewrite an opaque provider payload; translate mode is supported. A late failure in an already-started stream is returned as an in-band SSE error followed by [DONE].