Hugging Face Transformers 5.13.0

Release v5.13.0

New Model additions

KimiK 2.5, 2.6, and 2.7

This release includes the architecture for Kimi 2.5 which is used by 2.5-2.7:

Kimi K2.5 is an open-source, native multimodal agentic model that advances practical capabilities in long-horizon coding, coding-driven design, proactive autonomous execution, and swarm-based task orchestration. The model was proposed in Kimi K2.5: Visual Agentic Intelligence and further improved in Kimi K2.6: Advancing Open-Source Coding.

Kimi K2.5 achieves significant improvements on complex, end-to-end coding tasks, generalizing robustly across programming languages (Rust, Go, Python) and domains spanning front-end, DevOps, and performance optimization. The model is capable of transforming simple prompts and visual inputs into production-ready interfaces and lightweight full-stack workflows, generating structured layouts, interactive elements, and rich animations with deliberate aesthetic precision.

Links: Documentation

• Add new model: Kimi2-6 (#45630) by @zucchini-nlp in #45630

MiMo-V2-Flash

MiMo-V2-Flash is a Mixture-of-Experts (MoE) language model developed by the Xiaomi MiMo team. Designed to establish a new balance between long-context modeling capabilities and inference efficiency, the model is built for strong performance in complex reasoning and agentic tasks. Trained on 27T tokens with native 32k sequence lengths, MiMo-V2-Flash seamlessly supports an extended 256K context window while significantly reducing KV-cache storage compared to standard global attention models.

Links: Documentation

• Add Xiaomi MiMo-V2 (#45144) by @casinca in #45144

Nemotron 3.5 ASR

Nemotron 3.5 ASR is a 600M-parameter multilingual speech recognition model from NVIDIA, built for high-quality transcription in both low-latency streaming and high-throughput batch settings, with native punctuation and capitalization. For streaming, it offers configurable chunk sizes-80ms, 160ms, 560ms, and 1120ms, letting users trade off latency against accuracy to suit their application. Its cache-aware FastConformer-RNNT architecture is central to this capability: unlike traditional buffered streaming, which repeatedly reprocesses overlapping audio windows, the model processes only each new incoming chunk while reusing cached encoder context from prior chunks. This eliminates redundant computation, significantly improves efficiency, and minimizes end-to-end delay without sacrificing accuracy, making it well suited to real-time transcription workloads.

Links: Documentation

• Add Nemotron 3.5 ASR Streaming (#46565) by @eustlb in #46565

NemotronAsrStreaming

Nemotron ASR Streaming is a 600M-parameter English speech recognition model from NVIDIA, built for high-quality transcription in both low-latency streaming and high-throughput batch settings, with native punctuation and capitalization. For streaming, it offers configurable chunk sizes-80ms, 160ms, 560ms, and 1120ms, letting users trade off latency against accuracy to suit their application. Its cache-aware FastConformer-RNNT architecture is central to this capability: unlike traditional buffered streaming, which repeatedly reprocesses overlapping audio windows, the model processes only each new incoming chunk while reusing cached encoder context from prior chunks. This eliminates redundant computation, significantly improves efficiency, and minimizes end-to-end delay without sacrificing accuracy, making it well suited to real-time transcription workloads.

Links: Documentation

• Add Nemotron ASR Streaming (#46332) by @eustlb in #46332

Qwen3 ASR

Qwen3 ASR is an automatic speech recognition model from Alibaba's Qwen team that combines a Whisper-style audio encoder with a Qwen3 language model decoder for speech-to-text transcription. The model supports automatic language detection and multilingual transcription.

A forced aligner model is also included. It can be used to timestamp a provided transcript and its audio. It uses the same audio encoder model with a classification head that predicts a word's length. This model can be used with the transcript from any ASR model (see the example below with Parakeet CTC).

Links: Documentation

• Qwen3 ASR and Forced Aligner (#43838) by @mbtariq82 in #43838

ZAYA

ZAYA1 is a 760M active / 8.4B total parameter MoE language model trained by Zyphra. It combines Compressed Convolutional Attention (CCA), a nonlinear ZAYA1 router, and residual scaling.

Links: Documentation

• [new model] Add Zyphra/ZAYA1-8B (#45862) by @JJJYmmm in #45862

VideoPrism

The VideoPrism model was proposed in the paper VideoPrism: A Foundational Visual Encoder for Video Understanding by Google DeepMind (blog post).

VideoPrism is a general-purpose video encoder that tackles diverse video understanding tasks with a single frozen model. The model is pretrained on a large-scale heterogeneous corpus containing 36M high-quality video-caption pairs and 582M video clips with noisy parallel text (e.g., ASR transcripts). The pretraining approach improves upon masked autoencoding through global-local distillation of semantic video embeddings and a token shuffling scheme, enabling the model to focus primarily on the video modality while leveraging text associated with videos. VideoPrism achieves state-of-the-art performance on 31 out of 33 video understanding benchmarks across four broad task groups, from web video question answering to computer vision for science.

Links: Documentation

• Add Videoprism (#39895) by @MHRDYN7 in #39895

RADIO

RADIO (Reduce All Domains Into One) is a family of vision foundation models from NVIDIA trained by multi-teacher distillation (e.g. CLIP, DINOv2, SAM) into a single ViT backbone. It produces both an image-level summary embedding and dense spatial features, and supports variable input resolutions through a Cropped Position Embedding (CPE) patch generator.

Links: Documentation

• Add support for RADIO models (#46425) by @meatybobby in #46425

MiniCPM3

MiniCPM3 is the third-generation MiniCPM dense language model from OpenBMB. The 4B variant (openbmb/MiniCPM3-4B) outperforms many 7B-9B open models on standard benchmarks while remaining lightweight enough for on-device usage.

MiniCPM3 combines several architectural ideas:

• Multi-head Latent Attention (MLA) from DeepSeek-V2, which compresses the key/value cache

into a low-rank latent representation while still using rotary embeddings on a portion of the query/key heads.

• A standard SwiGLU MLP (no MoE).
• Three scalar scaling factors that govern signal flow:
• scale_emb - scales input embeddings.
• scale_depth / sqrt(num_hidden_layers) - scales residual connections.
• hidden_size / dim_model_base - scales hidden states before the language model head.

Links: Documentation

• Add MiniCPM3 (#41116) by @bzantium in #41116

Breaking changes

A broad set of modeling changes have been made to standardize layer declarations, mask/cache construction, and hybrid-attention handling, making many models cleanly exportable (ONNX, torch.export, ExecuTorch) and fullgraph-compilable - users relying on internal modeling APIs may need to update their code accordingly.

• Modeling changes for export, compile, and hybrid-attention standardization (#46738) by @IlyasMoutawwakil

Attention masking for image tokens in Gemma 3/4 models has been fixed to correctly respect sliding window boundaries in local layers, which changes model behavior and may affect reproducibility of previous results.

• [gemma 3/4] Fix bidirectional attention masking crossing sliding window boundaries (#46850) by @douglas-reid

The Expert Parallelism (EP) router contract has been corrected across many models and FP8 scale format handling has been fixed, requiring users of EP or FP8 quantization with affected models to verify their configurations and potentially update conversion mappings.

• EP: fix EP router contract for many models + honor FP8 scale format (#46818) by @IlyasMoutawwakil

The Kernels integration has been synced to the latest version, which includes a breaking change where model-type repositories are no longer accepted by the kernels interface - users must migrate to the updated kernel repository format as shown in the updated tests.

• :rotating_light: [Kernels] Sync to latest version (#46039) by @vasqu

HfExporters: Native, Unified export for PyTorch / ONNX / ExecuTorch

A native, in-Transformers export pipeline - one base class (HfExporter), three subclasses for the runtimes we care about, one unified API:

| Exporter | Output | Runtime | |---|---|---| | DynamoExporter | ExportedProgram | Any PyTorch runtime, AOT compilation | | OnnxExporter | ONNXProgram | Any ONNX runtime (ORT, TensorRT, OpenVINO, ...) | | ExecutorchExporter | ExecutorchProgramManager | Mobile and edge (ExecuTorch) |

Same call shape across all three. Dynamic shapes by default. Generation-style models split automatically into prefill + decode (+ vision/audio sub-encoders for VLMs).

from transformers import AutoModelForMaskedLM, AutoTokenizer
from transformers.exporters import OnnxExporter, OnnxConfig

model_id = "hf-internal-testing/tiny-random-BertForMaskedLM"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForMaskedLM.from_pretrained(model_id).eval()
inputs = tokenizer(["Hello, my dog is cute"] * 2, return_tensors="pt")
onnx_program = OnnxExporter().export(model, inputs, config=OnnxConfig(dynamic=True))

new_input = tokenizer("Hello, my cat is so adorable!", return_tensors="pt")
torch.testing.assert_close(
    onnx_program.call_reference(**new_input)[0],   # numpy reference
    onnx_program(**new_input)[0],                  # onnxruntime
    rtol=1e-4, atol=1e-4,
)

Swap one line for another runtime - DynamoExporter() / DynamoConfig or ExecutorchExporter() / ExecutorchConfig(backend=...).

For generative models the prefill/decode split is captured automatically:

from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.exporters import OnnxExporter, OnnxConfig

model_id = "hf-internal-testing/tiny-random-LlamaForCausalLM"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id).eval()
inputs = tokenizer(["Hello, my dog is cute"] * 2, return_tensors="pt")

artifacts = OnnxExporter().export_for_generation(model, inputs, config=OnnxConfig(dynamic=True))
## {"prefill": ONNXProgram, "decode": ONNXProgram}
## For VLMs: also vision_encoder, audio_encoder, multi_modal_projector, language_model, lm_head

Kernels

Kernels: Fixed a silent SDPA math-kernel fallback for GQA models with head_dim > 256 (e.g., Gemma4) that caused O(S2) memory materialization, and resolved a regression where use_kernels=True failed to apply kernel mappings. Additional improvements include lazy loading of the default kernel mapping to prevent import failures with incompatible kernel versions, ROCm routing to AITER Triton kernels for AMD GPUs, GB10/SM121 Hub-kernel support for Qwen3.6 Gated DeltaNet, and expanded documentation for the kernel API.

• Fix silent SDPA math-kernel fallback for GQA when key/value head_dim > 256 or differ (#46960) by @Butterfingrz in [#46960]
• [docs] AITER kernels (#46871) by @stevhliu in [#46871]
• Documentation for the kernel API (#46754) by @michaelbenayoun in [#46754]
• update kernels-community/aiter-rope version (#46810) by @Abdennacer-Badaoui in [#46810]
• Add GB10/SM121 Hub-kernel path for Qwen3.6 Gated DeltaNet (#46423) by @AzeezIsh in [#46423]
• [Kernels] Trigger proper kernelization on use_kernels=True (#46755) by @vasqu in [#46755]
• Lazily build the default kernel mapping to decouple kernels from normal transformers usage (#46681) by @jiqing-feng in [#46681]
• Add some AITER kernel routing for ROCm (#46268) by @Abdennacer-Badaoui in [#46268]
• fix: position ids does not exist in upstream rotary kernel (#46619) by @NanoCode012 in [#46619]
• docs(zh): add Chinese translation of kernels.md (#46621) by @shoushinya123 in [#46621]

Generation

Several generation bugs were fixed, including Mamba2 chunked-prefill and speculative decoding for hybrid models (Zamba2, Nemotron-H, Bamba, FalconH1, GraniteMoeHybrid), beam search for Mamba models, prompt lookup decoding crashes with no EOS token, and incorrect stateful model handling for LFM2. Additional improvements include reduced unnecessary generation warnings, a fix for continuous batching output mutation, and a new option to keep input tensors on CPU during generation to avoid retracing on Neuron/TPU devices.