Hugging Face Transformers 5.17.0

Release v5.17.0

New Model additions

HYV4

Hy4-Preview is a 780B-parameter mixture-of-experts language model that activates 49B parameters per token. Each MoE layer holds 256 routed experts plus one always-active shared expert and routes every token to 8 of them. The context window is 1M tokens.

The architecture combines four features:

• Multi-head Latent Attention (MLA) compresses keys and values into a low-rank latent

(kv_lora_rank) that kv_b_proj expands back to one key/value per query head.

• DeepSeek Sparse Attention (DSA) selects index_topk keys per query with a lightweight indexer.

Following IndexShare, only the layers marked "full" in indexer_types run an indexer; "shared" layers reuse the previous full layer's selection.

• Gated MLA with learnable attention sinks, where each head owns a sink logit that participates

in the softmax and contributes no value, as in GPT-OSS.

• Independent Hyper-Connections (iHC) replace the plain residual path with hc_mult parallel

residual streams that are collapsed before, and redistributed after, every sublayer.

The implementation does not execute the multi-token prediction (MTP) layers. Released checkpoints keep those weights so that other runtimes can use them for speculative decoding; they are ignored at load time.

Links: Documentation

• Add h4 (#48473) by @ArthurZucker in #48473

VibeVoice

VibeVoice is a novel framework for synthesizing high-fidelity, long-form speech with multiple speakers by employing a next-token diffusion approach within a Large Language Model (LLM) structure. It's designed to capture the authentic conversational "vibe" and is particularly suited for generating audio content like podcasts and multi-participant audiobooks.

Links: Documentation

• Implement VibeVoice (#40546) by @pengzhiliang in #40546

NeoMME

NeoMME is a family of efficient 260M and 800M parameter multimodal-native multilingual foundation encoders from H Company. It processes multilingual text tokens and raw image patches in a single bidirectional Transformer encoder, without a separately pretrained vision tower or causal language model.

NeoMME-Retriever is a model fine-tuned from the NeoMME backbone for visual document retrieval with joint late-interaction and dense objectives. It takes text queries and documents (text or page screenshots) and produces multi-vector embeddings for MeanMaxSim scoring (late-interaction) and mean-pooled embeddings for cosine similarity (dense).

Links: Documentation

• Add NeoMME and NeoMME-Retriever (#47992) by @tonywu71 in #47992

Fun-ASR-Nano

Fun-ASR-Nano is an 800M-parameter end-to-end speech recognition model developed by Alibaba DAMO Academy's FunAudioLLM team. It achieves state-of-the-art performance on Chinese, English, and Japanese ASR benchmarks while being significantly smaller than comparable models.

Key features are

• Chinese, English, and Japanese, including 7 Chinese dialects and 26 regional accents
• Hotword customization for domain-specific vocabulary
• Native punctuation output (no separate punctuation model needed)

Links: Documentation

• Add Fun-ASR-Nano model (#46180) by @LauraGPT in #46180

KimiLinear

Kimi Linear is a hybrid linear attention architecture from Moonshot AI, introduced in Kimi Linear: An Expressive, Efficient Attention Architecture.

At its core is Kimi Delta Attention (KDA), a refinement of Gated DeltaNet that gives each key channel its own forget gate, so the recurrent state decays per channel instead of per head. KDA is used in most layers; every fourth layer keeps a full-attention block that reuses DeepSeek-V3's Multi-head Latent Attention (MLA), and the feed-forward blocks are DeepSeek-V3-style MoE with a shared expert.

Links: Documentation

• Kimi linear (#48250) by @remi-or in #48250

Canary

Canary-1B-v2, a fast, robust multilingual model for Automatic Speech Recognition (ASR) and Speech-to-Text Translation (AST):

Canary reuses the Fast Conformer encoder from Parakeet (loaded through [ParakeetEncoder] / [ParakeetEncoderConfig]) and pairs it with a Transformer decoder that uses fixed sinusoidal positional embeddings, cross-attention to the encoder outputs and tied input/output embeddings. The task is selected through a decoder prompt prefix built by [CanaryProcessor] of the form <|startofcontext|> <|startoftranscript|> <|emo:undefined|> <source_lang> <target_lang> <pnc|nopnc> <|noitn|> <|notimestamp|> <|nodiarize|>, where source_lang == target_lang selects transcription and otherwise selects translation.

Links: Documentation

• model: Add NVIDIA Canary-1B-v2 to Transformers (#46825) by @harshaljanjani in #46825

NeuCodec

The NeuCodec model was proposed in Finite Scalar Quantization Enables Redundant and Transmission-Robust Neural Audio Compression at Low Bit-rates.

NeuCodec is a neural audio codec extending on XCodec2. It takes advantage of the following features:

• Finite Scalar Quantization (FSQ) quantisation resulting in a single codebook, making it ideal for downstream modeling with Speech Language Models.
• Trained with CC data such that there are no Non-Commercial data restrictions.
• At 50 tokens/sec and 16 bits per token, the overall bit-rate is 0.8kbps.
• The codec takes in 16kHz input and outputs 24kHz using an upsampling decoder.
• The FSQ encoding scheme allows for bit-level error resistance suitable for unreliable and noisy channels.

Links: Documentation

• Add support for NeuCodec (#47143) by @harryjulian in #47143

Breaking changes

Vision rotary embeddings (2D/3D) have been standardized into a unified RoPE frequency computation module, so users with custom vision models relying on attention-layer-level or model-specific RoPE grid interleaving logic must migrate to the new centralized modeling_rope_utils.py implementation.

• :rotating_light: Vision (2d/3d) rotary embeddings (#48105) by @zucchini-nlp

Generation

Generation improvements include a performance optimization that avoids unnecessary accelerator synchronization on every decode step (reducing per-step overhead), and a fix to prevent unconditional downloading of remote hub files during generation. Several correctness fixes were also applied, including enforcing auto-compile cache checks for encoder-decoder models, standardizing past_key_values naming in AfMoE, and resolving flaky export and integration test failures.

• [Generate] Avoid unconditionally downloading remote hub file (#48620) by @vasqu in [#48620]
• [generate] stop synchronizing the accelerator on every decode step (#47975) by @SunMarc in [#47975]
• [AfMoE] Standardize past_key_values argument naming across forward and generate (#48430) by @shenhuaqingshi in [#48430]
• Fix MTP generation test regex gate for escaped layer ignore keys (#48003) (#48262) by @Noxtimo in [#48262]
• fix(generation): Enforce the auto-compile cache check for encoder-decoder models (#48364) by @harshaljanjani in [#48364]
• [serge] Fix 4 integration tests for model generation failing with output_mismatch (list output differs (4)) (#48133) by @sergereview[bot] in [#48133]
• [VibeVoice] Skip generate export tests (flaky) (#48396) by @ydshieh in [#48396]

Cache

Fixed several cache-related bugs, including a quantized cache issue in VibeVoice, incorrect rejection of non-static cache implementations in VoxtralRealtime, missing auto-compile cache checks for encoder-decoder models, and a silent failure when paged attention is called without a cache. Documentation was also updated to clarify ContinuousBatchingConfig usage and sliding window model limitations.

• vibevoice: fix bug for quant cache (#48487) by @kaixuanliu in [#48487]
• Fix VoxtralRealtime rejecting non-static cache implementations (#48082) by @jiqing-feng in [#48082]
• Raise when a paged attention forward is called with no cache (#48297) by @qgallouedec in [#48297]
• [docs] Pass ContinuousBatchingConfig and sliding window models (#48381) by @stevhliu in [#48381]
• Retry get_daily_ci_runs on stale GitHub API cache (#48374) by @ydshieh in [#48374]

Kernels

Kernel support was improved with fixes for nested FLA kernel imports when only fla-core is installed, a warning when hub-kernel functions silently fall back to slower pure-PyTorch reference implementations, and the ability to register standalone functions (e.g., RoPE) in KernelConfig with optional non-inheritance of default mappings. Additional fixes include corrected repository paths for ESMFold2 kernels and updated documentation for KernelConfig customization.

• Support nested FLA kernel imports for fla-core (#48221) by @DimensionSTP in [#48221]
• Warn once when a hub-kernel function falls back to its reference PyTorch path (#48185) by @qgallouedec in [#48185]
• [docs] Kernel updates (#48465) by @stevhliu in [#48465]
• [Kernels] Enable functions into kernels registry and allow non inheritance (#48443) by @vasqu in [#48443]
• Fix kernel commit and repo paths for ESMFold2 (#48186) by @Rocketknight1 in [#48186]

Quantization

Fixed several quantization bugs, including a quant cache issue in VibeVoice, incorrect FP8 embedding handling for Qwen models, missing FP8 tensor parallelism layer overrides, and unnecessary MXFP4 weight dequantization on XPU devices.

• fix qwen4exp-fp8 ple embedding (#48368) by @JJJYmmm in [#48368]
• Keep MXFP4 weights quantized on XPU when use_kernels is set (#47923) by @jiqing-feng in [#47923]
• Fix missing FP8 TP layer overrides (#48343) by @changwangss in [#48343]

Bugfixes and improvements

• MRoPE continued (#48594) by @zucchini-nlp in [#48594]
• [fix] Update stale expected strings in HunYuanVL integration tests (#48646) by @ydshieh in [#48646]
• [Quantizaiton]support 5/6/7 bits in AutoRound (#48481) by @wenhuach21 in [#48481]
• [fix] Update stale golden values and fix expected_logits shape in FlavaForPreTraining integration tests (#48639) by @ydshieh in [#48639]
• Fix YOLOS device mismatch with device_map="auto" (#46886) by @swankystark in [#46886]
• Honor shift_labels in decoder-only LLM/VLM losses (#48493) by @qgallouedec in [#48493]
• [KimiLinear] Fix test_cpu_offload: set num_local_experts=4 in model tester (#48624) by @ydshieh in [#48624]
• [docs] Per-layer config (#48601) by @stevhliu in [#48601]
• Fix generate_flags parsing in transformers chat (#48597) by @SunMarc in [#48597]
• [fix] Fix how we read package versions - triggered by torch 2.14+ (#48615) by @ydshieh in [#48615]
• [Docker] Upgrade CPU torch to <=2.14.0, torchcodec to <=0.16.0 (#48614) by @ydshieh in [#48614]
• Fix AttributeError in gradient_checkpointing_enable(offload=True) (#48590) by @tarekziade in [#48590]
• Another day fixing CI (#48591) by @zucchini-nlp in [#48591]
• esmfold2: keep distogram_head in fp32 as well (#48488) by @kaixuanliu in [#48488]
• Add supports_context_parallel to PreTrainedModel (#48442) by @qgallouedec in [#48442]
• [docs] mlinter reference (#48460) by @stevhliu in [#48460]
• [docs] Add a LiteRT page under community integrations (#48540) by @john-rocky in [#48540]
• [GLM 5.3 Flash] Fix NaN gradients in chunked KDA (#48455) by @imvladikon in [#48455]
• [docs] Fix [[autodoc]] directives in ALBERT model documentation (#48593) by @samyuktahegde in [#48593]
• [Fix] Fix A10 expectations for a test (#48454) by @remi-or in [#48454]
• Add PR comment CI for AMD (MI300) (#48065) by @ydshieh in [#48065]
• docs: fix docstring parameter names that do not match signatures (#48575) by @simpleqt in [#48575]
• docs: remove phantom parameters from docstrings (#48576) by @simpleqt in [#48576]
• extend some case to xpu as well (#48502) by @sywangyi in [#48502]
• [serge] Fix 2 integration tests for model glm4_moe failing with OOM (other (2)) (#48551) by @sergereview[bot] in [#48551]
• [serge] Fix 2 integration tests for model nemotron failing with import_or_config (other (2)) (#48582) by @sergereview[bot] in [#48582]
• Compress the agent conventions file and document two modular pitfalls (#48586) by @tarekziade in [#48586]
• Guard against a None video processor class when the backend is unavailable (#48557) by @caiotheodoro in [#48557]
• [serge] Fix 2 integration tests regressed by commit 83d46aa2a2c4 (PR #47625) (#48580) by @sergereview[bot] in [#48580]
• [nit] use requires_backends (#47576) by @eustlb in [#47576]
• [serge] Fix 2 integration tests for model kosmos2 failing with import_or_config (other (2)) (#48552) by @sergereview[bot] in [#48552]
• Fix failing tests for cohere_compass (#48005) by @kaixuanliu in [#48005]
• [Fix] Use dedicated helpers for DeepGEMM and SonicMoE tests (#48523) by @remi-or in [#48523]
• CI: Point test fixtures at hf-internal-testing copies we already host (#48521) by @tarekziade in [#48521]
• fix failed test cases for glm5_next (#48497) by @kaixuanliu in [#48497]