Original release notes
CUDA: Improve NVFP4 W4A4 activation quantization (#25730)
- Squash history before conflict-resolution during rebase on master
WIP commit
Add 32-byte loads, restore per-block amax
Use nvfp4x4 intrinsic when available
Fuse per-channel amax and quantization kernels
Do pointer arithmetic only once on x
Remove unnecessary ternary in the load
We assert on host side that ne00 is 64-aligned
Add back scale-search, but optimize it with intrinsics
Code cleanup
Make scale in MMQ-epilogue NVFP4-specific/restrictive for now
Remove unneeded include, add comment
Fix trailing whitespace
Guard __builtin_align__(32) struct to NVIDIA
Seems like HIP doesn't have this available, see https://github.com/ggml-org/llama.cpp/actions/runs/29438651734/job/87431623001
- compiler massaging to avoid unnecessary LDCs
- kvalues_mxfp4 -> kvalues_nvfp4 in quantize_mmq_nvfp4
- Always pass in src1_scale.ptr
- Extract ggml_cuda_is_aligned helper
Website: -
UI:
