llamafile 0.10.4

What's Changed

The most exciting feature that this release of llamafile brings is the first version of transcribefile! Our transcribefile is nothing more than a cosmocc-build of the CLI tool from transcribe.cpp, @cjpais' ggml-based Speech-to-Text library that supports 16+ model families For more details, see CJ's post and Mozilla.ai's annoucement.

In addition to this, a new sync with upstream llama.cpp (which will be soon replaced by a more recent one, as I let too much time pass before the release ); few improvements to Vulkan and ROCm acceleration; HTTPS support (both for the server and to download models from HF given their IDs); and pledge/SECCOMP sandboxing.

Thanks everyone for improving llamafile with issues and PRs!

• Probe GPU device count out-of-process on Windows (#988 follow-up) by @aittalam in https://github.com/mozilla-ai/llamafile/pull/994
• docs: redirect GitHub Pages docs to GitBook by @angpt in https://github.com/mozilla-ai/llamafile/pull/993
• ROCm shell script improvements by @aittalam in https://github.com/mozilla-ai/llamafile/pull/995
• docs(support): add GPU backend status table & acceleration verification (#996) by @Anai-Guo in https://github.com/mozilla-ai/llamafile/pull/997
• skill: guardrail llama.cpp update workflow with patch commands by @aittalam in https://github.com/mozilla-ai/llamafile/pull/998
• Update llama.cpp to b9747 (d6d8995) by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1000
• chore: upgrade and digest pin all action digests by @RichardoC in https://github.com/mozilla-ai/llamafile/pull/1002
• CPU / GPU fixes + harness for upstream ggml's test-backend-ops by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1004
• vulkan: clearer message when --gpu vulkan is explicitly unavailable by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1006
• Fixes CTRL+C bug in chat mode by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1009
• transcribefile: CPU build of transcribe.cpp + sync to df1a4ad by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1007
• Updated runner to 16-core by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1005
• transcribefile: Metal GPU support via llamafile's runtime loader by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1010
• Add HTTPS support via cpp-httplib's mbedtls backend by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1011
• feat(help): list all accepted llama.cpp options in --help by @RichardoC in https://github.com/mozilla-ai/llamafile/pull/1003
• Bring back pledge/SECCOMP sandboxing (#930) by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1015

New Contributors

• @Anai-Guo made their first contribution in https://github.com/mozilla-ai/llamafile/pull/997
• @RichardoC made their first contribution in https://github.com/mozilla-ai/llamafile/pull/1002

Full Changelog: https://github.com/mozilla-ai/llamafile/compare/0.10.3...0.10.4