Human in the loop tool approval. Where an administrator has turned it on, you can switch a conversation from letting tools run freely to being asked first, so a model that wants to use a tool stops and waits for you to allow or deny it, one call at a time in a saved conversation, by button or by keyboard shortcut, with your choice remembered for this conversation and for future ones, switching back to running freely releasing anything already waiting, and automations, channel replies, and temporary chats unaffected. Commit, Commit, Commit, Commit, Commit, Commit, Commit, Commit, Commit, Commit
Models that can ask you a question. A new built-in tool lets a model pause and put up to three multiple-choice questions to you before continuing, with room to type your own answer instead, and the question survives a reload in a saved conversation, so you can come back and answer it later rather than losing the conversation. Commit, Commit, Commit, Commit, Commit, Commit, Commit, Commit
Agents can now display terminal files directly. A model can now show a file it made in a terminal directly in its reply, with a preview and a download button, instead of describing a path that led nowhere when clicked, and a new interface setting chooses whether these open in the reply or in the files pane. Commit, Commit, Commit, #27650
Streaming rebuilt from the ground up. A reply now streams as small pieces of new text instead of resending the whole message so far with every update, so the data sent over a reply grows with its length rather than with its length squared, which on a server with many people chatting at once means far less processor time spent encoding, passing, and decoding those updates, far less load and memory on the shared cache that carries them between instances, and far less work in your browser, which no longer takes in the whole reply again and redraws the parts of it that have not changed on every update, cutting the data sent and the server work spent appending to a message by up to 1000x on a very long reply, and a reply still in progress is now kept aside on the server, so reopening the conversation after a refresh picks it up where it is rather than showing a blank message, on deployments backed by Redis. Commit, Commit, Commit, Commit
Much faster throughout. Hundreds of places across the application no longer assemble detailed log text that is switched off and thrown away unread, so sending messages, uploading and indexing files, running searches, signing in, and loading admin pages all get that time back, with the largest gains on busy servers, in long conversations, and on chats that draw from a large knowledge base. #27834, #27837
Faster model list lookups. Title generation, tag suggestions, autocomplete, and other background steps of a chat turn now fetch the model list in one go, which keeps other people's responses flowing on busy Redis-backed instances with many models. #27821
Cheaper log export. Deployments that export their logs to a telemetry collector, with "ENABLE_OTEL" and "ENABLE_OTEL_LOGS" both set, now prepare each exported line once instead of twice, which matters more than it used to now that log text is only assembled when something will actually read it. #27840
Faster permission checks on large instances. Working out which groups you belong to is now a direct lookup rather than a scan of every membership on the server, so chats and the admin user list stay quick as an organization grows. #27822
Much faster JSON handling. Saving and opening chats, reading settings, returning results from built-in tools, streaming replies, signing in and signing up, working out your permissions, and reading stored chunk details during knowledge base searches on Valkey and Oracle vector storage are all handled much faster across the application when the "ENABLE_ORJSON" option is turned on. Commit, #28396, #27841, #27807, #27805, #27813
Much faster outbound requests. Conversations and embedding batches sent to Ollama and Anthropic models are packaged for delivery much faster, which is most noticeable in long chats when the "ENABLE_ORJSON" option is turned on. #27811, #27810
Much faster code interpreter output. Printed output and generated images from code run in chat appear much faster when the "ENABLE_ORJSON" option is turned on. #27812
Lighter page loads. Several small requests the interface makes on every page load, along with a few administrative ones, no longer set up database access they never used, which took several times longer than the rest of the request put together. #28178
One less read per message. Sending a message no longer loads the whole conversation from the database twice over, which mattered most in long chats where that record is largest. #28809
Faster skills on large instances. Opening the skills list, or sending a message that uses one, no longer checks every skill on the instance one at a time, so both are far quicker where many skills exist and most of them are not yours. #28798
Faster tools on large instances. Listing or exporting tools no longer checks every tool on the instance one at a time, so the integrations menu and the tools workspace open faster where many exist. Commit
Faster file access checks. Checking whether you may reach a file no longer walks every workspace model you can see looking for it, so opening a folder of files, downloading one, or retrieving from one is much quicker on instances with many models. #28802
Faster folder listings. Listing your folders now works out your group memberships once for the whole listing rather than again for every item in every folder. #28810
Less work per update in a long chat. Each update saved while a reply streams no longer re-examines the entire conversation, only the part being added, so the cost of an update stops growing with the length of the chat. #28820
Cheaper attaching of sources and files to a reply. Adding a source, file, or embedded item to a reply now reads just that one field rather than rebuilding the whole conversation to find it, which on a two hundred message chat is around 3.1 ms per item down to 0.65 ms, and no longer grows with the length of the conversation. Commit, Commit
Faster workspace model lookups. Working out which workspace models you may edit no longer loads every model on the instance and discards most of them, which also speeds up exporting models and the file access checks that relied on it. #28795
Faster handing off a streaming reply. Passing a reply in progress between instances now writes it once rather than converting it back and forth and scanning it for characters that only matter elsewhere, which on a large non-English conversation took most of the time spent on each write. #28833
Constant load on an idle instance. An instance sitting idle no longer works through every chat you have once a second looking for timers that are due, which on a large history kept about a quarter of a processor core busy doing nothing and could exhaust memory until the application was killed. #27663, #27622, #27745
Sidebar folders fetched once. Refreshing the sidebar now asks for your folders once rather than three times, on page load and on every action that refreshes it. #28662, #28661
Far fewer writes just from being signed in. Recording that someone is online now writes at most once a minute for each person rather than on every single request, where an open tab alone caused two write transactions a minute before anyone touched anything. #28177, #28165
Less overhead on every request. The layers each request passes through before it is handled are now one instead of five, which also removes a quarter of that cost from every piece of a streamed reply on instances that set security headers. Commit, #28171
Turning off compression of live updates. A new "UVICORN_WS_PER_MESSAGE_DEFLATE" setting stops the server compressing every live update it sends, which costs processor time on each one for almost no saving now that a reply streams as small pieces; compression stays on unless it is turned off. #28613
Faster chat list and unread counts. Opening the sidebar, and the unread markers on folders, no longer read through your whole chat history to produce a short list, which on an instance with 15000 chats took 2 to 4 seconds. #27663, #27622, #27745
Long replies no longer slow as they grow. A long reply is no longer re-examined from the beginning for reasoning and code blocks on every piece that arrives, so the work stops growing with the length of the reply, which on a long reply is around 190x less time spent on it. #28861
Faster saving of long chats. A chat is now written to the database in one go rather than one message at a time, so saving a long conversation is much quicker and puts far less strain on the database, and saving one where nothing has changed writes nothing at all. #28806
Faster loading of shared folders. Folders shared with you now load in a couple of queries rather than one for each folder and each owner, so the list appears sooner for anyone with many of them. #28804
