We'll now use the anthropic provider to get credentials for `claude` and
embed its configuration view in the panel when they are not present.
Release Notes:
- N/A
Some APIs fail when they get this parameter
Closes#36215
Release Notes:
- Fixed OpenAI-compatible providers that don't support prompt caching
and/or reasoning
Release Notes:
- Added `reasoning_effort` support to custom models
Tested using the following config:
```json5
"language_models": {
"openai": {
"available_models": [
{
"name": "gpt-5-mini",
"display_name": "GPT 5 Mini (custom reasoning)",
"max_output_tokens": 128000,
"max_tokens": 272000,
"reasoning_effort": "high" // Can be minimal, low, medium (default), and high
}
],
"version": "1"
}
}
```
Docs:
https://platform.openai.com/docs/api-reference/chat/create#chat_create-reasoning_effort
This work could be used to split the GPT 5/5-mini/5-nano into each of
it's reasoning effort variant. E.g. `gpt-5`, `gpt-5 low`, `gpt-5
minimal`, `gpt-5 high`, and same for mini/nano.
Release Notes:
* Added a setting to control `reasoning_effort` in OpenAI models
## Summary
Enable image processing capabilities for GPT-5 series models by updating
the `supports_images()` method.
## Changes
- Add vision support for `gpt-5`, `gpt-5-mini`, and `gpt-5-nano` models
- Update `supports_images()` method in
`crates/language_models/src/provider/open_ai.rs`
## Models with Vision Support (after this PR)
- gpt-4o
- gpt-4o-mini
- gpt-4.1
- gpt-4.1-mini
- gpt-4.1-nano
- gpt-5 (new)
- gpt-5-mini (new)
- gpt-5-nano (new)
- o1
- o3
- o4-mini
This brings GPT-5 vision capabilities in line with other OpenAI models
that support image processing.
Release Notes:
- Added vision support for OpenAI models
TODO
- [x] OpenAI Compatible API Icon
- [x] Docs
- [x] Link to docs in OpenAI provider section about configuring OpenAI
API compatible providers
Closes#33992
Related to #30010
Release Notes:
- agent: Add support for adding multiple OpenAI API compatible providers
---------
Co-authored-by: MrSubidubi <dev@bahn.sh>
Co-authored-by: Danilo Leal <daniloleal09@gmail.com>
This introduces a new field `thinking_allowed` on `LanguageModelRequest`
which lets us control whether thinking should be enabled if the model
supports it.
We permit thinking in the Inline Assistant, Edit File tool and the Git
Commit message generator, this should make generation faster when using
a thinking model, e.g. `claude-sonnet-4-thinking`
Release Notes:
- N/A
* Updates to `zed_llm_client-0.8.5` which adds support for `retry_after`
when anthropic provides it.
* Distinguishes upstream provider errors and rate limits from errors
that originate from zed's servers
* Moves `LanguageModelCompletionError::BadInputJson` to
`LanguageModelCompletionEvent::ToolUseJsonParseError`. While arguably
this is an error case, the logic in thread is cleaner with this move.
There is also precedent for inclusion of errors in the event type -
`CompletionRequestStatus::Failed` is how cloud errors arrive.
* Updates `PROVIDER_ID` / `PROVIDER_NAME` constants to use proper types
instead of `&str`, since they can be constructed in a const fashion.
* Removes use of `CLIENT_SUPPORTS_EXA_WEB_SEARCH_PROVIDER_HEADER_NAME`
as the server no longer reads this header and just defaults to that
behavior.
Release notes for this is covered by #33275
Release Notes:
- N/A
---------
Co-authored-by: Richard Feldman <oss@rtfeldman.com>
Co-authored-by: Richard <richard@zed.dev>
This cleans up our settings to not include any `version` fields, as we
have an actual settings migrator now.
This PR removes `language_models > anthropic > version`,
`language_models > openai > version` and `agent > version`.
We had migration paths in the code for a long time, so in practice
almost everyone should be using the latest version of these settings.
Release Notes:
- Remove `version` fields in settings for `agent`, `language_models >
anthropic`, `language_models > openai`. Your settings will automatically
be migrated. If you're running into issues with this open an issue
[here](https://github.com/zed-industries/zed/issues)
Ran into this while adding support for Vercel v0s models:
- The timestamp seems to be returned in Milliseconds instead of seconds
so it breaks the bounds of `created: u32`. We did not use this field
anywhere so just decided to remove it
- Sometimes the `choices` field can be empty when the last chunk comes
in because it only contains `usage`
Release Notes:
- N/A
The `api_url` setting is one that most providers already support and can
be changed via the `settings.json`. We're adding the ability to change
it via the UI for OpenAI specifically so it can be more easily connected
to v0.
Release Notes:
- agent: Added ability to change the API base URL for OpenAI via the UI
---------
Co-authored-by: Bennet Bo Fenner <53836821+bennetbo@users.noreply.github.com>
Previously we were using a mix of `u32` and `usize`, e.g. `max_tokens:
usize, max_output_tokens: Option<u32>` in the same `struct`.
Although [tiktoken](https://github.com/openai/tiktoken) uses `usize`,
token counts should be consistent across targets (e.g. the same model
doesn't suddenly get a smaller context window if you're compiling for
wasm32), and these token counts could end up getting serialized using a
binary protocol, so `usize` is not the right choice for token counts.
I chose to standardize on `u64` over `u32` because we don't store many
of them (so the extra size should be insignificant) and future models
may exceed `u32::MAX` tokens.
Release Notes:
- N/A
Bubbles up rate limit information so that we can retry after a certain
duration if needed higher up in the stack.
Also caps the number of concurrent evals running at once to also help.
Release Notes:
- N/A
This PR adds a new `intent` field to completion requests to assist in
categorizing them correctly.
Release Notes:
- N/A
---------
Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
This expands our deserialization of JSON from models to be more tolerant
of different variations that the model may send, including
capitalization, wrapping things in objects vs. being plain strings, etc.
Also when deserialization fails, it reports the entire error in the JSON
so we can see what failed to deserialize. (Previously these errors were
very unhelpful at diagnosing the problem.)
Finally, also removes the `WrappedText` variant since the custom
deserializer just turns that style of JSON into a normal `Text` variant.
Release Notes:
- N/A
Fixes regression caused by:
https://github.com/zed-industries/zed/pull/30639
Assistant messages can come back with no content, and we no longer
allowed that in the deserialization.
Release Notes:
- open_ai: fixed deserialization issue if assistant content was empty
https://github.com/zed-industries/zed/issues/30972 brought up another
case where our context is not enough to track the actual source of the
issue: we get a general top-level error without inner error.
The reason for this was `.ok_or_else(|| anyhow!("failed to read HEAD
SHA"))?; ` on the top level.
The PR finally reworks the way we use anyhow to reduce such issues (or
at least make it simpler to bubble them up later in a fix).
On top of that, uses a few more anyhow methods for better readability.
* `.ok_or_else(|| anyhow!("..."))`, `map_err` and other similar error
conversion/option reporting cases are replaced with `context` and
`with_context` calls
* in addition to that, various `anyhow!("failed to do ...")` are
stripped with `.context("Doing ...")` messages instead to remove the
parasitic `failed to` text
* `anyhow::ensure!` is used instead of `if ... { return Err(...); }`
calls
* `anyhow::bail!` is used instead of `return Err(anyhow!(...));`
Release Notes:
- N/A
Some providers sometimes send `{ "type": "text", "text": ... }` instead
of just the text as a string. Now we accept those instead of erroring.
Release Notes:
- N/A
I was able to get this fix in upstream, so now we can have simpler code
paths for our model selection.
I also added a test to catch if this would cause a bug again in the
future.
Release Notes:
- N/A
This is very basic support for them. There are a number of other TODOs
before this is really a first-class supported feature, so not adding any
release notes for it; for now, this PR just makes it so that if
read_file tries to read a PNG (which has come up in practice), it at
least correctly sends it to Anthropic instead of messing up.
This also lays the groundwork for future PRs for more first-class
support for images in tool calls across more image file formats and LLM
providers.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <hi@aguz.me>
Co-authored-by: Agus Zubiaga <agus@zed.dev>
tiktoken_rs is a bit behind (and even upstream tiktoken doesn't have all
of these models)
We were incorrectly using the cl100k tokenizer for some models that
actually use the o200k tokenizers. So that is updated.
I also made the match arms specific so that we do a better job of
catching whether or not tiktoken-rs accurately supports new models we
add in.
I will also do a PR upstream to see if we can move some of this logic
back out if tiktoken better supports the newer models.
Release Notes:
- Improved tokenizer support for openai models.
This sets us up to display queue position information to the user, once
our language model backend is updated to support request queuing.
The JSON returned by the LLM backend will need to look like this:
```json
{"queue": {"status": "queued", "position": 1}}
{"queue": {"status": "started"}}
{"event": {"THE_UPSTREAM_MODEL_PROVIDER_EVENT": "..."}}
```
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <git@maxdeviant.com>
This is to enable alternative streaming solutions at the application
layer. I'm not sure we really should have performed parsing of the input
at this layer. Either way I want to experiment with streaming approaches
in a separate crate on a branch, and this will help.
/cc @maxdeviant @bennetbo @rtfeldman
Closes #ISSUE
Release Notes:
- N/A
* Adds a fast / cheaper model to providers and defaults thread
summarization to this model. Initial motivation for this was that
https://github.com/zed-industries/zed/pull/29099 would cause these
requests to fail when used with a thinking model. It doesn't seem
correct to use a thinking model for summarization.
* Skips system prompt, context, and thinking segments.
* If tool use is happening, allows 2 tool uses + one more agent response
before summarizing.
Downside of this is that there was potential for some prefix cache reuse
before, especially for title summarization (thread summarization omitted
tool results and so would not share a prefix for those). This seems fine
as these requests should typically be fairly small. Even for full thread
summarization, skipping all tool use / context should greatly reduce the
token use.
Release Notes:
- N/A
Looks like the required backend component of this was deployed.
https://github.com/zed-industries/monorepo/actions/runs/14541199197
Release Notes:
- N/A
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Agus Zubiaga <hi@aguz.me>
Co-authored-by: Richard Feldman <oss@rtfeldman.com>
Co-authored-by: Nathan Sobo <nathan@zed.dev>
This PR disables `parallel_tool_calls` for the models that support it,
as the Agent currently expects at most one tool use per turn.
It was a bit of trial and error to figure this out. OpenAI's API
annoyingly will return an error if passing `parallel_tool_calls` to a
model that doesn't support it.
Release Notes:
- N/A
This PR removes the `use_any_tool` method from the `LanguageModel`
trait.
It was not being used anywhere, and doesn't really fit in our new tool
use story.
Release Notes:
- N/A
This is the core change:
https://github.com/zed-industries/zed/pull/26758/files#diff-044302c0d57147af17e68a0009fee3e8dcdfb4f32c27a915e70cfa80e987f765R1052
TODO:
- [x] Use AsyncFn instead of Fn() -> Future in GPUI spawn methods
- [x] Implement it in the whole app
- [x] Implement it in the debugger
- [x] Glance at the RPC crate, and see if those box future methods can
be switched over. Answer: It can't directly, as you can't make an
AsyncFn* into a trait object. There's ways around that, but they're all
more complex than just keeping the code as is.
- [ ] Fix platform specific code
Release Notes:
- N/A
I've been bothered by using simple hyphens for bullet lists here for a
while; it kinda looked cheap and not well-formatted. So, in this PR, I'm
adding a new, custom UI component in the `language_models` crate, called
`InstructionListItem`, based off the `ListItem` that's somewhat
mimic'ing what a `<li>` would be on the web.
It does have a "rigid" structure as in it's always a label followed by a
button (which is optional), but that seems okay given it has been the
overall shape of the copy we've been using here. Also, never really
loved that we were pasting URLs directly, that kinda felt cheap, too. I
could see an argument where it's just clearer, but it looks too
cluttered, as URLs aren't super pretty, necessarily.
| Before | After |
|--------|--------|
| <img
src="https://github.com/user-attachments/assets/ffd1ac27-b1f4-450d-abf5-079285fc9877"
width="700px" /> | <img
src="https://github.com/user-attachments/assets/28fb9d0d-205d-45d8-9e43-1aaa947adc96"
width="700px" /> |
Release Notes:
- N/A
