This PR is in preparation for doing automatic retries for certain
errors, e.g. Overloaded. It doesn't change behavior yet (aside from some
granularity of error messages shown to the user), but rather mostly
changes some error handling to be exhaustive enum matches instead of
`anyhow` downcasts, and leaves some comments for where the behavior
change will be in a future PR.
Release Notes:
- N/A
Did some bit cleanup of code for loading models for settings as that is
not required as we are fetching all the models from openrouter so it's
better to maintain one source of truth
Release Notes:
- Add thinking support to OpenRouter provider
Having `Thread::last_usage` as an override of the initially fetched
usage could cause the initial usage to be displayed when the current
thread is empty or in text threads. Fix is to just store last usage info
in `UserStore` and not have these overrides
Release Notes:
- Agent: Fixed request usage display to always include the most recently
known usage - there were some cases where it would show the initially
requested usage.
Previously, the OpenRouter models list (~412kb) was being downloaded
around 10 times during startup -- even when OpenRouter was not
configured.
This update addresses the issue by:
1. Fetching the models list only when OpenRouter settings change.
2. Skipping API calls if OpenRouter is not configured.
Release Notes:
- Avoid unnecessary requests to OpenRouter
Ran into this while adding support for Vercel v0s models:
- The timestamp seems to be returned in Milliseconds instead of seconds
so it breaks the bounds of `created: u32`. We did not use this field
anywhere so just decided to remove it
- Sometimes the `choices` field can be empty when the last chunk comes
in because it only contains `usage`
Release Notes:
- N/A
The `api_url` setting is one that most providers already support and can
be changed via the `settings.json`. We're adding the ability to change
it via the UI for OpenAI specifically so it can be more easily connected
to v0.
Release Notes:
- agent: Added ability to change the API base URL for OpenAI via the UI
---------
Co-authored-by: Bennet Bo Fenner <53836821+bennetbo@users.noreply.github.com>
Related to #32888, but will not fix the issue.
Turns out these assertions are wrong (Not sure if they were correct at
some point).
I tested with this code:
```
request = LanguageModelRequest {
messages: vec![
LanguageModelRequestMessage {
role: Role::User,
content: vec![MessageContent::Text("Give me 10 jokes".to_string())],
cache: false,
},
LanguageModelRequestMessage {
role: Role::Assistant,
content: vec![MessageContent::Text("Sure, here are 10 jokes:".to_string())],
cache: false,
},
],
..request
};
```
The API happily accepted this and Claude proceeded to tell me 10 jokes.
Release Notes:
- N/A
Updates google_ai to use latest model information from the respective
model cards: https://ai.google.dev/gemini-api/docs/models
Release Notes:
- google: Update to latest Gemini 2.5 models
Previously we were using a mix of `u32` and `usize`, e.g. `max_tokens:
usize, max_output_tokens: Option<u32>` in the same `struct`.
Although [tiktoken](https://github.com/openai/tiktoken) uses `usize`,
token counts should be consistent across targets (e.g. the same model
doesn't suddenly get a smaller context window if you're compiling for
wasm32), and these token counts could end up getting serialized using a
binary protocol, so `usize` is not the right choice for token counts.
I chose to standardize on `u64` over `u32` because we don't store many
of them (so the extra size should be insignificant) and future models
may exceed `u32::MAX` tokens.
Release Notes:
- N/A
We push the usage data whenever we receive it from the provider to make
sure the counting is correct after the turn has ended.
- [x] Ollama
- [x] Copilot
- [x] Mistral
- [x] OpenRouter
- [x] LMStudio
Put all the changes into a single PR open to move these to separate PR
if that makes the review and testing easier.
Release Notes:
- N/A
This addresses:
https://github.com/zed-industries/zed/pull/32248#issuecomment-2952060834.
This PR address two main things one allowing enterprise users to use
copilot chat and completion while also introducing the new way to handle
copilot url specific their subscription. Simplifying the UX around the
github copilot and removes the burden of users figuring out what url to
use for their subscription.
- [x] Pass enterprise_uri to copilot lsp so that it can redirect users
to their enterprise server. Ref:
https://github.com/github/copilot-language-server-release#configuration-management
- [x] Remove the old ui and config language_models.copilot which allowed
users to specify their copilot_chat specific endpoint. We now derive
that automatically using token endpoint for copilot so that we can send
the requests to specific copilot endpoint for depending upon the url
returned by copilot server.
- [x] Tested this for checking the both enterprise and non-enterprise
flow work. Thanks to @theherk for the help to debug and test it.
- [ ] Udpdate the zed.dev/docs to refelect how to setup enterprise
copilot.
What this doesn't do at the moment:
* Currently zed doesn't allow to have two seperate accounts as the token
used in chat is same as the one generated by lsp. After this changes
also this behaviour remains same and users can't have both enterprise
and personal copilot installed.
P.S: Might need to do some bit of code cleanup and other things but
overall I felt this PR was ready for atleast first pass of review to
gather feedback around the implementation and code itself.
Release Notes:
- Add enterprise support for GitHub copilot
---------
Signed-off-by: Umesh Yadav <git@umesh.dev>
- [x] Manual Testing(Tested this with Qwen2.5 VL 32B Instruct (free) and
Llama 4 Scout (free), Llama 4 Maverick (free). Llama models have some
issues in write profile due to one of the in built tools schema, so I
tested it with minimal profile.
Closes #ISSUE
Release Notes:
- Add image support to OpenRouter models
---------
Signed-off-by: Umesh Yadav <umesh4257@gmail.com>
Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
Bubbles up rate limit information so that we can retry after a certain
duration if needed higher up in the stack.
Also caps the number of concurrent evals running at once to also help.
Release Notes:
- N/A
Add support for environment variables as authentication alternatives to
OAuth flow for Copilot. Closes#31172
We can include the token in HTTPS request headers to hopefully resolve
the rate limiting issue in #9483. This change will be part of a separate
PR.
Release Notes:
- Added support for manually providing an OAuth token for GitHub Copilot
Chat by assigning the GH_COPILOT_TOKEN environment variable
---------
Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
Tested with following models. Hallucinates with whites outline images
like white lined zed logo but works fine with zed black outlined logo:
Pixtral 12B (pixtral-12b-latest)
Pixtral Large (pixtral-large-latest)
Mistral Medium (mistral-medium-latest)
Mistral Small (mistral-small-latest)
After this PR, almost all of the zed's llm provider who support images
are now supported. Only remaining one is LMStudio. Hopefully we will get
that one as well soon.
Release Notes:
- Add support for images to mistral models
---------
Signed-off-by: Umesh Yadav <git@umesh.dev>
Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
Co-authored-by: Bennet Bo Fenner <bennet@zed.dev>
It works similar to how deepseek works where the thinking is returned as
reasoning_content and we don't have to send the reasoning_content back
in the request.
This is a experiment feature which can be enabled from settings like
this:
<img width="1381" alt="Screenshot 2025-06-08 at 4 26 06 PM"
src="https://github.com/user-attachments/assets/d2f60f3c-0f93-45fc-bae2-4ded42981820"
/>
Here is how it looks to use(tested with
`deepseek/deepseek-r1-0528-qwen3-8b`
<img width="528" alt="Screenshot 2025-06-08 at 5 12 33 PM"
src="https://github.com/user-attachments/assets/f7716f52-5417-4f14-82b8-e853de054f63"
/>
Release Notes:
- Add thinking support to LM Studio provider
For DeepSeek provider thinking is returned as reasoning_content and we
don't have to send the reasoning_content back in the request.
Release Notes:
- Add thinking support to DeepSeek provider
Recently in this PR: https://github.com/zed-industries/zed/pull/32248
github copilot settings was introduced. This had missing settings update
which was leading to github copilot models not getting fetched. This had
missing subscription to update the settings inside the copilot language
model provider. Which caused it not show models at all.
cc @osiewicz
Release Notes:
- N/A
---------
Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com>
The info for `max_tokens` for the model is included in
`{api_url}/models`
I don't think this needs to be `.clamp` like in
`crates/ollama/src/ollama.rs` `get_max_tokens`, but it might need to be
## Before:
Every model shows 2k
## After:
### Json from `{api_url}/models` with model not loaded
```json
{
"id": "qwen2.5-coder-1.5b-instruct-mlx",
"object": "model",
"type": "llm",
"publisher": "lmstudio-community",
"arch": "qwen2",
"compatibility_type": "mlx",
"quantization": "4bit",
"state": "not-loaded",
"max_context_length": 32768
},
```
## Notes
The response from `{api_url}/models` seems to return the `max_tokens`
for the model, not the currently configured context length, but I think
showing the `max_tokens` for the model is better than setting 2k for
everything
`loaded_context_length` exists, but only if the model is loaded at the
startup of zed, which usually isn't the case
maybe `fetch_models` should be rerun when swapping lmstudio models
### Currently configured context
this isn't shown in `{api_url}/models`
### Json from `{api_url}/models` with model loaded
```json
{
"id": "qwen2.5-coder-1.5b-instruct-mlx",
"object": "model",
"type": "llm",
"publisher": "lmstudio-community",
"arch": "qwen2",
"compatibility_type": "mlx",
"quantization": "4bit",
"state": "loaded",
"max_context_length": 32768,
"loaded_context_length": 4096
},
```
Release Notes:
- lmstudio: Fixed showing `max_tokens` in the assistant panel
---------
Co-authored-by: Peter Tripp <peter@zed.dev>
Closes#31491
### Problem
Mistral API enforces strict conversation flow requirements that other
providers don't. Specifically, after a `tool` message, the next message
**must** be from the `assistant` role, not `user`. This causes the
error:
```
"Unexpected role 'user' after role 'tool'"
```
This can also occur in normal conversation flow where mistral doesn't
return the assistant message but that is something which can't be
reproduce reliably.
### Root Cause
When users interrupt an ongoing tool call sequence by sending a new
message, we insert a `user` message directly after a `tool` message,
violating Mistral's protocol.
**Expected Mistral flow:**
```
user → assistant (with tool_calls) → tool (results) → assistant (processes results) → user (next input)
```
**What we were doing:**
```
user → assistant (with tool_calls) → tool (results) → user (interruption) ❌
```
### Solution
Insert an empty `assistant` message between any `tool` → `user` sequence
in the Mistral provider's request construction. This satisfies Mistral's
API requirements without affecting other providers or requiring UX
changes.
### Testing
To reproduce the original error:
1. Start agent chat with `codestral-latest`
2. Send: "Describe this project using tool call only"
3. Once tool calls begin, send: "stop this"
4. Main branch: API error
5. This fix: Works correctly
Release Notes:
- Fixed Mistral tool calling in some cases
Closes#31438
Release Notes:
- agent: Fixed an edge case were the request would fail when using
Claude and multiple images were attached
---------
Co-authored-by: Richard Feldman <oss@rtfeldman.com>
Removes the load_model trait method and its implementations in Ollama
and LM Studio providers, along with associated preload_model functions
and unused imports.
Release Notes:
- N/A
This pull request adds full integration with OpenRouter, allowing users
to access a wide variety of language models through a single API key.
**Implementation Details:**
* **Provider Registration:** Registers OpenRouter as a new language
model provider within the application's model registry. This includes UI
for API key authentication, token counting, streaming completions, and
tool-call handling.
* **Dedicated Crate:** Adds a new `open_router` crate to manage
interactions with the OpenRouter HTTP API, including model discovery and
streaming helpers.
* **UI & Configuration:** Extends workspace manifests, the settings
schema, icons, and default configurations to surface the OpenRouter
provider and its settings within the UI.
* **Readability:** Reformats JSON arrays within the settings files for
improved readability.
**Design Decisions & Discussion Points:**
* **Code Reuse:** I leveraged much of the existing logic from the
`openai` provider integration due to the significant similarities
between the OpenAI and OpenRouter API specifications.
* **Default Model:** I set the default model to `openrouter/auto`. This
model automatically routes user prompts to the most suitable underlying
model on OpenRouter, providing a convenient starting point.
* **Model Population Strategy:**
* <strike>I've implemented dynamic population of available models by
querying the OpenRouter API upon initialization.
* Currently, this involves three separate API calls: one for all models,
one for tool-use models, and one for models good at programming.
* The data from the tool-use API call sets a `tool_use` flag for
relevant models.
* The data from the programming models API call is used to sort the
list, prioritizing coding-focused models in the dropdown.</strike>
* <strike>**Feedback Welcome:** I acknowledge this multi-call approach
is API-intensive. I am open to feedback and alternative implementation
suggestions if the team believes this can be optimized.</strike>
* **Update: Now this has been simplified to one api call.**
* **UI/UX Considerations:**
* <strike>Authentication Method: Currently, I've implemented the
standard API key input in settings, similar to other providers like
OpenAI/Anthropic. However, OpenRouter also supports OAuth 2.0 with PKCE.
This could offer a potentially smoother, more integrated setup
experience for users (e.g., clicking a button to authorize instead of
copy-pasting a key). Should we prioritize implementing OAuth PKCE now,
or perhaps add it as an alternative option later?</strike>(PKCE is not
straight forward and complicated so skipping this for now. So that we
can add the support and work on this later.)
* <strike>To visually distinguish models better suited for programming,
I've considered adding a marker (e.g., `</>` or `🧠`) next to their
names. Thoughts on this proposal?</strike>. (This will require a changes
and discussion across model provider. This doesn't fall under the scope
of current PR).
* OpenRouter offers 300+ models. The current implementation loads all of
them. **Feedback Needed:** Should we refine this list or implement more
sophisticated filtering/categorization for better usability?
**Motivation:**
This integration directly addresses one of the most highly upvoted
feature requests/discussions within the Zed community. Adding OpenRouter
support significantly expands the range of AI models accessible to
users.
I welcome feedback from the Zed team on this implementation and the
design choices made. I am eager to refine this feature and make it
available to users.
ISSUES: https://github.com/zed-industries/zed/discussions/16576
Release Notes:
- Added support for OpenRouter as a language model provider.
---------
Signed-off-by: Umesh Yadav <umesh4257@gmail.com>
Co-authored-by: Marshall Bowers <git@maxdeviant.com>
Closes#30535
Release Notes:
- AWS Bedrock: Add support for Meta Llama 4 Scout and Maverick models.
- AWS Bedrock: Fixed cross-region inference for all regions.
- AWS Bedrock: Updated all models available through Cross Region
inference.
---------
Co-authored-by: Marshall Bowers <git@maxdeviant.com>
Hello,
This is my first contribution so apologies if I'm not following the
proper process (I haven't seen anything special in
https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md). Also,
I have tested my changes manually, but I could not figure out an easy we
to instantiate a `LanguageModelSelector` in the unit tests, so I didn't
write a test. If you can provide some guidance I'd be happy to write a
test.
---
If the user configured the models with custom names via `display_name`,
we want the ollama models to be sorted based on the name that is
actually displayed.
~~The original issue is only about ollama but this change will also
affect the other providers.~~
Closes#30854
Release Notes:
- Ollama: Changed models to be sorted by name.
Closes#31243
As described in my issue, the [thinking
budget](https://ai.google.dev/gemini-api/docs/thinking) gets
automatically chosen by Gemini unless it is specifically set to
something. In order to have fast responses (inline assistant) I prefer
to set it to 0.
Release Notes:
- ai: Added `thinking` mode for custom Google models with configurable
token budget
---------
Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
[deepseek function call
api](https://api-docs.deepseek.com/guides/function_calling)
has been released and it is same as openai.
Release Notes:
- Added tool calling support for Deepseek Models
---------
Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
This PR updates how we handle Ollama responses, leveraging the new
[v0.9.0](https://github.com/ollama/ollama/releases/tag/v0.9.0) release.
Previously, thinking text was embedded within the model's main content,
leading to it appearing directly in the agent's response. Now, thinking
content is provided as a separate parameter, allowing us to display it
correctly within the agent panel, similar to other providers. I have
tested this with qwen3:8b and works nicely. ~~We can release this once
the ollama is release is stable.~~ It's released now as stable.
<img width="433" alt="image"
src="https://github.com/user-attachments/assets/2983ef06-6679-4033-82c2-231ea9cd6434"
/>
Release Notes:
- Add thinking support for ollama
---------
Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
We report the total number of input tokens by summing the numbers of
1. Prompt tokens
2. Cached tokens
But Google API returns prompt tokens (1) that already include cached
tokens (2), so we were double counting tokens in some cases.
Release Notes:
- Fixed bug with double-counting tokens in Gemini
This PR adds a new `intent` field to completion requests to assist in
categorizing them correctly.
Release Notes:
- N/A
---------
Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
This expands our deserialization of JSON from models to be more tolerant
of different variations that the model may send, including
capitalization, wrapping things in objects vs. being plain strings, etc.
Also when deserialization fails, it reports the entire error in the JSON
so we can see what failed to deserialize. (Previously these errors were
very unhelpful at diagnosing the problem.)
Finally, also removes the `WrappedText` variant since the custom
deserializer just turns that style of JSON into a normal `Text` variant.
Release Notes:
- N/A
Closes#30004
**Quick demo:**
https://github.com/user-attachments/assets/0ac93851-81d7-4128-a34b-1f3ae4bcff6d
**Additional notes:**
I've tried to stick to existing code in OpenAI provider as much as
possible without changing much to keep the diff small.
This PR is done in collaboration with @yagil from LM Studio. We agreed
upon the format in which LM Studio will return information about tool
use support for the model in the upcoming version. As of current stable
version nothing is going to change for the users, but once they update
to a newer LM Studio tool use gets automatically enabled for them. I
think this is much better UX then defaulting to true right now.
Release Notes:
- Added support for tool calls to LM Studio provider
---------
Co-authored-by: Ben Brandt <benjamin.j.brandt@gmail.com>
Fixes regression caused by:
https://github.com/zed-industries/zed/pull/30639
Assistant messages can come back with no content, and we no longer
allowed that in the deserialization.
Release Notes:
- open_ai: fixed deserialization issue if assistant content was empty
This PR updates the Zed LLM provider to fetch the available models from
the server instead of hard-coding them in the binary.
Release Notes:
- Updated the Zed provider to fetch the list of available language
models from the server.
This PR updates the default/recommended models for the Anthropic and Zed
providers to be Claude Sonnet 4.
Release Notes:
- Updated default/recommended Anthropic models to Claude Sonnet 4.
This PR adds support for [Claude
4](https://www.anthropic.com/news/claude-4).
Release Notes:
- Added support for Claude Opus 4 and Claude Sonnet 4.
---------
Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Richard Feldman <oss@rtfeldman.com>
https://github.com/zed-industries/zed/issues/30972 brought up another
case where our context is not enough to track the actual source of the
issue: we get a general top-level error without inner error.
The reason for this was `.ok_or_else(|| anyhow!("failed to read HEAD
SHA"))?; ` on the top level.
The PR finally reworks the way we use anyhow to reduce such issues (or
at least make it simpler to bubble them up later in a fix).
On top of that, uses a few more anyhow methods for better readability.
* `.ok_or_else(|| anyhow!("..."))`, `map_err` and other similar error
conversion/option reporting cases are replaced with `context` and
`with_context` calls
* in addition to that, various `anyhow!("failed to do ...")` are
stripped with `.context("Doing ...")` messages instead to remove the
parasitic `failed to` text
* `anyhow::ensure!` is used instead of `if ... { return Err(...); }`
calls
* `anyhow::bail!` is used instead of `return Err(anyhow!(...));`
Release Notes:
- N/A
Some providers sometimes send `{ "type": "text", "text": ... }` instead
of just the text as a string. Now we accept those instead of erroring.
Release Notes:
- N/A
