This PR adds a new `language_models` crate to house the various language
model providers.
By extracting the provider definitions out of `language_model`, we're
able to remove `language_model`'s dependency on `editor`, which improves
incremental compilation when changing `editor`.
Release Notes:
- N/A
This removes the `low_speed_timeout` setting from all providers as a
response to issue #19509.
Reason being that the original `low_speed_timeout` was only as part of
#9913 because users wanted to _get rid of timeouts_. They wanted to bump
the default timeout from 5sec to a lot more.
Then, in the meantime, the meaning of `low_speed_timeout` changed in
#19055 and was changed to a normal `timeout`, which is a different thing
and breaks slower LLMs that don't reply with a complete response in the
configured timeout.
So we figured: let's remove the whole thing and replace it with a
default _connect_ timeout to make sure that we can connect to a server
in 10s, but then give the server as long as it wants to complete its
response.
Closes#19509
Release Notes:
- Removed the `low_speed_timeout` setting from LLM provider settings,
since it was only used to _increase_ the timeout to give LLMs more time,
but since we don't have any other use for it, we simply remove the
setting to give LLMs as long as they need.
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Peter Tripp <peter@zed.dev>
Closes https://github.com/zed-industries/zed/issues/16568
This PR introduces some changes to how we display models in the model
selector within the assistant panel. Basically, it comes down to this:
- If you don't have any provider configured, you should see _all_
available models in the picker
- But, once you've configured some, you should _only_ see models from
them in the picker
Visually, nothing's changed much aside from the added "Configured
Models" label at the top to ensure the understanding that that's a list
of, well, configured models only. 😬
<img width="700" alt="Screenshot 2024-11-07 at 23 42 41"
src="https://github.com/user-attachments/assets/219ed386-2318-43a6-abea-1de0cda8dc53">
Release Notes:
- Change model selector in the assistant panel to only show configured
models
This PR adds a new `zed_urls` module to the `client` crate.
This module contains functions for constructing URLs to Zed properties,
such as zed.dev.
The URLs produced by this module will respect the server URL set via
settings or the `ZED_SERVER_URL` environment variable. This allows them
to correctly reflect the current environment (such as when testing Zed
against a local collab/zed.dev).
Release Notes:
- N/A
This PR adds support to the assistant for display billing-related
errors.
Pulling this out of #19081 to make it easier to cherry-pick.
Release Notes:
- N/A
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Richard <richard@zed.dev>
Users of our http_client crate knew they were interacting with isahc as
they set its extensions on the request. This change adds our own
equivalents for their APIs in preparation for changing the default http
client.
Release Notes:
- N/A
Release Notes:
- Allow Anthropic custom models to override "temperature"
This also centralized the defaulting of "temperature" to be inside of
each model's `into_x` call instead of being sprinkled around the code.
This is a barebones modification of the OpenAI provider code to
accommodate non-streaming completions. This is specifically for the o1
models, which do not support streaming. Tested that this is working by
running a `/workflow` with the following (arbitrarily chosen) settings:
```json
{
"language_models": {
"openai": {
"version": "1",
"available_models": [
{
"name": "o1-preview",
"display_name": "o1-preview",
"max_tokens": 128000,
"max_completion_tokens": 30000
},
{
"name": "o1-mini",
"display_name": "o1-mini",
"max_tokens": 128000,
"max_completion_tokens": 20000
}
]
}
},
}
```
Release Notes:
- Changed `low_speed_timeout_in_seconds` option to `600` for OpenAI
provider to accommodate recent o1 model release.
---------
Co-authored-by: Peter <peter@zed.dev>
Co-authored-by: Bennet <bennet@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
This PR makes it so we propagate the `stop_reason` from Anthropic up to
the Assistant so that we can take action based on it.
The `extract_content_from_events` function was moved from `anthropic` to
the `anthropic` module in `language_model` since it is more useful if it
is able to name the `LanguageModelCompletionEvent` type, as otherwise
we'd need an additional layer of plumbing.
Release Notes:
- N/A
This PR adjusts the approach we use to encoding tool uses in the
completion response to use a structured format rather than simply
injecting it into the response stream as text.
In #17170 we would encode the tool uses as XML and insert them as text.
This would require then re-parsing the tool uses out of the buffer in
order to use them.
The approach taken in this PR is to make `stream_completion` return a
stream of `LanguageModelCompletionEvent`s. Each of these events can be
either text, or a tool use.
A new `stream_completion_text` method has been added to `LanguageModel`
for scenarios where we only care about textual content (currently,
everywhere that isn't the Assistant context editor).
Release Notes:
- N/A
This PR updates the Assistant with support for receiving tool uses from
Anthropic models and capturing them as text in the context editor.
This is just laying the foundation for tool use. We don't yet fulfill
the tool uses yet, or define any tools for the model to use.
Here's an example of what it looks like using the example `get_weather`
tool from the Anthropic docs:
<img width="644" alt="Screenshot 2024-08-30 at 1 51 13 PM"
src="https://github.com/user-attachments/assets/3614f953-0689-423c-8955-b146729ea638">
Release Notes:
- N/A
This PR makes it so the model's cache configuration gets passed through
from the base model when using the Zed provider.
Release Notes:
- Fixed caching for Anthropic models when using the Zed provider.
### Pull Request Title
Introduce `max_output_tokens` Field for OpenAI Models
https://platform.deepseek.com/api-docs/news/news0725/#4-8k-max_tokens-betarelease-longer-possibilities
### Description
This commit introduces a new field `max_output_tokens` to the OpenAI
models, which allows specifying the maximum number of tokens that can be
generated in the output. This field is now integrated into the request
handling across multiple crates, ensuring that the output token limit is
respected during language model completions.
Changes include:
- Adding `max_output_tokens` to the `Custom` variant of the
`open_ai::Model` enum.
- Updating the `into_open_ai` method in `LanguageModelRequest` to accept
and use `max_output_tokens`.
- Modifying the `OpenAiLanguageModel` and `CloudLanguageModel`
implementations to pass `max_output_tokens` when converting requests.
- Ensuring that the `max_output_tokens` field is correctly serialized
and deserialized in relevant structures.
This enhancement provides more control over the output length of OpenAI
model responses, improving the flexibility and accuracy of language
model interactions.
### Changes
- Added `max_output_tokens` to the `Custom` variant of the
`open_ai::Model` enum.
- Updated the `into_open_ai` method in `LanguageModelRequest` to accept
and use `max_output_tokens`.
- Modified the `OpenAiLanguageModel` and `CloudLanguageModel`
implementations to pass `max_output_tokens` when converting requests.
- Ensured that the `max_output_tokens` field is correctly serialized and
deserialized in relevant structures.
### Related Issue
https://github.com/zed-industries/zed/pull/16358
### Screenshots / Media
N/A
### Checklist
- [x] Code compiles correctly.
- [x] All tests pass.
- [ ] Documentation has been updated accordingly.
- [ ] Additional tests have been added to cover new functionality.
- [ ] Relevant documentation has been updated or added.
### Release Notes
- Added `max_output_tokens` field to OpenAI models for controlling
output token length.
This commit adds a custom icon for Anthropic hosted models.
- Adding a new SVG icon for Anthropic hosted models.
- The new icon is located at: `assets/icons/ai_anthropic_hosted.svg`
- Updating the LanguageModel trait to include an optional icon method
- Implementing the icon method for CloudModel to return the custom icon
for Anthropic hosted models
- Updating the UI components to use the model-specific icon when
available
- Adding a new IconName variant for the Anthropic hosted icon
We should change the non-hosted icon in some small way to distinguish it
from the hosted version. I duplicated the path for now so we can
hopefully add it for the next release.
Release Notes:
- N/A
Release Notes:
- Adds support for Prompt Caching in Anthropic. For models that support
it this can dramatically lower cost while improving performance.
This PR is a refactor to pave the way for allowing the user to view and
edit workflow step resolutions. I've made tool calls work more like
normal streaming completions for all providers. The `use_any_tool`
method returns a stream of strings (which contain chunks of JSON). I've
also done some minor cleanup of language model providers in general,
removing the duplication around handling streaming responses.
Release Notes:
- N/A
This PR adds feature-flagged access to the LLM service.
We've repurposed the `language-models` feature flag to be used for
providing access to Claude 3.5 Sonnet through the Zed provider.
The remaining RPC endpoints that were previously behind the
`language-models` feature flag are now behind a staff check.
We also put some Zed Pro related messaging behind a feature flag.
Release Notes:
- N/A
---------
Co-authored-by: Max <max@zed.dev>
Now, when an anthropic request is invalid or anthropic's API is down,
we'll expose that to the user instead of just returning a generic 500.
Release Notes:
- N/A
Co-authored-by: Marshall <marshall@zed.dev>
This PR makes it so hitting upstream rate limits from Anthropic result
in an HTTP 429 response instead of an HTTP 500.
To do this we need to surface structured errors out of the `anthropic`
crate.
Release Notes:
- N/A
This adds the requirement for users to accept the terms of service the
first time they send a message with the Cloud provider.
Once this is out and in a nightly, we need to add the check to the
server side too, to authenticate access to the models.
Demo:
https://github.com/user-attachments/assets/0edebf74-8120-4fa2-b801-bb76f04e8a17
Release Notes:
- N/A
This PR removes the `llm-service` feature flag and makes it so all
completions are done via the LLM service when using the Zed provider.
Release Notes:
- N/A
- [x] OpenAI
- [ ] ~Google~ Moved into a separate branch at:
https://github.com/zed-industries/zed/tree/tool-calls-in-google-ai I've
ran into issues with having the API digest our schema without tripping
over itself - the function call parameters are malformed and whatnot. We
can resume from that branch if needed.
- [x] Ollama
- [x] Cloud
- [ ] ~Copilot Chat (?)~
Release Notes:
- Added tool calling capabilities to OpenAI and Ollama models.
This PR introduces a separate backend service for making LLM calls.
It exposes an HTTP interface that can be called by Zed clients. To call
these endpoints, the client must provide a `Bearer` token. These tokens
are issued/refreshed by the collab service over RPC.
We're adding this in a backwards-compatible way. Right now the access
tokens can only be minted for Zed staff, and calling this separate LLM
service is behind the `llm-service` feature flag (which is not
automatically enabled for Zed staff).
Release Notes:
- N/A
---------
Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
TODOs for follow-up:
- [ ] When opening panel: nudge user to sign in if they're not signed-in
and have no provider configured (or if they're not signed-in and have
Zed AI configured)
- [ ] Configuration page is not scrollable
- [ ] Design tweaks
Current status:
https://github.com/user-attachments/assets/d26d65ea-43e8-481b-81a3-b3cba01704a8
Release Notes:
- N/A
This PR updates the `LanguageModel` trait with a new method for denoting
the availability of a model.
Right now we have two variants:
- `Public` for models that have no additional restrictions (other than
their respective setup/authentication requirements)
- `RequiresPlan` for models that require a specific Zed plan
Release Notes:
- N/A
This PR renames the links to the `zed.dev/settings` page to the
`zed.dev/account`.
Some of these spots will likely link out to a marketing page later.
Release Notes:
- N/A
- [x] bug: setting a key doesn't update anything
- [x] show high-level text on configuration page to explain what it is
- [x] show "everything okay!" status when credentials are set
- [x] maybe: add "verify" button to check credentials
- [x] open configuration page when opening panel for first time and
nothing is configured
- [x] BUG: need to fix empty assistant panel if provider is `zed.dev`
but not logged in
Co-Authored-By: Thorsten <thorsten@zed.dev>
Release Notes:
- N/A
---------
Co-authored-by: Thorsten <thorsten@zed.dev>
Co-authored-by: Nate Butler <iamnbutler@gmail.com>
Co-authored-by: Thorsten Ball <mrnugget@gmail.com>
There was/is some race condition that gets triggered only with the
zed.dev provider when opening the provider that would cause a
double-borrow on workspace.
This PR fixes the issue by cloning the workspace weakly. Turns out we
can go very far with just the weak reference.
We're still a bit unsure why exactly the race condition happened, since
it's hard to reproduce, but we're working on configuration
view/management in #15490 anyway.
Release Notes:
- N/A
Co-authored-by: Bennet <bennet@zed.dev>
This is the revised version of #15527.
We also added new events to notify subscribers when new providers are
added or removed.
Co-Authored-by: Thorsten <thorsten@zed.dev>
Release Notes:
- N/A
---------
Co-authored-by: Thorsten <thorsten@zed.dev>
Co-authored-by: Thorsten Ball <mrnugget@gmail.com>
We also eliminate the `completion` crate and moved its logic into
`LanguageModelRegistry`.
Release Notes:
- N/A
---------
Co-authored-by: Nathan <nathan@zed.dev>
