This PR adds a new `language_models` crate to house the various language
model providers.
By extracting the provider definitions out of `language_model`, we're
able to remove `language_model`'s dependency on `editor`, which improves
incremental compilation when changing `editor`.
Release Notes:
- N/A
This removes the `low_speed_timeout` setting from all providers as a
response to issue #19509.
Reason being that the original `low_speed_timeout` was only as part of
#9913 because users wanted to _get rid of timeouts_. They wanted to bump
the default timeout from 5sec to a lot more.
Then, in the meantime, the meaning of `low_speed_timeout` changed in
#19055 and was changed to a normal `timeout`, which is a different thing
and breaks slower LLMs that don't reply with a complete response in the
configured timeout.
So we figured: let's remove the whole thing and replace it with a
default _connect_ timeout to make sure that we can connect to a server
in 10s, but then give the server as long as it wants to complete its
response.
Closes#19509
Release Notes:
- Removed the `low_speed_timeout` setting from LLM provider settings,
since it was only used to _increase_ the timeout to give LLMs more time,
but since we don't have any other use for it, we simply remove the
setting to give LLMs as long as they need.
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Peter Tripp <peter@zed.dev>
Before this change, we'd see a ton of requests from the Ollama provider
trying to fetch models:
```
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: https://api.zed.dev/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
[2024-10-28T15:00:52+01:00 DEBUG reqwest::connect] starting new connection: http://localhost:11434/
```
Turns out we'd send a request on *every* change to settings.
Now, with this change, we only send a single request.
Release Notes:
- N/A
Co-authored-by: Bennet <bennet@zed.dev>
Release Notes:
- Allow Anthropic custom models to override "temperature"
This also centralized the defaulting of "temperature" to be inside of
each model's `into_x` call instead of being sprinkled around the code.
This PR adjusts the approach we use to encoding tool uses in the
completion response to use a structured format rather than simply
injecting it into the response stream as text.
In #17170 we would encode the tool uses as XML and insert them as text.
This would require then re-parsing the tool uses out of the buffer in
order to use them.
The approach taken in this PR is to make `stream_completion` return a
stream of `LanguageModelCompletionEvent`s. Each of these events can be
either text, or a tool use.
A new `stream_completion_text` method has been added to `LanguageModel`
for scenarios where we only care about textual content (currently,
everywhere that isn't the Assistant context editor).
Release Notes:
- N/A
This PR is a refactor to pave the way for allowing the user to view and
edit workflow step resolutions. I've made tool calls work more like
normal streaming completions for all providers. The `use_any_tool`
method returns a stream of strings (which contain chunks of JSON). I've
also done some minor cleanup of language model providers in general,
removing the duplication around handling streaming responses.
Release Notes:
- N/A
For future reference: WIP branch of copy/pasting a mixture of images and
text: https://github.com/zed-industries/zed/tree/copy-paste-images -
we'll come back to that one after landing this one.
Release Notes:
- You can now paste images into the Assistant Panel to include them as
context. Currently works only on Mac, and with Anthropic models. Future
support is planned for more models, operating systems, and image
clipboard operations.
---------
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Mikayla <mikayla@zed.dev>
Co-authored-by: Jason <jason@zed.dev>
Co-authored-by: Kyle <kylek@zed.dev>
- [x] OpenAI
- [ ] ~Google~ Moved into a separate branch at:
https://github.com/zed-industries/zed/tree/tool-calls-in-google-ai I've
ran into issues with having the API digest our schema without tripping
over itself - the function call parameters are malformed and whatnot. We
can resume from that branch if needed.
- [x] Ollama
- [x] Cloud
- [ ] ~Copilot Chat (?)~
Release Notes:
- Added tool calling capabilities to OpenAI and Ollama models.
TODOs for follow-up:
- [ ] When opening panel: nudge user to sign in if they're not signed-in
and have no provider configured (or if they're not signed-in and have
Zed AI configured)
- [ ] Configuration page is not scrollable
- [ ] Design tweaks
Current status:
https://github.com/user-attachments/assets/d26d65ea-43e8-481b-81a3-b3cba01704a8
Release Notes:
- N/A
- [x] bug: setting a key doesn't update anything
- [x] show high-level text on configuration page to explain what it is
- [x] show "everything okay!" status when credentials are set
- [x] maybe: add "verify" button to check credentials
- [x] open configuration page when opening panel for first time and
nothing is configured
- [x] BUG: need to fix empty assistant panel if provider is `zed.dev`
but not logged in
Co-Authored-By: Thorsten <thorsten@zed.dev>
Release Notes:
- N/A
---------
Co-authored-by: Thorsten <thorsten@zed.dev>
Co-authored-by: Nate Butler <iamnbutler@gmail.com>
Co-authored-by: Thorsten Ball <mrnugget@gmail.com>
This is the revised version of #15527.
We also added new events to notify subscribers when new providers are
added or removed.
Co-Authored-by: Thorsten <thorsten@zed.dev>
Release Notes:
- N/A
---------
Co-authored-by: Thorsten <thorsten@zed.dev>
Co-authored-by: Thorsten Ball <mrnugget@gmail.com>
We also eliminate the `completion` crate and moved its logic into
`LanguageModelRegistry`.
Release Notes:
- N/A
---------
Co-authored-by: Nathan <nathan@zed.dev>
This PR mostly refines the model selector popover design by formatting
the models names' and adjusting spacing/alignment in the list-related
items. The list component changes could've been made in a separate PR
but it was also very practical to do it here as I was already
in-context. Either way, I'm happy to separate if that's better!
One thing I couldn't necessarily figure out, though, is why the order
changed (e.g., Anthropic at last ). I wonder if that was because of the
separator logic somehow? I'd love guidance here—new to Rust!
| Before | After |
|--------|--------|
| <img width="228" alt="Screenshot 2024-07-23 at 21 02 33"
src="https://github.com/user-attachments/assets/3372c6c9-08dc-4d71-9265-26f015e2dbc2">
| <img width="228" alt="Screenshot 2024-07-23 at 21 01 45"
src="https://github.com/user-attachments/assets/624cc7db-a3d9-48e3-99d7-c29829501130">
|
---
Release Notes:
- N/A
---------
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
Co-authored-by: Bennet Bo Fenner <bennet@zed.dev>
Co-authored-by: Antonio <antonio@zed.dev>
Co-authored-by: Antonio Scandurra <me@as-cii.com>
<img width="624" alt="image"
src="https://github.com/user-attachments/assets/f492b0bd-14c3-49e2-b2ff-dc78e52b0815">
- [x] Correctly set custom model token count
- [x] How to count tokens for Gemini models?
- [x] Feature flag zed.dev provider
- [x] Figure out how to configure custom models
- [ ] Update docs
Release Notes:
- Added support for quickly switching between multiple language model
providers in the assistant panel
---------
Co-authored-by: Antonio <antonio@zed.dev>
2024-07-23 19:48:41 +02:00
Renamed from crates/completion/src/ollama.rs (Browse further)
