Add capabilities to OpenAI-compatible model settings (#36370)
### TL;DR * Adds `capabilities` configuration for OpenAI-compatible models * Relates to https://github.com/zed-industries/zed/issues/36215#issuecomment-3193920491 ### Summary This PR introduces support for configuring model capabilities for OpenAI-compatible language models. The implementation addresses the issue that not all OpenAI-compatible APIs support the same features - for example, Cerebras' API explicitly does not support `parallel_tool_calls` as documented in their [OpenAI compatibility guide](https://inference-docs.cerebras.ai/resources/openai#currently-unsupported-openai-features). ### Changes 1. **Model Capabilities Structure**: - Added `ModelCapabilityToggles` struct for UI representation with boolean toggle states - Implemented proper parsing of capability toggles into `ModelCapabilities` 2. **UI Updates**: - Modified the "Add LLM Provider" modal to include checkboxes for each capability - Each OpenAI-compatible model can now be configured with its specific capabilities through the UI 3. **Configuration File Structure**: - Updated the settings schema to support a `capabilities` object for each `openai_compatible` model - Each capability (`tools`, `images`, `parallel_tool_calls`, `prompt_cache_key`) can be individually specified per model ### Example Configuration ```json { "openai_compatible": { "Cerebras": { "api_url": "https://api.cerebras.ai/v1", "available_models": [ { "name": "gpt-oss-120b", "max_tokens": 131000, "capabilities": { "tools": true, "images": false, "parallel_tool_calls": false, "prompt_cache_key": false } } ] } } } ``` ### Tests Added - Added tests to verify default capability values are correctly applied - Added tests to verify that deselected toggles are properly parsed as `false` - Added tests to verify that mixed capability selections work correctly Thanks to @osyvokon for the desired `capabilities` configuration structure! Release Notes: - OpenAI-compatible models now have configurable capabilities (#36370; thanks @calesennett) --------- Co-authored-by: Oleksiy Syvokon <oleksiy@zed.dev>
This commit is contained in:
parent
ea828c0c59
commit
61ce07a91b
3 changed files with 208 additions and 12 deletions
|
@ -427,7 +427,7 @@ Custom models will be listed in the model dropdown in the Agent Panel.
|
|||
Zed supports using [OpenAI compatible APIs](https://platform.openai.com/docs/api-reference/chat) by specifying a custom `api_url` and `available_models` for the OpenAI provider.
|
||||
This is useful for connecting to other hosted services (like Together AI, Anyscale, etc.) or local models.
|
||||
|
||||
You can add a custom, OpenAI-compatible model via either via the UI or by editing your `settings.json`.
|
||||
You can add a custom, OpenAI-compatible model either via the UI or by editing your `settings.json`.
|
||||
|
||||
To do it via the UI, go to the Agent Panel settings (`agent: open settings`) and look for the "Add Provider" button to the right of the "LLM Providers" section title.
|
||||
Then, fill up the input fields available in the modal.
|
||||
|
@ -443,7 +443,13 @@ To do it via your `settings.json`, add the following snippet under `language_mod
|
|||
{
|
||||
"name": "mistralai/Mixtral-8x7B-Instruct-v0.1",
|
||||
"display_name": "Together Mixtral 8x7B",
|
||||
"max_tokens": 32768
|
||||
"max_tokens": 32768,
|
||||
"capabilities": {
|
||||
"tools": true,
|
||||
"images": false,
|
||||
"parallel_tool_calls": false,
|
||||
"prompt_cache_key": false
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
|
@ -451,6 +457,13 @@ To do it via your `settings.json`, add the following snippet under `language_mod
|
|||
}
|
||||
```
|
||||
|
||||
By default, OpenAI-compatible models inherit the following capabilities:
|
||||
|
||||
- `tools`: true (supports tool/function calling)
|
||||
- `images`: false (does not support image inputs)
|
||||
- `parallel_tool_calls`: false (does not support `parallel_tool_calls` parameter)
|
||||
- `prompt_cache_key`: false (does not support `prompt_cache_key` parameter)
|
||||
|
||||
Note that LLM API keys aren't stored in your settings file.
|
||||
So, ensure you have it set in your environment variables (`OPENAI_API_KEY=<your api key>`) so your settings can pick it up.
|
||||
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue