
This pull request adds full integration with OpenRouter, allowing users to access a wide variety of language models through a single API key. **Implementation Details:** * **Provider Registration:** Registers OpenRouter as a new language model provider within the application's model registry. This includes UI for API key authentication, token counting, streaming completions, and tool-call handling. * **Dedicated Crate:** Adds a new `open_router` crate to manage interactions with the OpenRouter HTTP API, including model discovery and streaming helpers. * **UI & Configuration:** Extends workspace manifests, the settings schema, icons, and default configurations to surface the OpenRouter provider and its settings within the UI. * **Readability:** Reformats JSON arrays within the settings files for improved readability. **Design Decisions & Discussion Points:** * **Code Reuse:** I leveraged much of the existing logic from the `openai` provider integration due to the significant similarities between the OpenAI and OpenRouter API specifications. * **Default Model:** I set the default model to `openrouter/auto`. This model automatically routes user prompts to the most suitable underlying model on OpenRouter, providing a convenient starting point. * **Model Population Strategy:** * <strike>I've implemented dynamic population of available models by querying the OpenRouter API upon initialization. * Currently, this involves three separate API calls: one for all models, one for tool-use models, and one for models good at programming. * The data from the tool-use API call sets a `tool_use` flag for relevant models. * The data from the programming models API call is used to sort the list, prioritizing coding-focused models in the dropdown.</strike> * <strike>**Feedback Welcome:** I acknowledge this multi-call approach is API-intensive. I am open to feedback and alternative implementation suggestions if the team believes this can be optimized.</strike> * **Update: Now this has been simplified to one api call.** * **UI/UX Considerations:** * <strike>Authentication Method: Currently, I've implemented the standard API key input in settings, similar to other providers like OpenAI/Anthropic. However, OpenRouter also supports OAuth 2.0 with PKCE. This could offer a potentially smoother, more integrated setup experience for users (e.g., clicking a button to authorize instead of copy-pasting a key). Should we prioritize implementing OAuth PKCE now, or perhaps add it as an alternative option later?</strike>(PKCE is not straight forward and complicated so skipping this for now. So that we can add the support and work on this later.) * <strike>To visually distinguish models better suited for programming, I've considered adding a marker (e.g., `</>` or `🧠`) next to their names. Thoughts on this proposal?</strike>. (This will require a changes and discussion across model provider. This doesn't fall under the scope of current PR). * OpenRouter offers 300+ models. The current implementation loads all of them. **Feedback Needed:** Should we refine this list or implement more sophisticated filtering/categorization for better usability? **Motivation:** This integration directly addresses one of the most highly upvoted feature requests/discussions within the Zed community. Adding OpenRouter support significantly expands the range of AI models accessible to users. I welcome feedback from the Zed team on this implementation and the design choices made. I am eager to refine this feature and make it available to users. ISSUES: https://github.com/zed-industries/zed/discussions/16576 Release Notes: - Added support for OpenRouter as a language model provider. --------- Signed-off-by: Umesh Yadav <umesh4257@gmail.com> Co-authored-by: Marshall Bowers <git@maxdeviant.com>
440 lines
16 KiB
Rust
440 lines
16 KiB
Rust
use std::sync::Arc;
|
|
|
|
use anyhow::Result;
|
|
use gpui::App;
|
|
use language_model::LanguageModelCacheConfiguration;
|
|
use project::Fs;
|
|
use schemars::JsonSchema;
|
|
use serde::{Deserialize, Serialize};
|
|
use settings::{Settings, SettingsSources, update_settings_file};
|
|
|
|
use crate::provider::{
|
|
self,
|
|
anthropic::AnthropicSettings,
|
|
bedrock::AmazonBedrockSettings,
|
|
cloud::{self, ZedDotDevSettings},
|
|
copilot_chat::CopilotChatSettings,
|
|
deepseek::DeepSeekSettings,
|
|
google::GoogleSettings,
|
|
lmstudio::LmStudioSettings,
|
|
mistral::MistralSettings,
|
|
ollama::OllamaSettings,
|
|
open_ai::OpenAiSettings,
|
|
open_router::OpenRouterSettings,
|
|
};
|
|
|
|
/// Initializes the language model settings.
|
|
pub fn init(fs: Arc<dyn Fs>, cx: &mut App) {
|
|
AllLanguageModelSettings::register(cx);
|
|
|
|
if AllLanguageModelSettings::get_global(cx)
|
|
.openai
|
|
.needs_setting_migration
|
|
{
|
|
update_settings_file::<AllLanguageModelSettings>(fs.clone(), cx, move |setting, _| {
|
|
if let Some(settings) = setting.openai.clone() {
|
|
let (newest_version, _) = settings.upgrade();
|
|
setting.openai = Some(OpenAiSettingsContent::Versioned(
|
|
VersionedOpenAiSettingsContent::V1(newest_version),
|
|
));
|
|
}
|
|
});
|
|
}
|
|
|
|
if AllLanguageModelSettings::get_global(cx)
|
|
.anthropic
|
|
.needs_setting_migration
|
|
{
|
|
update_settings_file::<AllLanguageModelSettings>(fs, cx, move |setting, _| {
|
|
if let Some(settings) = setting.anthropic.clone() {
|
|
let (newest_version, _) = settings.upgrade();
|
|
setting.anthropic = Some(AnthropicSettingsContent::Versioned(
|
|
VersionedAnthropicSettingsContent::V1(newest_version),
|
|
));
|
|
}
|
|
});
|
|
}
|
|
}
|
|
|
|
#[derive(Default)]
|
|
pub struct AllLanguageModelSettings {
|
|
pub anthropic: AnthropicSettings,
|
|
pub bedrock: AmazonBedrockSettings,
|
|
pub ollama: OllamaSettings,
|
|
pub openai: OpenAiSettings,
|
|
pub open_router: OpenRouterSettings,
|
|
pub zed_dot_dev: ZedDotDevSettings,
|
|
pub google: GoogleSettings,
|
|
pub copilot_chat: CopilotChatSettings,
|
|
pub lmstudio: LmStudioSettings,
|
|
pub deepseek: DeepSeekSettings,
|
|
pub mistral: MistralSettings,
|
|
}
|
|
|
|
#[derive(Default, Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct AllLanguageModelSettingsContent {
|
|
pub anthropic: Option<AnthropicSettingsContent>,
|
|
pub bedrock: Option<AmazonBedrockSettingsContent>,
|
|
pub ollama: Option<OllamaSettingsContent>,
|
|
pub lmstudio: Option<LmStudioSettingsContent>,
|
|
pub openai: Option<OpenAiSettingsContent>,
|
|
pub open_router: Option<OpenRouterSettingsContent>,
|
|
#[serde(rename = "zed.dev")]
|
|
pub zed_dot_dev: Option<ZedDotDevSettingsContent>,
|
|
pub google: Option<GoogleSettingsContent>,
|
|
pub deepseek: Option<DeepseekSettingsContent>,
|
|
pub copilot_chat: Option<CopilotChatSettingsContent>,
|
|
pub mistral: Option<MistralSettingsContent>,
|
|
}
|
|
|
|
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
#[serde(untagged)]
|
|
pub enum AnthropicSettingsContent {
|
|
Versioned(VersionedAnthropicSettingsContent),
|
|
Legacy(LegacyAnthropicSettingsContent),
|
|
}
|
|
|
|
impl AnthropicSettingsContent {
|
|
pub fn upgrade(self) -> (AnthropicSettingsContentV1, bool) {
|
|
match self {
|
|
AnthropicSettingsContent::Legacy(content) => (
|
|
AnthropicSettingsContentV1 {
|
|
api_url: content.api_url,
|
|
available_models: content.available_models.map(|models| {
|
|
models
|
|
.into_iter()
|
|
.filter_map(|model| match model {
|
|
anthropic::Model::Custom {
|
|
name,
|
|
display_name,
|
|
max_tokens,
|
|
tool_override,
|
|
cache_configuration,
|
|
max_output_tokens,
|
|
default_temperature,
|
|
extra_beta_headers,
|
|
mode,
|
|
} => Some(provider::anthropic::AvailableModel {
|
|
name,
|
|
display_name,
|
|
max_tokens,
|
|
tool_override,
|
|
cache_configuration: cache_configuration.as_ref().map(
|
|
|config| LanguageModelCacheConfiguration {
|
|
max_cache_anchors: config.max_cache_anchors,
|
|
should_speculate: config.should_speculate,
|
|
min_total_token: config.min_total_token,
|
|
},
|
|
),
|
|
max_output_tokens,
|
|
default_temperature,
|
|
extra_beta_headers,
|
|
mode: Some(mode.into()),
|
|
}),
|
|
_ => None,
|
|
})
|
|
.collect()
|
|
}),
|
|
},
|
|
true,
|
|
),
|
|
AnthropicSettingsContent::Versioned(content) => match content {
|
|
VersionedAnthropicSettingsContent::V1(content) => (content, false),
|
|
},
|
|
}
|
|
}
|
|
}
|
|
|
|
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct LegacyAnthropicSettingsContent {
|
|
pub api_url: Option<String>,
|
|
pub available_models: Option<Vec<anthropic::Model>>,
|
|
}
|
|
|
|
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
#[serde(tag = "version")]
|
|
pub enum VersionedAnthropicSettingsContent {
|
|
#[serde(rename = "1")]
|
|
V1(AnthropicSettingsContentV1),
|
|
}
|
|
|
|
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct AnthropicSettingsContentV1 {
|
|
pub api_url: Option<String>,
|
|
pub available_models: Option<Vec<provider::anthropic::AvailableModel>>,
|
|
}
|
|
|
|
#[derive(Default, Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct AmazonBedrockSettingsContent {
|
|
available_models: Option<Vec<provider::bedrock::AvailableModel>>,
|
|
endpoint_url: Option<String>,
|
|
region: Option<String>,
|
|
profile: Option<String>,
|
|
authentication_method: Option<provider::bedrock::BedrockAuthMethod>,
|
|
}
|
|
|
|
#[derive(Default, Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct OllamaSettingsContent {
|
|
pub api_url: Option<String>,
|
|
pub available_models: Option<Vec<provider::ollama::AvailableModel>>,
|
|
}
|
|
|
|
#[derive(Default, Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct LmStudioSettingsContent {
|
|
pub api_url: Option<String>,
|
|
pub available_models: Option<Vec<provider::lmstudio::AvailableModel>>,
|
|
}
|
|
|
|
#[derive(Default, Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct DeepseekSettingsContent {
|
|
pub api_url: Option<String>,
|
|
pub available_models: Option<Vec<provider::deepseek::AvailableModel>>,
|
|
}
|
|
|
|
#[derive(Default, Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct MistralSettingsContent {
|
|
pub api_url: Option<String>,
|
|
pub available_models: Option<Vec<provider::mistral::AvailableModel>>,
|
|
}
|
|
|
|
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
#[serde(untagged)]
|
|
pub enum OpenAiSettingsContent {
|
|
Versioned(VersionedOpenAiSettingsContent),
|
|
Legacy(LegacyOpenAiSettingsContent),
|
|
}
|
|
|
|
impl OpenAiSettingsContent {
|
|
pub fn upgrade(self) -> (OpenAiSettingsContentV1, bool) {
|
|
match self {
|
|
OpenAiSettingsContent::Legacy(content) => (
|
|
OpenAiSettingsContentV1 {
|
|
api_url: content.api_url,
|
|
available_models: content.available_models.map(|models| {
|
|
models
|
|
.into_iter()
|
|
.filter_map(|model| match model {
|
|
open_ai::Model::Custom {
|
|
name,
|
|
display_name,
|
|
max_tokens,
|
|
max_output_tokens,
|
|
max_completion_tokens,
|
|
} => Some(provider::open_ai::AvailableModel {
|
|
name,
|
|
max_tokens,
|
|
max_output_tokens,
|
|
display_name,
|
|
max_completion_tokens,
|
|
}),
|
|
_ => None,
|
|
})
|
|
.collect()
|
|
}),
|
|
},
|
|
true,
|
|
),
|
|
OpenAiSettingsContent::Versioned(content) => match content {
|
|
VersionedOpenAiSettingsContent::V1(content) => (content, false),
|
|
},
|
|
}
|
|
}
|
|
}
|
|
|
|
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct LegacyOpenAiSettingsContent {
|
|
pub api_url: Option<String>,
|
|
pub available_models: Option<Vec<open_ai::Model>>,
|
|
}
|
|
|
|
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
#[serde(tag = "version")]
|
|
pub enum VersionedOpenAiSettingsContent {
|
|
#[serde(rename = "1")]
|
|
V1(OpenAiSettingsContentV1),
|
|
}
|
|
|
|
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct OpenAiSettingsContentV1 {
|
|
pub api_url: Option<String>,
|
|
pub available_models: Option<Vec<provider::open_ai::AvailableModel>>,
|
|
}
|
|
|
|
#[derive(Default, Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct GoogleSettingsContent {
|
|
pub api_url: Option<String>,
|
|
pub available_models: Option<Vec<provider::google::AvailableModel>>,
|
|
}
|
|
|
|
#[derive(Default, Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct ZedDotDevSettingsContent {
|
|
available_models: Option<Vec<cloud::AvailableModel>>,
|
|
}
|
|
|
|
#[derive(Default, Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct CopilotChatSettingsContent {}
|
|
|
|
#[derive(Default, Clone, Debug, Serialize, Deserialize, PartialEq, JsonSchema)]
|
|
pub struct OpenRouterSettingsContent {
|
|
pub api_url: Option<String>,
|
|
pub available_models: Option<Vec<provider::open_router::AvailableModel>>,
|
|
}
|
|
|
|
impl settings::Settings for AllLanguageModelSettings {
|
|
const KEY: Option<&'static str> = Some("language_models");
|
|
|
|
const PRESERVED_KEYS: Option<&'static [&'static str]> = Some(&["version"]);
|
|
|
|
type FileContent = AllLanguageModelSettingsContent;
|
|
|
|
fn load(sources: SettingsSources<Self::FileContent>, _: &mut App) -> Result<Self> {
|
|
fn merge<T>(target: &mut T, value: Option<T>) {
|
|
if let Some(value) = value {
|
|
*target = value;
|
|
}
|
|
}
|
|
|
|
let mut settings = AllLanguageModelSettings::default();
|
|
|
|
for value in sources.defaults_and_customizations() {
|
|
// Anthropic
|
|
let (anthropic, upgraded) = match value.anthropic.clone().map(|s| s.upgrade()) {
|
|
Some((content, upgraded)) => (Some(content), upgraded),
|
|
None => (None, false),
|
|
};
|
|
|
|
if upgraded {
|
|
settings.anthropic.needs_setting_migration = true;
|
|
}
|
|
|
|
merge(
|
|
&mut settings.anthropic.api_url,
|
|
anthropic.as_ref().and_then(|s| s.api_url.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.anthropic.available_models,
|
|
anthropic.as_ref().and_then(|s| s.available_models.clone()),
|
|
);
|
|
|
|
// Bedrock
|
|
let bedrock = value.bedrock.clone();
|
|
merge(
|
|
&mut settings.bedrock.profile_name,
|
|
bedrock.as_ref().map(|s| s.profile.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.bedrock.authentication_method,
|
|
bedrock.as_ref().map(|s| s.authentication_method.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.bedrock.region,
|
|
bedrock.as_ref().map(|s| s.region.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.bedrock.endpoint,
|
|
bedrock.as_ref().map(|s| s.endpoint_url.clone()),
|
|
);
|
|
|
|
// Ollama
|
|
let ollama = value.ollama.clone();
|
|
|
|
merge(
|
|
&mut settings.ollama.api_url,
|
|
value.ollama.as_ref().and_then(|s| s.api_url.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.ollama.available_models,
|
|
ollama.as_ref().and_then(|s| s.available_models.clone()),
|
|
);
|
|
|
|
// LM Studio
|
|
let lmstudio = value.lmstudio.clone();
|
|
|
|
merge(
|
|
&mut settings.lmstudio.api_url,
|
|
value.lmstudio.as_ref().and_then(|s| s.api_url.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.lmstudio.available_models,
|
|
lmstudio.as_ref().and_then(|s| s.available_models.clone()),
|
|
);
|
|
|
|
// DeepSeek
|
|
let deepseek = value.deepseek.clone();
|
|
|
|
merge(
|
|
&mut settings.deepseek.api_url,
|
|
value.deepseek.as_ref().and_then(|s| s.api_url.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.deepseek.available_models,
|
|
deepseek.as_ref().and_then(|s| s.available_models.clone()),
|
|
);
|
|
|
|
// OpenAI
|
|
let (openai, upgraded) = match value.openai.clone().map(|s| s.upgrade()) {
|
|
Some((content, upgraded)) => (Some(content), upgraded),
|
|
None => (None, false),
|
|
};
|
|
|
|
if upgraded {
|
|
settings.openai.needs_setting_migration = true;
|
|
}
|
|
|
|
merge(
|
|
&mut settings.openai.api_url,
|
|
openai.as_ref().and_then(|s| s.api_url.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.openai.available_models,
|
|
openai.as_ref().and_then(|s| s.available_models.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.zed_dot_dev.available_models,
|
|
value
|
|
.zed_dot_dev
|
|
.as_ref()
|
|
.and_then(|s| s.available_models.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.google.api_url,
|
|
value.google.as_ref().and_then(|s| s.api_url.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.google.available_models,
|
|
value
|
|
.google
|
|
.as_ref()
|
|
.and_then(|s| s.available_models.clone()),
|
|
);
|
|
|
|
// Mistral
|
|
let mistral = value.mistral.clone();
|
|
merge(
|
|
&mut settings.mistral.api_url,
|
|
mistral.as_ref().and_then(|s| s.api_url.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.mistral.available_models,
|
|
mistral.as_ref().and_then(|s| s.available_models.clone()),
|
|
);
|
|
|
|
// OpenRouter
|
|
let open_router = value.open_router.clone();
|
|
merge(
|
|
&mut settings.open_router.api_url,
|
|
open_router.as_ref().and_then(|s| s.api_url.clone()),
|
|
);
|
|
merge(
|
|
&mut settings.open_router.available_models,
|
|
open_router
|
|
.as_ref()
|
|
.and_then(|s| s.available_models.clone()),
|
|
);
|
|
}
|
|
|
|
Ok(settings)
|
|
}
|
|
|
|
fn import_from_vscode(_vscode: &settings::VsCodeSettings, _current: &mut Self::FileContent) {}
|
|
}
|