Introduce a separate backend service for LLM calls (#15831)

This PR introduces a separate backend service for making LLM calls.

It exposes an HTTP interface that can be called by Zed clients. To call
these endpoints, the client must provide a `Bearer` token. These tokens
are issued/refreshed by the collab service over RPC.

We're adding this in a backwards-compatible way. Right now the access
tokens can only be minted for Zed staff, and calling this separate LLM
service is behind the `llm-service` feature flag (which is not
automatically enabled for Zed staff).

Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
This commit is contained in:
Max Brunsfeld 2024-08-05 17:26:21 -07:00 committed by GitHub
parent 4ed43e6e6f
commit 8e9c2b1125
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
20 changed files with 478 additions and 102 deletions

View file

@ -175,6 +175,22 @@ impl HttpClientWithUrl {
query,
)?)
}
/// Builds a Zed LLM URL using the given path.
pub fn build_zed_llm_url(&self, path: &str, query: &[(&str, &str)]) -> Result<Url> {
let base_url = self.base_url();
let base_api_url = match base_url.as_ref() {
"https://zed.dev" => "https://llm.zed.dev",
"https://staging.zed.dev" => "https://llm-staging.zed.dev",
"http://localhost:3000" => "http://localhost:8080",
other => other,
};
Ok(Url::parse_with_params(
&format!("{}{}", base_api_url, path),
query,
)?)
}
}
impl HttpClient for Arc<HttpClientWithUrl> {