Introduce a separate backend service for LLM calls (#15831)

This PR introduces a separate backend service for making LLM calls.

It exposes an HTTP interface that can be called by Zed clients. To call
these endpoints, the client must provide a `Bearer` token. These tokens
are issued/refreshed by the collab service over RPC.

We're adding this in a backwards-compatible way. Right now the access
tokens can only be minted for Zed staff, and calling this separate LLM
service is behind the `llm-service` feature flag (which is not
automatically enabled for Zed staff).

Release Notes:

- N/A

---------

Co-authored-by: Marshall <marshall@zed.dev>
Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
This commit is contained in:
Max Brunsfeld 2024-08-05 17:26:21 -07:00 committed by GitHub
parent 4ed43e6e6f
commit 8e9c2b1125
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
20 changed files with 478 additions and 102 deletions

2
Cargo.lock generated
View file

@ -2466,6 +2466,7 @@ dependencies = [
"hex",
"http_client",
"indoc",
"jsonwebtoken",
"language",
"language_model",
"live_kit_client",
@ -2507,6 +2508,7 @@ dependencies = [
"telemetry_events",
"text",
"theme",
"thiserror",
"time",
"tokio",
"toml 0.8.16",