Add configurable low-speed timeout for OpenAI provider (#11668)

This PR adds a setting to allow configuring the low-speed timeout for
the Assistant when using the OpenAI provider.

The `low_speed_timeout_in_seconds` accepts a number of seconds that the
HTTP client can go below a minimum speed limit (currently set to 100
bytes/second) before it times out.

```json
{
  "assistant": {
    "version": "1",
    "provider": { "name": "openai", "low_speed_timeout_in_seconds": 60 }
  },
}
```

This should help the case where the `openai` provider is being used with
a local model that requires higher timeouts.

Issue: https://github.com/zed-industries/zed/issues/9913

Release Notes:

- Added a `low_speed_timeout_in_seconds` setting to the Assistant's
OpenAI provider
([#9913](https://github.com/zed-industries/zed/issues/9913)).
This commit is contained in:
Marshall Bowers 2024-05-10 13:19:21 -04:00 committed by GitHub
parent 19994fc190
commit 0d26beb91b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 59 additions and 9 deletions

View file

@ -4344,6 +4344,7 @@ async fn complete_with_open_ai(
OPEN_AI_API_URL,
&api_key,
crate::ai::language_model_request_to_open_ai(request)?,
None,
)
.await
.context("open_ai::stream_completion request failed within collab")?;