/auto (#16696)
Add `/auto` behind a feature flag that's disabled for now, even for staff. We've decided on a different design for context inference, but there are parts of /auto that will be useful for that, so we want them in the code base even if they're unused for now. Release Notes: - N/A --------- Co-authored-by: Antonio Scandurra <me@as-cii.com> Co-authored-by: Marshall Bowers <elliott.codes@gmail.com>
This commit is contained in:
parent
93a3e8bc94
commit
91ffa02e2c
42 changed files with 2776 additions and 1054 deletions
|
@ -221,6 +221,10 @@ impl HttpClient for HttpClientWithUrl {
|
|||
|
||||
pub fn client(user_agent: Option<String>, proxy: Option<Uri>) -> Arc<dyn HttpClient> {
|
||||
let mut builder = isahc::HttpClient::builder()
|
||||
// Some requests to Qwen2 models on Runpod can take 32+ seconds,
|
||||
// especially if there's a cold boot involved. We may need to have
|
||||
// those requests use a different http client, because global timeouts
|
||||
// of 50 and 60 seconds, respectively, would be very high!
|
||||
.connect_timeout(Duration::from_secs(5))
|
||||
.low_speed_timeout(100, Duration::from_secs(5))
|
||||
.proxy(proxy.clone());
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue