More resilient eval (#32257)
Bubbles up rate limit information so that we can retry after a certain duration if needed higher up in the stack. Also caps the number of concurrent evals running at once to also help. Release Notes: - N/A
This commit is contained in:
parent
fa54fa80d0
commit
e4bd115a63
22 changed files with 147 additions and 56 deletions
|
@ -1563,6 +1563,9 @@ impl Thread {
|
|||
Err(LanguageModelCompletionError::Other(error)) => {
|
||||
return Err(error);
|
||||
}
|
||||
Err(err @ LanguageModelCompletionError::RateLimit(..)) => {
|
||||
return Err(err.into());
|
||||
}
|
||||
};
|
||||
|
||||
match event {
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue