Yehowshua/ZIm
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Thread Anthropic errors into LanguageModelKnownError (#33261)

Browse source 
This PR is in preparation for doing automatic retries for certain
errors, e.g. Overloaded. It doesn't change behavior yet (aside from some
granularity of error messages shown to the user), but rather mostly
changes some error handling to be exhaustive enum matches instead of
`anyhow` downcasts, and leaves some comments for where the behavior
change will be in a future PR.

Release Notes:

- N/A
This commit is contained in:
Richard Feldman 2025-06-23 14:48:26 -04:00 • committed by GitHub
parent aabfea4c10
commit c610ebfb03
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 283 additions and 123 deletions
Download patch file Download diff file Expand all files Collapse all files

8
crates/assistant_tools/src/edit_agent/evals.rs
View file

@ -1659,13 +1659,13 @@ async fn retry_on_rate_limit<R>(mut request: impl AsyncFnMut() -> Result<R>) ->
Ok(result) => return Ok(result),
Err(err) => match err.downcast::<LanguageModelCompletionError>() {
Ok(err) => match err {
LanguageModelCompletionError::RateLimit(duration) => {
LanguageModelCompletionError::RateLimitExceeded { retry_after } => {
// Wait for the duration supplied, with some jitter to avoid all requests being made at the same time.
let jitter = duration.mul_f64(rand::thread_rng().gen_range(0.0..1.0));
let jitter = retry_after.mul_f64(rand::thread_rng().gen_range(0.0..1.0));
eprintln!(
"Attempt #{attempt}: Rate limit exceeded. Retry after {duration:?} + jitter of {jitter:?}"
"Attempt #{attempt}: Rate limit exceeded. Retry after {retry_after:?} + jitter of {jitter:?}"
);
Timer::after(duration + jitter).await;
Timer::after(retry_after + jitter).await;
continue;
}
_ => return Err(err.into()),