copilot: Fix rate limit due to Copilot-Vision-Request header (#30989)

Issues: #30994

I've implemented an important optimisation in response to GitHub
Copilot's recent rate limit on concurrent Vision API calls. Previously,
our system was defaulting to vision header: true for all API calls. To
prevent unnecessary calls and adhere to the new limits, I've updated our
logic: the vision header is now only sent if the current message is a
vision message, specifically when the preceding message includes an
image.

Prompt used to reproduce and verify the fix: `Give me a context for my
agent crate about. Browse my repo.`

Release Notes:

- copilot: Set Copilot-Vision-Request header based on message content
This commit is contained in:
Umesh Yadav 2025-05-21 22:21:35 +05:30 committed by GitHub
parent bdd9e015ab
commit f915c24279
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -581,6 +581,15 @@ async fn stream_completion(
api_key: String,
request: Request,
) -> Result<BoxStream<'static, Result<ResponseEvent>>> {
let is_vision_request = request.messages.last().map_or(false, |message| match message {
ChatMessage::User { content }
| ChatMessage::Assistant { content, .. }
| ChatMessage::Tool { content, .. } => {
matches!(content, ChatMessageContent::Multipart(parts) if parts.iter().any(|part| matches!(part, ChatMessagePart::Image { .. })))
}
_ => false,
});
let request_builder = HttpRequest::builder()
.method(Method::POST)
.uri(COPILOT_CHAT_COMPLETION_URL)
@ -594,7 +603,7 @@ async fn stream_completion(
.header("Authorization", format!("Bearer {}", api_key))
.header("Content-Type", "application/json")
.header("Copilot-Integration-Id", "vscode-chat")
.header("Copilot-Vision-Request", "true");
.header("Copilot-Vision-Request", is_vision_request.to_string());
let is_streaming = request.stream;