Implement Anthropic prompt caching (#16274)
Release Notes: - Adds support for Prompt Caching in Anthropic. For models that support it this can dramatically lower cost while improving performance.
This commit is contained in:
parent
09b6e3f2a6
commit
46fb917e02
11 changed files with 338 additions and 70 deletions
|
@ -277,6 +277,7 @@ impl TerminalInlineAssistant {
|
|||
messages.push(LanguageModelRequestMessage {
|
||||
role: Role::User,
|
||||
content: vec![prompt.into()],
|
||||
cache: false,
|
||||
});
|
||||
|
||||
Ok(LanguageModelRequest {
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue