agent: Fix conversation token usage and estimate unsent message (#28878)
The UI was mistakenly using the cumulative token usage for the token counter. It will now display the last request token count, plus an estimation of the tokens in the message editor and context entries that haven't been sent yet. https://github.com/user-attachments/assets/0438c501-b850-4397-9135-57214ca3c07a Additionally, when the user edits a message, we'll display the actual token count up to it and estimate the tokens in the new message. Note: We don't currently estimate the delta when switching profiles. In the future, we want to use the count tokens API to measure every part of the request and display a breakdown. Release Notes: - agent: Made the token count more accurate and added back estimation of used tokens as you type and add context. --------- Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de> Co-authored-by: Danilo Leal <daniloleal09@gmail.com>
This commit is contained in:
parent
8de53bd89f
commit
0286b8ab3e
8 changed files with 507 additions and 130 deletions
|
@ -97,7 +97,10 @@ pub struct TokenUsage {
|
|||
|
||||
impl TokenUsage {
|
||||
pub fn total_tokens(&self) -> u32 {
|
||||
self.input_tokens + self.output_tokens
|
||||
self.input_tokens
|
||||
+ self.output_tokens
|
||||
+ self.cache_read_input_tokens
|
||||
+ self.cache_creation_input_tokens
|
||||
}
|
||||
}
|
||||
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue