ZIm/crates/eval/src
Oleksiy Syvokon 6df4c537b9
agent: Less disruptive changed file notification (#31693)
When the user edits one of the tracked files, we used to notify the
agent by inserting a user message at the end of the thread. This was
causing a few problems:
- The agent would stop doing its work and start reading changed files
- The agent would write something like, "Thank you for letting me know
about these changed files."

This fix contains two parts:
1. Changing the prompt to indicate this is a service message
2. Moving the message higher in the conversation thread

This works, but it slightly hurts caching.

We may consider making these notification messages stick in history,
trading context tokens count for the cache.

This might be related to #30906

Release Notes:

- N/A

---------

Co-authored-by: Marshall Bowers <git@maxdeviant.com>
2025-06-16 18:45:24 +03:00
..
examples agent: Less disruptive changed file notification (#31693) 2025-06-16 18:45:24 +03:00
assertions.rs eval: Count execution errors as failures (#30712) 2025-05-14 20:44:19 +03:00
eval.rs Replace async-watch with a custom watch (#32245) 2025-06-06 16:00:09 +00:00
example.rs Store profile per thread (#31907) 2025-06-06 12:05:27 +00:00
explorer.html eval: Add HTML overview for evaluation runs (#29413) 2025-04-25 17:49:05 +03:00
explorer.rs evals: Allow threads explorer to search for JSON files recursively (#31509) 2025-05-27 14:18:47 +00:00
ids.rs Use anyhow more idiomatically (#31052) 2025-05-20 23:06:07 +00:00
instance.rs agent: Less disruptive changed file notification (#31693) 2025-06-16 18:45:24 +03:00
judge_diff_prompt.hbs eval: Fine-grained assertions (#29246) 2025-04-22 23:58:58 -03:00
judge_thread_prompt.hbs eval: Fine-grained assertions (#29246) 2025-04-22 23:58:58 -03:00
tool_metrics.rs eval: Fine-grained assertions (#29246) 2025-04-22 23:58:58 -03:00