ZIm/crates/eval
Nathan Sobo 107d8ca483
Rename regex search tool to grep and accept an include glob pattern (#29100)
This PR renames the `regex_search` tool to `grep` because I think it
conveys more meaning to the model, the idea of searching the filesystem
with a regular expression. It's also one word and the model seems to be
using it effectively after some additional prompt tuning.

It also takes an include pattern to filter on the specific files we try
to search. I'd like to encourage the model to scope its searches more
aggressively, as in my testing, I'm only seeing it filter on file
extension.

Release Notes:

- N/A
2025-04-20 00:53:30 +00:00
..
examples Rename regex search tool to grep and accept an include glob pattern (#29100) 2025-04-20 00:53:30 +00:00
src Rename regex search tool to grep and accept an include glob pattern (#29100) 2025-04-20 00:53:30 +00:00
.gitignore Add judge to new eval + provide LSP diagnostics (#28713) 2025-04-14 20:18:47 +00:00
Cargo.toml Pretty tool inputs in eval output markdown + numbered assistant messages (#29082) 2025-04-19 06:59:22 +00:00
LICENSE-GPL Lay the groundwork for a Rust-based eval (#28488) 2025-04-10 04:45:27 +00:00
README.md Lay the groundwork for a Rust-based eval (#28488) 2025-04-10 04:45:27 +00:00
runner_settings.json eval: Fix stalling on tool confirmation (#28786) 2025-04-15 16:53:45 +00:00

Eval

This eval assumes the working directory is the root of the repository. Run it with:

cargo run -p eval