ZIm/crates/eval
Piotr Osiewicz 77dadfedfe
chore: Make terminal_view own the TerminalSlashCommand (#31070)
This reduces 'touch crates/editor/src/editor.rs && cargo +nightly build'
from 8.9s to 8.5s. That same scenario used to take 8s less than a week
ago. :)
I'm measuring with nightly rustc, because it's compile times are better
than those of stable thanks to
https://github.com/rust-lang/rust/pull/138522

main (8.2s total):

![image](https://github.com/user-attachments/assets/767a2ac4-7bba-4147-bd16-9b09eed5b433)

[cargo-timing.html.zip](https://github.com/user-attachments/files/20364175/cargo-timing.html.zip)

#22be776 (7.5s total):

[cargo-timing-20250521T085303.892834Z.html.zip](https://github.com/user-attachments/files/20364391/cargo-timing-20250521T085303.892834Z.html.zip)

![image](https://github.com/user-attachments/assets/c4476df9-cb6e-4403-b0db-de00521f1fd0)


Release Notes:

- N/A
2025-05-21 09:27:54 +00:00
..
docs eval: Add HTML overview for evaluation runs (#29413) 2025-04-25 17:49:05 +03:00
src chore: Make terminal_view own the TerminalSlashCommand (#31070) 2025-05-21 09:27:54 +00:00
.gitignore Add judge to new eval + provide LSP diagnostics (#28713) 2025-04-14 20:18:47 +00:00
Cargo.toml chore: Make terminal_view own the TerminalSlashCommand (#31070) 2025-05-21 09:27:54 +00:00
LICENSE-GPL Lay the groundwork for a Rust-based eval (#28488) 2025-04-10 04:45:27 +00:00
README.md eval: Add support for reading from a .env file (#29426) 2025-04-25 15:53:02 +00:00
runner_settings.json Introduce a new StreamingEditFileTool (#29733) 2025-05-01 17:37:43 +02:00

Eval

This eval assumes the working directory is the root of the repository. Run it with:

cargo run -p eval

The eval will optionally read a .env file in crates/eval if you need it to set environment variables, such as API keys.

Explorer Tool

The explorer tool generates a self-contained HTML view from one or more thread JSON file. It provides a visual interface to explore the agent thread, including tool calls and results. See ./docs/explorer.md for more details.

Usage

cargo run -p eval --bin explorer -- --input <path-to-json-files> --output <output-html-path>

Example:

cargo run -p eval --bin explorer -- --input ./runs/2025-04-23_15-53-30/fastmcp_bugifx/*/last.messages.json --output /tmp/explorer.html