00fd045844
This expands our deserialization of JSON from models to be more tolerant of different variations that the model may send, including capitalization, wrapping things in objects vs. being plain strings, etc. Also when deserialization fails, it reports the entire error in the JSON so we can see what failed to deserialize. (Previously these errors were very unhelpful at diagnosing the problem.) Finally, also removes the `WrappedText` variant since the custom deserializer just turns that style of JSON into a normal `Text` variant. Release Notes: - N/A |
||
|---|---|---|
| .. | ||
| docs | ||
| src | ||
| .gitignore | ||
| Cargo.toml | ||
| LICENSE-GPL | ||
| README.md | ||
| runner_settings.json | ||
Eval
This eval assumes the working directory is the root of the repository. Run it with:
cargo run -p eval
The eval will optionally read a .env file in crates/eval if you need it to set environment variables, such as API keys.
Explorer Tool
The explorer tool generates a self-contained HTML view from one or more thread JSON file. It provides a visual interface to explore the agent thread, including tool calls and results. See ./docs/explorer.md for more details.
Usage
cargo run -p eval --bin explorer -- --input <path-to-json-files> --output <output-html-path>
Example:
cargo run -p eval --bin explorer -- --input ./runs/2025-04-23_15-53-30/fastmcp_bugifx/*/last.messages.json --output /tmp/explorer.html