ab017129d8
This change improves `eval_extract_handle_command_output` results for all models: Model | Pass rate before | Pass rate after ----------------------------|------------------|---------------- claude-3.7-sonnet | 0.96 | 0.98 gemini-2.5-pro | 0.35 | 0.86 gpt-4.1 | 0.81 | 1.00 Part of this improvement comes from more robust evaluation, which now accepts multiple possible outcomes. Another part is from the prompt adaptation: addressing common Gemini failure modes, adding a few-shot example, and, in the final commit, auto-rewriting instructions for clarity and conciseness. This change still needs validation from larger end-to-end evals. Release Notes: - N/A |
||
|---|---|---|
| .. | ||
| src | ||
| Cargo.toml | ||
| LICENSE-GPL | ||