ZIm/crates/eval/examples/optimizer_schema_refactor/criteria.md
Thomas Mickley-Doyle d74f0735c2
Add more eval examples + filtering examples by language + fix git concurrent usage (#28719)
Release Notes:

- N/A

---------

Co-authored-by: michael <michael@zed.dev>
Co-authored-by: agus <agus@zed.dev>
2025-04-14 22:05:46 +00:00

1.3 KiB

  1. The changes involve renaming the TestData class to LabeledData across multiple files. This includes updating the import statements in __init__.py, cache.py, router.py, schema.py, and utils.py to reflect this new class name. The __all__ list in __init__.py is also updated to export LabeledData instead of TestData. This appears to be a conceptual renaming to better reflect the purpose of the data structure.
  2. The modifications update all function signatures and type hints that previously used TestData to now use LabeledData. This affects several functions in cache.py including _generate_run_cache, _eval_cache, and _grid_search_opt_cache, as well as functions in router.py like _generate_run_router and _eval_router. The utility functions in utils.py are also updated to work with LabeledData instead of TestData.
  3. The changes introduce a new search_step parameter in the router optimization logic within router.py, with a default value of 0.10. This parameter is passed through to the _router_random_search function and is used in the optimization process. The test file test_threshold_optimizer.py is updated to explicitly set this parameter to 0.5 when calling the optimize method, demonstrating how it can be configured for different search granularities during threshold optimization.