5405c2c2d3
Previously we were using a mix of `u32` and `usize`, e.g. `max_tokens: usize, max_output_tokens: Option<u32>` in the same `struct`. Although [tiktoken](https://github.com/openai/tiktoken) uses `usize`, token counts should be consistent across targets (e.g. the same model doesn't suddenly get a smaller context window if you're compiling for wasm32), and these token counts could end up getting serialized using a binary protocol, so `usize` is not the right choice for token counts. I chose to standardize on `u64` over `u32` because we don't store many of them (so the extra size should be insignificant) and future models may exceed `u32::MAX` tokens. Release Notes: - N/A |
||
|---|---|---|
| .. | ||
| model | ||
| fake_provider.rs | ||
| language_model.rs | ||
| rate_limiter.rs | ||
| registry.rs | ||
| request.rs | ||
| role.rs | ||
| telemetry.rs | ||