Allow Anthropic custom models to override temperature (#18160)

Release Notes:

- Allow Anthropic custom models to override "temperature"

This also centralized the defaulting of "temperature" to be inside of
each model's `into_x` call instead of being sprinkled around the code.
This commit is contained in:
Roy Williams 2024-09-20 16:59:12 -04:00 committed by GitHub
parent 7d62fda5a3
commit 5905fbb9ac
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 54 additions and 17 deletions

View file

@ -235,7 +235,7 @@ impl OllamaLanguageModel {
options: Some(ChatOptions {
num_ctx: Some(self.model.max_tokens),
stop: Some(request.stop),
temperature: Some(request.temperature),
temperature: request.temperature.or(Some(1.0)),
..Default::default()
}),
tools: vec![],