Adjust erf estimation function (#15423)

Release Notes:

- Fixed a (potential) small error in erf estimation. Technically, the
error is negligible.

I am not sure where the current calculation for erf come from and if it
is intended or a simple mistake. However it looks slightly different
from the official calculation, notably
[this](https://en.wikipedia.org/wiki/Error_function#Approximation_with_elementary_functions)
from Wikipedia.

I will add a comment if it is intended.
This commit is contained in:
Son 2024-08-07 22:12:26 +10:00 committed by GitHub
parent 2db2b636f2
commit f24f601e05
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 37 additions and 4 deletions

View file

@ -150,7 +150,7 @@ fn gaussian(x: f32, sigma: f32) -> f32{
fn erf(v: vec2<f32>) -> vec2<f32> {
let s = sign(v);
let a = abs(v);
let r1 = 1.0 + (0.278393 + (0.230389 + 0.078108 * (a * a)) * a) * a;
let r1 = 1.0 + (0.278393 + (0.230389 + (0.000972 + 0.078108 * a) * a) * a) * a;
let r2 = r1 * r1;
return s - s / (r2 * r2);
}