Forum

> > CS2D > Scripts > Computational error in Lua
Forums overviewCS2D overview Scripts overviewLog in to reply

English Computational error in Lua

11 replies
To the start Previous 1 Next To the start

old Computational error in Lua

Lee
Moderator Off Offline

Quote
I had a friend who asked me to help him figure out what IMG:https://mathbin.net/equation_previews/83005_0.png
evaluates to. This is a pretty simple problem for a computer program. All we need to do is set x to some really small number and see what it prints out

1
2
3
4
> x = 10^-10
> y = x
> =(cos(x*y)-1)/(x*y*x*y)
0

Looks like it's a pretty simple answer.

Now let's be a bit more pedantic and check with wolframalpha

http://www.wolframalpha.com/input/?i=%5Clim_%7Bx+%5Crightarrow+0%7D%5Cfrac%7Bcos%28x%5E2%29-1%7D%7Bx%5E4%7D

-1/2? What the hell, Lua just told me that the answer is 0. What went wrong?

Here's your problem: Try to explain why

1
(-sin(x*y)^2)/(x*y*x*y*(cos(x*y)+1))

gives the correct answer of -0.5

but

1
(cos(x*y)-1)/(x*y*x*y)

does not.


Hint: every input and computation is associated with a relative error delta that is less than some constant epsilon that we can call the machine epsilon = O(2^-52), so try to figure out why subtraction amplifies this relative error per computation

These types of truncation errors occur often in numerical computation problems (especially within large linear systems) and are often referred to as cancellation error.

old Re: Computational error in Lua

Flacko
User Off Offline

Quote
cos(x*y) returns 1 when it shouldn't.
Even when wolfram alpha tried to quick-solve it in my PC it returned 1, what an ironic failure

http://www.wolframalpha.com/input/?i=cos%28%2810^-10%29*%2810^-10%29%29

Taking a look at the approximate result:
Quote
0.9999999999999999999999999999999999999999500000000000000000000000000000000000000004166...

I've bolded the maximum number of digits that the significand can take which is about:
1
log10(2^52) = 15,65...
since the significand can't take that many digits, it's rounded to 1 thx to the adjacent 9

The big problem is that substracting 1 from the cos in the numerator makes it equal to 0 (it cancels it), fucking up the entire division.

Apart from that, in the alternate function
1
sin(x*y)
returns 1*10^-20 which isn't correct either, yet the numerator remains... not so fucked up because there wasn't any substraction that would cancel it.

old Re: Computational error in Lua

ohaz
User Off Offline

Quote
When I run
1
lim x y = (cos(x*y)-1) / (x^2 * y^2)
in Haskell, it prints out the correct Value of -0.5 when using Values like (0.5,0.5) or (0.2, 0.2)
IMG:https://endless-aerospace.de/img/haskellLimes.png

So, when you use a language that is designed for calculating, it calculates correctly.

old Re: Computational error in Lua

Lee
Moderator Off Offline

Quote
Haskell uses IEEE 64 bit floating point numbers as well, just like lua. xy = O(10^-3) gives an error amplification of O(n^9) on an existing machine epsilon on the order of 2^-52, the error introduced is extremely small.

Lua gives me -0.49999999696126 as well with xy = 0.0001.

old Re: Computational error in Lua

Flacko
User Off Offline

Quote
user Apache uwu has written
so is this lua's fault?

It will happen on most programming languages. If it's Lua's fault it's very probably C's fault too.

If there was a 256-bit floating point datatype, this wouldn't happen in this case.
Although you can never have enough bits, you can't have infinite bits either.

Blame computers.

old Re: Computational error in Lua

EP
User Off Offline

Quote
Check this one in lua, the tangent of 90° (its infinite or no solution but the lua gives another solution :o)
To the start Previous 1 Next To the start
Log in to reply Scripts overviewCS2D overviewForums overview