Weird ass excel error

Posted by: lectric

Weird ass excel error - 22/04/2016 21:02

Hey guys. I have an odd question that I just KNOW you can help with.

I cannot for the life of me figure out why the if statement in D20 is not evaluating correctly.

It's not a big deal, but I am really curious where my logic flaw is.

Many thanks to anyone interested in this puzzle!

Lec
Posted by: larry818

Re: Weird ass excel error - 22/04/2016 22:23

It's evaluating correctly, just not calculating correctly.

I believe everything is a floating point calc to excel, so errors creep in. See the attached. I changed nothing but the format of the cell.
Posted by: tfabris

Re: Weird ass excel error - 23/04/2016 00:54

Brilliant catch!

You can also set the configuration setting in excel to "Precision As Displayed", as described in this KB article:

https://support.microsoft.com/en-us/kb/78113

This calls to mind the old days of writing programs in BASIC to do things like printing simple invoices on a dot matrix printer. Every once in a while you'd get an invoice with a total that looked something like 1E+7 instead of a useful price. smile
Posted by: lectric

Re: Weird ass excel error - 23/04/2016 01:37

And you guys are awesome... Thank you! I was thinking it was a calculation error, but couldn't figure out why. Interesting.