Freerefill
23rd Oct 2009, 08:49 pm
So I was trying to code a base converter (you know, like base 10 to binary, that sort of deal) in LISP, and for some reason, what I was getting back wasn't working even though my math was spot on. I struggled with it for quite literally hours before stumbling upon the problem.
Now I don't know if this will work the same on every version, but try this out. Paste this in the command line
(/ 51.0 10.0)
Returns 5.1, right? Well yeah, that's what you'd think it would do.
Now try this:
(* 10 (rem (/ 51.0 10.0) 1))
For anyone not following me, I'm taking the remainder and multiplying it by 10. Returns 1.0, right? Well now, lets test that.
(fix (* 10 (rem (/ 51.0 10.0) 1)))
"fix" removes the remainder from a floating point number and returns the integer portion. So waitatic, why is this returning 0? Isn't the integer portion of 1.0 1? Go ahead and try it:
(fix 1.0)
So it seems yes, it is. So then why is it returning an incorrect number? There's only one way to peer deeper into a number, and that's by turning it into a string, and forcing it to display the extra digits.
(rtos (* 10 (rem (/ 51.0 10.0) 1)) 2 16)
"rtos" turns a real number into a string. As you can see, 0.9999999999999964 is not equal to 1.0. Close, but no cigar.
Is this a problem with AutoCADs division operator? I know computers only work a certain way, but this has caused me hours of grief. I could understand if the numbers were very large, how significant digits would definitely play a part, but I think 51 and 10 are relatively small, and should not be causing an error like this.
Now I don't know if this will work the same on every version, but try this out. Paste this in the command line
(/ 51.0 10.0)
Returns 5.1, right? Well yeah, that's what you'd think it would do.
Now try this:
(* 10 (rem (/ 51.0 10.0) 1))
For anyone not following me, I'm taking the remainder and multiplying it by 10. Returns 1.0, right? Well now, lets test that.
(fix (* 10 (rem (/ 51.0 10.0) 1)))
"fix" removes the remainder from a floating point number and returns the integer portion. So waitatic, why is this returning 0? Isn't the integer portion of 1.0 1? Go ahead and try it:
(fix 1.0)
So it seems yes, it is. So then why is it returning an incorrect number? There's only one way to peer deeper into a number, and that's by turning it into a string, and forcing it to display the extra digits.
(rtos (* 10 (rem (/ 51.0 10.0) 1)) 2 16)
"rtos" turns a real number into a string. As you can see, 0.9999999999999964 is not equal to 1.0. Close, but no cigar.
Is this a problem with AutoCADs division operator? I know computers only work a certain way, but this has caused me hours of grief. I could understand if the numbers were very large, how significant digits would definitely play a part, but I think 51 and 10 are relatively small, and should not be causing an error like this.