Freerefill Posted October 23, 2009 Share Posted October 23, 2009 So I was trying to code a base converter (you know, like base 10 to binary, that sort of deal) in LISP, and for some reason, what I was getting back wasn't working even though my math was spot on. I struggled with it for quite literally hours before stumbling upon the problem. Now I don't know if this will work the same on every version, but try this out. Paste this in the command line (/ 51.0 10.0) Returns 5.1, right? Well yeah, that's what you'd think it would do. Now try this: (* 10 (rem (/ 51.0 10.0) 1)) For anyone not following me, I'm taking the remainder and multiplying it by 10. Returns 1.0, right? Well now, lets test that. (fix (* 10 (rem (/ 51.0 10.0) 1))) "fix" removes the remainder from a floating point number and returns the integer portion. So waitatic, why is this returning 0? Isn't the integer portion of 1.0 1? Go ahead and try it: (fix 1.0) So it seems yes, it is. So then why is it returning an incorrect number? There's only one way to peer deeper into a number, and that's by turning it into a string, and forcing it to display the extra digits. (rtos (* 10 (rem (/ 51.0 10.0) 1)) 2 16) "rtos" turns a real number into a string. As you can see, 0.9999999999999964 is not equal to 1.0. Close, but no cigar. Is this a problem with AutoCADs division operator? I know computers only work a certain way, but this has caused me hours of grief. I could understand if the numbers were very large, how significant digits would definitely play a part, but I think 51 and 10 are relatively small, and should not be causing an error like this. Quote Link to comment Share on other sites More sharing options...
CarlB Posted October 23, 2009 Share Posted October 23, 2009 I see there is some error that crops up in the very small decimals; happens with computers/computing. What are you trying to determine? If its' a decimal portion of a real, there are better ways than with 'rem', which is more suited to division of integers. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.