Jump to content

Division, but not quite.


Freerefill

Recommended Posts

So I was trying to code a base converter (you know, like base 10 to binary, that sort of deal) in LISP, and for some reason, what I was getting back wasn't working even though my math was spot on. I struggled with it for quite literally hours before stumbling upon the problem.

 

Now I don't know if this will work the same on every version, but try this out. Paste this in the command line

 

(/ 51.0 10.0)

Returns 5.1, right? Well yeah, that's what you'd think it would do.

 

Now try this:

(* 10 (rem (/ 51.0 10.0) 1))

For anyone not following me, I'm taking the remainder and multiplying it by 10. Returns 1.0, right? Well now, lets test that.

 

(fix (* 10 (rem (/ 51.0 10.0) 1)))

"fix" removes the remainder from a floating point number and returns the integer portion. So waitatic, why is this returning 0? Isn't the integer portion of 1.0 1? Go ahead and try it:

 

(fix 1.0)

So it seems yes, it is. So then why is it returning an incorrect number? There's only one way to peer deeper into a number, and that's by turning it into a string, and forcing it to display the extra digits.

 

(rtos (* 10 (rem (/ 51.0 10.0) 1)) 2 16)

"rtos" turns a real number into a string. As you can see, 0.9999999999999964 is not equal to 1.0. Close, but no cigar.

 

Is this a problem with AutoCADs division operator? I know computers only work a certain way, but this has caused me hours of grief. I could understand if the numbers were very large, how significant digits would definitely play a part, but I think 51 and 10 are relatively small, and should not be causing an error like this.

Link to comment
Share on other sites

I see there is some error that crops up in the very small decimals; happens with computers/computing.

 

What are you trying to determine? If its' a decimal portion of a real, there are better ways than with 'rem', which is more suited to division of integers.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...