Jump to content

precision of the result


kalai

Recommended Posts

(setq L1 1000
       L2 1000
       L3 1000
       L4 1000
       L5 1000
)
(setq scount 10)
(setq L6 120
        H 6)
(setq sdist (/ (- (+ L1 L2 L3 L4 L5) (+ L6 (* 2 H))) (- scount 2)))

Actually the value of scount =608.5

but it is rounded to 608

How am i to get actual value?

Link to comment
Share on other sites

(setq sdist (/ (float (- (+ L1 L2 L3 L4 L5) (+ L6 (* 2 H)))) (- scount 2)))

 

if you divide a whole number by a whole number you will get ====> YES a whole number

Link to comment
Share on other sites

Set the System Variable dimzin to 0 and try again .

 

DIMZIN only affects (rtos) & (angtos) calls. Not the actual calculation. (float) is the correct way to fix a division of integer problem. -David

Link to comment
Share on other sites

DIMZIN only affects (rtos) & (angtos) calls. Not the actual calculation. (float) is the correct way to fix a division of integer problem. -David

 

Thank you David for the clarifications .

 

Appreciated a lot.

 

Tharwat

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...