Round a REAL number with rtos

[capocad]

Hi!

I have a question related to rtos and rounding real numbers.

If i do the following in autolisp:

(setq number 0.925)
(setq roundedNumber (rtos number 2 2))

I get as a result "0.93", and that's ok, is what I expected.

But if I do this:

(setq ss (ssget))
(setq e (ssname ss 0))
(setq ent (entget e))
(setq dimValue (cdr (assoc 42 ent)))
(setq roundedValue (rtos dimValue 2 2))

And select a dimension with a value of 0.925, the result is "0.92", despite both variable types,  "number" and "dimValue" are REAL and have the same value.

Does anyone know why this happen?

 

Thanks in advance

Edited February 5, 2021 by capocad

[BIGAL]

I drew a line then offset 0.925 then did code and get 0.93 is the 0.925 an overide dimension ?

[Lee Mac]

What is returned if you change (rtos dimValue 2 2) to (rtos dimValue 2 15)?

[capocad]

23 minutes ago, BIGAL said:

I drew a line then offset 0.925 then did code and get 0.93 is the 0.925 an overide dimension ?

 

No, it's not a override dimension.

1 - I create a new file using acadiso.dwt as template

2 - Create a line of 0.925 length

3 - Create the dimension for the line using the default ISO-25 style

4 - Use this code to test:

(defun c:testrtos( / )
	(setq ss (ssget))
	(setq e (ssname ss 0))
	(setq ent (entget e))
	(setq dimValue (cdr (assoc 42 ent)))
	(setq roundedValue (rtos dimValue 2 2))
  	(princ roundedValue)
  	(princ)
 )

As a result, I get "0.92"

[capocad]

12 minutes ago, Lee Mac said:

What is returned if you change (rtos dimValue 2 2) to (rtos dimValue 2 15)?

Despite creating the line of exactly 0.925, the result with (rtos dimValue 2 15) is "0.924999999999955".  Is there any way to get 0.93 as a result to match what the dimension shows on screen?

[BIGAL]

I did a line  Length =0.92500000,  Angle in XY Plane = 90d0'0" 

 

(setq x 0.924999999999955)
0.925

Command: (rtos x 2 2)
"0.92"

Command: (rtos x 2 3)
"0.925"

 

Command: (setq x 0.925)
0.925

Command: (rtos x 2 2)
"0.93"

 

Hmmmm ?

[capocad]

I came to a clunky solution but it works:

(defun c:roundDim (/ ss e ent dimValue d e f g) 
  (setq ss       (ssget)
        e        (ssname ss 0)
        ent      (entget e)
        dimValue (cdr (assoc 42 ent))
        d        (rtos dimValue 2 3)
        e        (atof d)
        f        (rtos e 2 2)
        g        (atof f)
  )
  (princ g)
  (princ)
)

Although I'm not sure what's exactly the cause of the problem.

[Lee Mac]

20 hours ago, capocad said:

Despite creating the line of exactly 0.925, the result with (rtos dimValue 2 15) is "0.924999999999955".  Is there any way to get 0.93 as a result to match what the dimension shows on screen?

 

The value of 0.924999...955 is a consequence of the accuracy problems associated with the double-precision floating-point format used to store real numbers in computer memory - this is described in more detail here, here, and here.

 

Your code demonstrates one possible way to cater for this, however, note that your method would yield undesirable results for measurements such as 0.9249:

_$ (rtos (distof (rtos 0.9249 2 3) 2) 2 2)
"0.93"
_$ (rtos 0.9249 2 2)
"0.92"

As such, a safer way to account for the rounding of doubles would be to add a small tolerance to the original value prior to rounding, e.g.:

_$ (rtos (+ 1e-8 0.924999999999955) 2 2)
"0.93"
_$ (rtos (+ 1e-8 0.9249) 2 2)
"0.92"

However, note that retrieving the dimension measurement value and rounding to a given number of decimal places may still not yield a string which matches that displayed by the dimension, as a result of dimension style configuration - to obtain the value displayed by the dimension, you can obtain the MText content found within the dimension block, e.g.:

;; Get Dimension String  -  Lee Mac
;; Returns the displayed content of a dimension

(defun LM:getdimstring ( ent / enx rtn )
    (if
        (and
            (setq enx (entget ent))
            (wcmatch (cdr (assoc 0 enx)) "*DIMENSION")
            (setq ent (tblobjname "block" (cdr (assoc 2 enx))))
            (setq ent (entnext ent)
                  enx (entget  ent)
            )
        )
        (while (and ent (null rtn))
            (if (= "MTEXT" (cdr (assoc 0 enx)))
                (setq rtn  (cdr (assoc 1 enx)))
            )
            (setq ent (entnext ent)
                  enx (entget  ent)
            )
        )
    )
    rtn
)

 

Edited February 6, 2021 by Lee Mac

[capocad]

Thanks so much Lee!! That "Get Dimension String" function was exactly what I was looking for, much better than my clunky solution.

[Lee Mac]

You're most welome! Glad it helps