atof conversion giving incorrect results

[milesperry]

Hi all

long time viewer first time poster.

The code here is an abbreviated version from a much larger file.

I am trying to get user input for degrees minutes seconds in the format dd.mmss entered into an input box.

I then convert this into decimal degrees which is returned below the input box

The problem seems to be with some angles but not all.

For example if I enter 89.52 (89 deg 52 minutes)

The decimal co

should give   dd=89, mm=52, ss=00

Instead I get dd=89, mm+51, ss=99

The Decimal conversion result is therefore incorrect, it returns 89.877777777777 but it should be 89.86666666

There seems to be an issue in either the atof or my code separating out dd,mm,ss

(setq mm-tmp (* (- (atof AzNew) dd) 100))

returns 51.99999999999999996 not 52 ?

 

The issue same if I enter 89.5230 (89 deg 52 min 30 sec)

Should give me dd=89, mm=52, ss=30

Instead I get       dd=89, mm=52, ss=29

 

Here is the complete code

;;-------------------------------------------------------------------------------------------------------------;;
;;  Display Line Dialog Box                                                                                    ;;
;;-------------------------------------------------------------------------------------------------------------;;
(defun dialog-AZ (/ dcl_id )

(setq dcl_id (load_dialog "AzSwing.dcl")) 

  (if (new_dialog "AzSwing" dcl_id)

    (progn
      (if (not ASlbl) (setq ASlbl "Azimuth Swing(DD.MMSS)"))
      (if (not Azimuth-decimal) (setq Azimuth-decimal "0.0000"))
;Check for Azimuth Angle user input if not apply default angle "0"
      (if (not Az) (setq Az "0.0000"))
      (if (not Azimuth-ddmmss) (setq Azimuth-ddmmss "0.0000"))

;;-----------------------------------------------------------;;
;; Azimuth Swing added/changed                               ;;
;;-----------------------------------------------------------;;
      (action_tile "Aswing" "(Aswing_Check)")

      (defun Aswing_Check ()
        (setq AzNew (get_tile "Aswing"))

        ;Enter 89.52 (89deg 52Min)
        ;Az_dd returns 89
        ;Az_mm-tmp returns 51.99999999999999996 this incorrect should be 52
        
        (setq dd (fix (atof AzNew)));Set Degrees
        (setq mm-tmp (* (- (atof AzNew) dd) 100))
        (setq ss (fix(* (- mm-tmp (fix mm-tmp)) 100)))  ;Set Seconds
        (setq mm (fix mm-tmp))  ;Set Minutes

        (setq Az_decimal (+ dd (/ mm 60.0) (/ ss 3600.0)))  ;Convert to decimal

        (updatestats)
      )  ;defun
;;-----------------------------------------------------------;;

      (action_tile "accept"
                   "(done_dialog)"
      )  ;action_tile

      (start_dialog)

      (unload_dialog dcl_id)
    )
  )
  
(princ)
)
;;-------------------------------------------------------------------------------------------------------------;;
;;  End Display Line Dialog Box                                                                                ;;
;;-------------------------------------------------------------------------------------------------------------;;


;;---------------------------------------------------------------------------------;;
;;  Update Stats Function                                                          ;;
;;---------------------------------------------------------------------------------;;
(defun updatestats()
      	(setq lbld (rtos Az_decimal 2 8))
        
        (set_tile "DeciamlDeg" lbld)
) 
;;---------------------------------------------------------------------------------;;


;;-------------------------------------------------------------------------------------------------------------;;
;;  Display Line (Main Routine)                                                                                ;;
;;-------------------------------------------------------------------------------------------------------------;;
(defun C:AZ ()

  (dialog-AZ)

  (princ)

);defun
;;-------------------------------------------------------------------------------------------------------------;;
;;  End Display Line                                                                                           ;;
;;-------------------------------------------------------------------------------------------------------------;;

and the DCL code

AzSwing : dialog {
label = "Diplay Line \"Bearing and Distance\"";
: row {
      : boxed_column {
    		: row {
              : text
              {
              label = "Azimuth Swing(DD.MMSS)";
              key = "AsLBL";
              width = 20;
              alignment = left;
              }
              :edit_box {
                        key = "Aswing";
                        //label = "Azimuth Swing(DD.MMSS)";
                        edit_width = 10 ;
                        } //end edit box
            }//end row
      label = "LINE DETAILS";
              : column {
              : row {
              : text
              {
              label = "0.000";
              key = "DeciamlDeg";
              //width = 10;
              alignment = left;
              }
              } //end row
              }//column
    
      }//end column 
    }//row
:row {
ok_cancel ;
     }

} //end dialog

Sorry for the long post but it's difficult to explain.

Regards Miles

[Roy_043]

You should use angtof instead.

[Isaac26a]

I couldn't find the reason why does your code does that, but I found a way to fix it, I don't think is the best way to do it, probably there is another much better way of doing it, but I did this.

        (setq dd (fix (atof AzNew)));Set Degrees
        (setq mm-tmp (* (- (atof AzNew) dd) 100))
	    (setq mm (atoi (rtos mm-tmp 2 0)))
	    (setq ss (atoi (rtos (* (- mm-tmp mm) 100)2 0)))
;;;        (setq ss (fix(* (- mm-tmp (fix mm-tmp)) 100)))  ;Set Seconds
;;;        (setq mm (fix mm-tmp))  ;Set Minutes

And it did work well at least here in my computer, give it a try.

[milesperry]

Thanks Roy 

I did see angtof but I believe the string needs to be input as 89d52'30" . The company I'm at has always used 89.5230 format for speed.

Thanks Isaac

That does seem to fix it, just don't know why it is doing it.

 

[milesperry]

Isaac

I just tried with 210.5959 and it give incorrect results.

[Isaac26a]

And what about making this mods:

        (setq dd (fix (atof AzNew)));Set Degrees
        (setq mm-tmp (* (- (atof AzNew) dd) 100))
	(setq mm (atoi (rtos mm-tmp 2 0)))
        (if (> (- mm-tmp mm) -0.000001)
           (setq mm (atoi (rtos mm-tmp 2 0)))
           (setq mm (fix mm-tmp))
        )					
	(setq ss (atoi (rtos (* (- mm-tmp mm) 100)2 0)))			

 

[lrm]

I think you should consider your input as a text string and not a real number.  This let's you avoid the inevitable round-off.  

 

Note, I tried the following with dimzin set to 0 and 12 but the final value was still limited to 3 decimal places.  Don't know why.

 

The following code assumes you are getting the input as a real number then converts it to a string to extract the degree, minutes, and seconds components.

 

(defun c:test (/ f a n dd mm sec decd)
  (setq	f    (getreal "\nEnter number")
	a    (rtos f 2 4)
	n    (strlen a)
	dd   (substr a 1 (- n 5))
	dd   (atof dd)
	mm   (substr (substr a (- n 3)) 1 2)
	mm   (/ (atof mm) 60.)
	sec  (substr (substr a (- n 3)) 3 4)
	sec  (/ (atof sec) 3600.)
	decd (+ dd mm sec)
  )
  (princ "\nDegrees = ")
  (princ dd)
  (princ "\nDecimal Minutes = ")
  (princ mm)
  (princ "\nDecimal Seconds = ")
  (princ sec)
  (princ "\nFull Decimal Degree notation = ")
  (princ decd)
  (princ)

)

 

 

[tombu]

Take a look at Lee Mac's Consistent rtos http://www.lee-mac.com/consistentrtos.html

it avoids having to set DIMZIN to 0 to get consistent results.

[lrm]

@tombu I'm not sure if your response was to my post but the question I have does not have to do with rtos but to the precision of the the sum of 3 numbers.

 

The issue Lee Mac discusses in his post on the inconsistencies of rtos addresses the way leading and trailing zeros are handled.   My code uses rtos to convert the format of the way @milesperry 's company wants to be able to specify an angle.   The function rtos does this fine making it possible to handle the characters (digits) preceding the decimal point as degrees, the first two characters after the decimal point as minutes and the 3rd and 4th character after the decimal point as seconds.  

 

Sample result of the program are as follows.  For an input value of 121.3031 (which should be interpreted as 121 degrees, 30 minutes, 31 seconds) the output is as follows:

 

 

Why is the output rounded to 3 decimal places if dd = 121.0, mm = 0.5, and sec = 0.00861111 with  (setq decd (+ dd mm sec))?    There's something simple I am missing. 

 

(defun c:test (/ f a n dd mm sec decd)
  (setq	f    (getreal "\nEnter number (dd.mmss): ")
	a    (rtos f 2 4)
	n    (strlen a)
	dd   (substr a 1 (- n 5))
	dd   (atof dd)
	mm   (substr (substr a (- n 3)) 1 2)
	mm   (/ (atof mm) 60.)
	sec  (substr (substr a (- n 3)) 3 4)
	sec  (/ (atof sec) 3600.)
	decd (+ dd mm sec)
  )
  (princ "\nDegrees = ")
  (princ dd)
  (princ "\nDecimal Minutes = ")
  (princ mm)
  (princ "\nDecimal Seconds = ")
  (princ sec)
  (princ "\nFull Decimal Degree notation = ")
  (princ decd)
  (princ)

)

 

[milesperry]

Thankyou lrm

I have adjusted your code to work with string input on the dialog box and it all seems to be working correctly

(setq a (rtos(atof (get_tile "Aswing"))2 4)

setting the string to float then back to string with 4 decimal places meant it would add the zeros for me eg: I could enter "89.52" and it returns "89.5200"

      (defun Aswing_Check ()
         (setq a (rtos(atof (get_tile "Aswing"))2 4)
               n    (strlen a)
               dd   (substr a 1 (- n 5))
               dd   (atof dd)
               mm   (substr (substr a (- n 3)) 1 2)
               mm   (/ (atof mm) 60.)
               sec  (substr (substr a (- n 3)) 3 4)
               sec  (/ (atof sec) 3600.)
               decd (+ dd mm sec)
               Az_decimal decd
         )       
        (updatestats)
      )  ;defun

I will test a few more bearings but it all seems to be working, thankyou.

[BIGAL]

like LRM and  others I would enter a real number. But if want a string then using the substr  of the values as a string, note this code is from like 1985.

(setq ans_len (strlen ang_ans))

  (setq x 0)
  (while (/= char_found ".")
    (setq x (+ x 1))
    (setq ans_deg (strcat ans_deg char_found))
    (setq char_found (substr ang_ans x 1))
    (if (= x 9)(setq char_found "."))
   )
   (setq x (+ x 1))
   (setq ans_min (substr ang_ans x 2))
   (setq x (+ x 2))
   (setq ans_secs (substr ang_ans x 2))
       
    (if (= ans_min "")(setq ans_min "0"))
    (if (= ans_secs "")(setq ans_secs "0"))

    (setq ang (+ (atof ans_deg)(/ (atof ans_min) 60.0)(/ (atof ans_secs) 3600.0)))

 

For me I would use something like this. Your welcome to use it.

 

(if (not AH:getvalsm)(load "Multi Getvals.lsp"))
(setq ans (AH:getvalsm (list "Enter brg and dist " "Deg" 5 4 "0" "Min" 5 4 "0" "Secs " 5 4 "0" "Length " 10 9 "0")))

(setq deg (atof (nth 0 ans))
              min  (/ (atof (nth 1 ans)) 60.0)
             sec  (/ (atof (nth 2 ans)) 3600.0)
             len (atof (nth 3 ans))
)
(setq decdeg (+ deg min sec))

 

Multi GETVALS.lsp

 

 

Edited August 24, 2021 by BIGAL

[milesperry]

Thanks @BIGAL We have a dialog box that displays bearings and distances of a survey traverse, with the option to add an Azimuth Swing in a text box and show the corrected bearings with Azimuth applied. This was created by someone that we bought is a small part of an autocad addon that we brought years ago in .vlx format that no longer installs on our latest version. I realize that getreal is better and less problematic but the boss really likes the old version so that's what I'm trying to recreate. We can easily change the units or swing without leaving the dialog box.

The substr seems to work, I just need to add some code to check for numeric string only.

[BIGAL]

Your right dcl returns a string, even though you appear to type in a real.