Taffu Posted May 18, 2010 Posted May 18, 2010 I'm having an issue with decimal precision when developing a lisp routine. I'm trying to get accurate decimal precision using (getpoint), however it seems to always round to 0.0 precision. Using (rtos) I can get the engineering precision in mode 2 to get full precision, however once I use this I can no longer use the number in arithmetic due to it becoming a string. Converting back results in it rounding back to 0.0 precision. I'm new to writing lisp routines, so if the answer is obvious, my apologies. But any help would be greatly appreciated. Quote
lpseifert Posted May 18, 2010 Posted May 18, 2010 The precision is still there with getpoint (to 16 decimal points I think), it's just not displayed. Quote
Taffu Posted May 18, 2010 Author Posted May 18, 2010 I see! I've been using (princ) to debug my development as I go and it doesn't display the full values, so I assumed it was rounding. Thank you for the information! Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.