This forum uses cookies
This forum makes use of cookies to store your login information if you are registered, and your last visit if you are not. Cookies are small text documents stored on your computer; the cookies set by this forum can only be used on this website and pose no security risk. Cookies on this forum also track the specific topics you have read and when you last read them. Please confirm that you accept these cookies being set.

LUA Scientific Conversion loose decimal
#1
Hi,

I'm facing an issue with a POST request and its return value.

I'm sending a POST request to a device and it replay with a payload containing a number.

this number is like:
1000000012345685

LUA automatically converts to scientific notation and shown as 1.00000001234568e+15

the issue cames during its back fortmat to string/number:

string.format('%.0f',1.00000001234568e+15) >> 1000000012345680

AND NOT 1000000012345685.
this invalidate my other function that is supposed to use the right number.

Actually the issue seems to be LUA itself that truncate to 14 decimals
Considering that there is no way to get the POST request as tring, is there a way to:
- Forse LUA to use 15 decimals in autoconversion
- Disable the automatic scientific notation

Thanks
ciao
M
Reply
#2
This should work:
Code:
num = 1000000012345685
str = string.format("%.16g", num)
Reply
#3
(09.05.2022, 09:33)admin Wrote: This should work:
Code:
num = 1000000012345685
str = string.format("%.16g", num)

My issue is elsewhere...

The POST request returns 1000000012345685 as body in a JSON formatted reply
LUA Automatically converts 1000000012345685 to 1.00000001234568e+15 
I need to prevent the automatical conversion to scientific notation OR assume such number as string
Reply
#4
It's just displays it this way. Internally this number is stored without any precision loss. Use string.format from my previous post to convert it to a string.
Reply
#5
(09.05.2022, 10:01)admin Wrote: It's just displays it this way. Internally this number is stored without any precision loss. Use string.format from my previous post to convert it to a string.

uh....
...this is quite unbelievable..... but now it works....
I SWEAR it was NOT going like this before.... Big Grin Big Grin Big Grin Big Grin 

...My understading is that the POST is stored into a user library, while the value is used sfter writing it down into a storage.... What I do believe is that it is stored like a string and not like a value...

Thanks
Reply
#6
Just my last post to specify the wired behaivour and by my understanding it is something to work on....

I was able to reproduce the issue and looks to me to be a sort of bug:

Have a double number

Code:
local d = 1000000012345685
log(d)
-- shown as 1.00000001234568e+15
log(string.format('%.0f', d)
-- shown as 1000000012345685
storage.set('double', d)
d = nil
d = storage.get('double')
log(d)
-- shown as 1.00000001234568e+15
log(string.format('%.0f', d)
-- shown as 1000000012345680
Reply
#7
Convert it to a string before putting into storage, then use tonumber() after storage.get.
Reply


Forum Jump: