LUA Scientific Conversion loose decimal

LUA Scientific Conversion loose decimal - Printable Version

+- Logic Machine Forum (https://forum.logicmachine.net)
+-- Forum: LogicMachine eco-system (https://forum.logicmachine.net/forumdisplay.php?fid=1)
+--- Forum: Scripting (https://forum.logicmachine.net/forumdisplay.php?fid=8)
+--- Thread: LUA Scientific Conversion loose decimal (/showthread.php?tid=4038)

LUA Scientific Conversion loose decimal - emme - 09.05.2022

Hi,

I'm facing an issue with a POST request and its return value.

I'm sending a POST request to a device and it replay with a payload containing a number.

this number is like:
1000000012345685

LUA automatically converts to scientific notation and shown as 1.00000001234568e+15

the issue cames during its back fortmat to string/number:

string.format('%.0f',1.00000001234568e+15) >> 1000000012345680

AND NOT 1000000012345685.
this invalidate my other function that is supposed to use the right number.

Actually the issue seems to be LUA itself that truncate to 14 decimals
Considering that there is no way to get the POST request as tring, is there a way to:
- Forse LUA to use 15 decimals in autoconversion
- Disable the automatic scientific notation

Thanks
ciao
M

RE: LUA Scientific Conversion loose decimal - admin - 09.05.2022

This should work:

Code:
num = 1000000012345685

str = string.format("%.16g", num)

RE: LUA Scientific Conversion loose decimal - emme - 09.05.2022

(09.05.2022, 09:33)admin Wrote: This should work:

Code:
num = 1000000012345685 str = string.format("%.16g", num)

My issue is elsewhere...

The POST request returns 1000000012345685 as body in a JSON formatted reply
LUA Automatically converts 1000000012345685 to 1.00000001234568e+15
I need to prevent the automatical conversion to scientific notation OR assume such number as string

RE: LUA Scientific Conversion loose decimal - admin - 09.05.2022

It's just displays it this way. Internally this number is stored without any precision loss. Use string.format from my previous post to convert it to a string.

RE: LUA Scientific Conversion loose decimal - emme - 09.05.2022

(09.05.2022, 10:01)admin Wrote: It's just displays it this way. Internally this number is stored without any precision loss. Use string.format from my previous post to convert it to a string.

uh....
...this is quite unbelievable..... but now it works....
I SWEAR it was NOT going like this before.... Big Grin

...My understading is that the POST is stored into a user library, while the value is used sfter writing it down into a storage.... What I do believe is that it is stored like a string and not like a value...

Thanks

RE: LUA Scientific Conversion loose decimal - emme - 10.05.2022

Just my last post to specify the wired behaivour and by my understanding it is something to work on....

I was able to reproduce the issue and looks to me to be a sort of bug:

Have a double number

Code:
local d = 1000000012345685

log(d) 

-- shown as 1.00000001234568e+15

log(string.format('%.0f', d)

-- shown as 1000000012345685

storage.set('double', d)

d = nil

d = storage.get('double')

log(d) 

-- shown as 1.00000001234568e+15

log(string.format('%.0f', d)

-- shown as 1000000012345680

RE: LUA Scientific Conversion loose decimal - admin - 10.05.2022

Convert it to a string before putting into storage, then use tonumber() after storage.get.