This forum uses cookies
This forum makes use of cookies to store your login information if you are registered, and your last visit if you are not. Cookies are small text documents stored on your computer; the cookies set by this forum can only be used on this website and pose no security risk. Cookies on this forum also track the specific topics you have read and when you last read them. Please confirm that you accept these cookies being set.

set value 5000 on float16 becomes 4999.68
#2
float16 has limited precision so you can't get 5000 precisely. If possible use float32 or int16.
Reply


Messages In This Thread
RE: set value 5000 on float16 becomes 4999.68 - by admin - 27.08.2021, 12:20

Forum Jump: