RE: Some issue with big numbers, Sergey, 04-30-2009

What problem are you seeing?

by Roger Harris, May 1, 2009 12:00

ODBscript uses standard C/C++ "long" (integer) and "double" (floating point) for numbers. The range of integers is -2147483647 to +2147483647. The range of floating point numbers is +/-1.0e37, but the number of significant digits is 15. (However, with floating point numbers, there can be a problem with loss of precision or "rounding errors" when converting from decimal fractions to binary and back, e.g. 1/10, 1/100, etc. cannot be represented exactly as base-2 fractions.)

So, I would think 70 000 000 000,12 would be okay for doing arithmetic. However, there was a bug at one time using the $format( ) function on a number with, I believe, 12 or more digits. Is that where you are having a problem?

Replies:

• (Reply), Sergey, 05-03-2009
Next | Previous | Top

Post Your Reply:

Name   
E-mail  optional
Topic  

HTTP Link: 
Link text: