» «

4 replies to this topic
Swoorup
• ##### Swoorup

Foot Soldier

• Feroci
• Joined: 28 Oct 2008

### #1 Posted 03 April 2012 - 05:53 PM

Floating point has more values than a unsigned int. But are there any precision losses in it? If so, then how much is the limitation?

Can this limitation be overcomed by using a double data type?

K^2
• ##### K^2

Vidi Vici Veni

• Moderator
• Joined: 14 Apr 2004
• Best Poster [Programming] 2015
Most Knowledgeable [Web Development/Programming] 2013
Most Knowledgeable [GTA Series] 2011
Best Debater 2010

### #2 Posted 03 April 2012 - 07:14 PM

Of course there is a precision loss. You have 1 bit for sign, which is effectively same as int, though it works differently. There are 8 bits for exponent, which is what lets you increase the range. And that only leaves 23 bits for mantissa, which provides you with precision. This translates to roughly 10^-8 of precision. But unlike integer, where precision is fixed, floating point precision is relative. So it's 10^-8 of whatever value you are working with.

Double precision does, in fact, double precision. It's good to about 10^-16. Problem is, not all compilers and not all hardware supports double precision natively, so using double can come at cost of performance. Furthermore, graphics hardware runs with single precision floats, so if you are making a game engine, you'd almost always try to make it work without using double. For scientific work, double is often necessary, and is in fact the default precision in Fortran, Matlab, Mathematica, etc.

Swoorup
• ##### Swoorup

Foot Soldier

• Feroci
• Joined: 28 Oct 2008

### #3 Posted 04 April 2012 - 04:22 AM

I have to work out with game units. Its bounds are:

East -10,000 units
West 4000 units

North 8000 units
South -4000 units

In between these bounds, the numbers must be able to have at least 2 decimal in places.
Furthermore the numbers must be multiplied by 16.
I am worried if floating point will be able to take this into consideration

K^2
• ##### K^2

Vidi Vici Veni

• Moderator
• Joined: 14 Apr 2004
• Best Poster [Programming] 2015
Most Knowledgeable [Web Development/Programming] 2013
Most Knowledgeable [GTA Series] 2011
Best Debater 2010

### #4 Posted 04 April 2012 - 09:05 AM

Sure. Your largest value by magnitude is 10^4. Plus two decimal places for precision is 10^6. And you need to multiply by 16. That's a little over 10^7. You have a little more than 10^8. So you have almost an order of magnitude to spare. Not a whole lot of breathing room, but sufficient for your needs as described.

Swoorup
• ##### Swoorup

Foot Soldier

• Feroci
• Joined: 28 Oct 2008

### #5 Posted 04 April 2012 - 01:59 PM

Thanks K^2, thank you a whole lot for every information. Wish I could somehow, reward you for the share of wisedom.

#### 1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users