 |
 |
| View previous topic :: View next topic |
| Author |
Message |
hemnath
Joined: 03 Oct 2012
Posts: 242
Location: chennai
|
| Error in displaying floating point |
 Posted: Thu Oct 08, 2015 11:38 pm |
 |
|
Hi all,
Controller: 18F2520; Frequency: 6.144Mhz; Compiler Version: V4.114.
I have tried to do some mathematical calculation. First I have tried with Excel, and I got 5.00000000000000000000
When I use the same calculation with the controller, it displays 5.000012397
How to achieve the same result which i got using Excel. Should I have to use 16 or 32 bit controller to perform this calculations? Or any other help?
Thanks in advance. |
|
 |
Ttelmah
Joined: 11 Mar 2010
Posts: 19447
|
|
| Posted: Fri Oct 09, 2015 12:36 am |
|
|
You probably can't.....
Internally Excel uses quite a lot (a huge amount...) of tricks to try to retain precision. It uses 8 byte fp, but then applies extra algorithms to this, to improve the actual accuracy obtained. There are some standard 'tests' to test for errors in fp maths (that are inherent in the format), but Excel deliberately avoids fp if it doesn't have to use it. So in some cases tests that are deliberately designed to show fp errors, do not show these in Excel. It tests if a reciprocal for example would give a result that can't be expressed accurately in fp, and if it couldn't, will attempt to invert the sum, and then reciprocate later. Things that don't involve fractions are also always solved in integer first, before switching to fp (using integer .01 steps). The actual maths library used, was available via Technet a few years ago, and compiled to over 400K.
The standard fp library on the PIC, is only 4byte fp, giving about 6 digits of precision. So start by just displaying the number using a sensible size: %7.4f for instance, would give you 5.0000. You could switch to using 8byte fp, but you still won't have the extras associated with the Excel library.
Why on earth are you worrying about 1 part in 500000?. Nothing you can physically 'measure' will ever be this accurate. |
|
|
hemnath
Joined: 03 Oct 2012
Posts: 242
Location: chennai
|
|
| Posted: Fri Oct 09, 2015 2:06 am |
|
|
Thanks Ttelmah. I want the precision of at least 10 decimal places.
Is it possible? |
|
|
RF_Developer
Joined: 07 Feb 2011
Posts: 839
|
| Re: Error in displaying floating point |
| Posted: Fri Oct 09, 2015 2:09 am |
|
|
| hemnath wrote: |
How to achieve the same result which i got using excel. |
Answer: You use Excel.
| hemnath wrote: | | Should I have to use 16 or 32 bit controller to perform this calculations? |
Answer: the bit length of the controller has pretty much nothing to do with it. All 8, 16, or 32 bit controllers using the same, or equivalent four byte 32 bit floating point format (and these days most do, i.e. IEEE754 or a variant) will produce the same or very similar result. The (lack of) precision is inherent to the floating point format used, not the bit length of the processor its run on.
Translation: using an ARM or other 32 bit processor won't magically improve the precision of single length floats. Using double or extended length on any processer would improve it, but it would still not necessarily be "right". For example 0.1 cannot be precisely represented in any floating point format that uses a binary exponent, which means most of them, including IEEE-754, as it requires an infinite length mantissa. However, neither double (64 bit) nor extended (80bit) floating point are available on most pics with CCS C. Doubles are only available on 24s and 33 series, i.e. the PCD compiler.
Why do you need ten digit precision? Why do you REALLY need it? What's the application? Often I've found that rearranging the calculations - optimising the algorithm for computability, gets round most precsion problems. The fact that you are, until now, unaware of float precision, suggest strongly that you may not be doing whaterver it is that you are doing in the best way.
No, ten digit precision floats are not possbile with CCS C on 18 series PICs... |
|
|
Ttelmah
Joined: 11 Mar 2010
Posts: 19447
|
|
| Posted: Fri Oct 09, 2015 3:00 am |
|
|
And as has often been said, consider if you could use integers.
99% of stuff can be coded as scaled integers. So (for instance) I have an instrument that reads down to individual parts per billion, yet in overrange can merrily switch up and handle unity values. Internally it uses int32 counts in PPB.
Don't get fooled into thinking Excel gets things right. Try this sum:
| Code: |
Sub Main()
MySum = 0
For I% = 1 To 10000
MySum = MySum + 0.0001
Next I%
Debug.Print MySum
End Sub
|
|
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ 
Search
Register
Profile 
Log in to check your private messages 
Log in
