|
|
View previous topic :: View next topic |
Author |
Message |
kev Guest
|
compacting floating point numbers |
Posted: Wed Apr 30, 2003 11:15 am |
|
|
Hello All,
I am currently working on an app. that reads in GPS lat or lon data into a floating point variable. I then want to compress this value so that I can make the most efficient use of memory when I come to store the value. I still want to retain the the floating-point precision when uncompressing. Any ideas?
Thanks, Kev
___________________________
This message was ported from CCS's old forum
Original Post ID: 14084 |
|
|
Sherpa Doug Guest
|
Re: compacting floating point numbers |
Posted: Wed Apr 30, 2003 11:56 am |
|
|
:=Hello All,
:=
:=I am currently working on an app. that reads in GPS lat or lon data into a floating point variable. I then want to compress this value so that I can make the most efficient use of memory when I come to store the value. I still want to retain the the floating-point precision when uncompressing. Any ideas?
:=
:=Thanks, Kev
The format of a float is in the manual (page 191 of my manual). If you can restrict the range you might save a few bits of the exponent. If you crop the mantissa you will lose precision. You will have to do some extreeme compression to save a whole byte! It doesn't look good...
___________________________
This message was ported from CCS's old forum
Original Post ID: 14085 |
|
|
mcafzap
Joined: 07 Sep 2003 Posts: 46 Location: Manchester, UK
|
Re: compacting floating point numbers |
Posted: Wed Apr 30, 2003 1:08 pm |
|
|
I have to agree with the previous writer, I suspect you will be discarding some accuracy. What I have done is store a 16 bit difference value for Lat and Long, which just requires 4 bytes per fix. This incurs some overheads in that a full accurate reference is needed as your first value and you must ensure each subsequent fix is valid (as given in the NMEA or SIRF stream. A final proviso is that if your subject moves too far between fixes, you again need to store a 'full' fix. With cheap flash memory readily available, do you really need to bother with all this?
Steve
___________________________
This message was ported from CCS's old forum
Original Post ID: 14092 |
|
|
Douglas Kennedy
Joined: 07 Sep 2003 Posts: 755 Location: Florida
|
Re: compacting floating point numbers |
Posted: Wed Apr 30, 2003 7:38 pm |
|
|
Assuming you want to keep the precision this is mathematically impossible. There is the possability of saving 1 bit since all floating pt numbers begin with the bit set in the the most significant bit of the mantissa but why bother?
___________________________
This message was ported from CCS's old forum
Original Post ID: 14105 |
|
|
R.J.Hamlett Guest
|
Re: compacting floating point numbers |
Posted: Thu May 01, 2003 2:24 am |
|
|
:=Hello All,
:=
:=I am currently working on an app. that reads in GPS lat or lon data into a floating point variable. I then want to compress this value so that I can make the most efficient use of memory when I come to store the value. I still want to retain the the floating-point precision when uncompressing. Any ideas?
:=
I have to query slightly, the 'logic' of using floating point if you want this compression. The floating point format, uses 23bits for the mantissa. Now this potentially gives an resolution of about 5 yards on the surface of the earth, when dealing with numbers a long way apart (where the 'scale' of the exponent is the same). Though the FP format, will allow you to code a 'local' point to greater precision (when dealing with something like a latitude/longitude close to the equator/meridian), the rest of the time, the storage for the exponent, is really doing very little.
Personally, I'd code a 24bit (23+sign) integer format, and use this to store the lat/long, which will still give resolution better than un-augmented GPS reception, without having to bother about the exponent.
You can convert such a number to FP, by simply using:
union convert {
float fp;
int8 b[4];
} value;
and putting the three bytes for the integer into value.b[1] to value.b[3], and 0x7F into value.b[0];
Best Wishes
___________________________
This message was ported from CCS's old forum
Original Post ID: 14111 |
|
|
PCM programmer
Joined: 06 Sep 2003 Posts: 21708
|
Re: compacting floating point numbers |
Posted: Thu May 01, 2003 1:52 pm |
|
|
:=Hello All,
:=
:=I am currently working on an app. that reads in GPS lat or lon data into a floating point variable. I then want to compress this value so that I can make the most efficient use of memory when I come to store the value. I still want to retain the the floating-point precision when uncompressing. Any ideas?
:=
----------------------------------------------------
I once did a data logger project, where it stored the starting
and ending position of a journey into an external FRAM chip.
( <a href="http://www.ramtron.com" TARGET="_blank">http://www.ramtron.com</a>)
I didn't need the full resolution, so I compressed it down
to this: (ie., 5 bytes to hold the lat and long).
<PRE>
// byte sample
// offset Value Use
//
// Starting position
//
// 6 60 latitude degrees (0-90)
// 7 01 latitude minutes (0-59) (N/S = bit 7. N = 0, S = 1)
// 8 00 longitude degrees (0-179)
// 9 00 longitude minutes (0-59) (E/W = bit 7. E = 0, W = 1)
// 10 00 tenths (0-9 in each nibble) Bits 4-7 = Lat., 0-3 = Long.
</PRE>
___________________________
This message was ported from CCS's old forum
Original Post ID: 14127 |
|
|
kev Guest
|
Re: compacting floating point numbers |
Posted: Fri May 02, 2003 3:57 am |
|
|
:=:=Hello All,
:=:=
:=:=I am currently working on an app. that reads in GPS lat or lon data into a floating point variable. I then want to compress this value so that I can make the most efficient use of memory when I come to store the value. I still want to retain the the floating-point precision when uncompressing. Any ideas?
:=:=
:=----------------------------------------------------
:=
:=I once did a data logger project, where it stored the starting
:=and ending position of a journey into an external FRAM chip.
:=( <a href="http://www.ramtron.com" TARGET="_blank"> <a href="http://www.ramtron.com" TARGET="_blank">http://www.ramtron.com</a></a>)
:=
:=I didn't need the full resolution, so I compressed it down
:=to this: (ie., 5 bytes to hold the lat and long).
:=
:=<PRE>
:=
:=// byte sample
:=// offset Value Use
:=//
:=// Starting position
:=//
:=// 6 60 latitude degrees (0-90)
:=// 7 01 latitude minutes (0-59) (N/S = bit 7. N = 0, S = 1)
:=// 8 00 longitude degrees (0-179)
:=// 9 00 longitude minutes (0-59) (E/W = bit 7. E = 0, W = 1)
:=// 10 00 tenths (0-9 in each nibble) Bits 4-7 = Lat., 0-3 = Long.
:=</PRE>
Thanks, this maybe what I am after. Any chance you could supply
more detail, i.e sample 'C' code etc.
Regards, Kev
___________________________
This message was ported from CCS's old forum
Original Post ID: 14139 |
|
|
PCM programmer
Joined: 06 Sep 2003 Posts: 21708
|
Re: compacting floating point numbers |
Posted: Fri May 02, 2003 11:52 am |
|
|
:=
:=Thanks, this is maybe what I am after. Any chance you could supply
:=more detail, i.e sample 'C' code etc.
:=
---------------------------------------------------------------
This seems like a tremendous amount of code to post.
Hopefully it will give you some ideas.
In some places it seems like I'm doing some unsafe things
like subtracting 3 from a global buffer index, but in fact
the message has been checked to see that it has at least a
minimum number of characters before it's ever sent to this
routine.
Code: | //------------------------------------------------------------
// Part of Trip.h
typedef struct
{ // Size in bytes must be = 16
char c_duration; // 1
T_DATE_TIME starting_date_time; // 5
T_LAT_LONG starting_position; // 5
T_LAT_LONG ending_position; // 5
}T_TRIP_RECORD;
#define TRIP_RECORD_SIZE 16 // The trip record size must be 16
// In this structure, bit fields are used to store tenths (ie., .0 to .9) of
// a minute, for latitude and longitude. The compiler stores them in 1 byte.
typedef struct
{
char c_latitude_degrees; // 0 to 90 degrees
char c_latitude_minutes; // 0 to 59 minutes, N/S in bit 7, 1 = South.
char c_longitude_degrees; // 0 to 180 degrees
char c_longitude_minutes; // 0 to 59 minutes, E/W in bit 7, 1 = West.
char nib_longitude_tenths:4; // 0 to 9 tenths of a minute (Long. in low nibble)
char nib_latitude_tenths:4; // 0 to 0 tenths of a minute (Lat. in high nibble)
}T_LAT_LONG;
// Create one instance of the trip record. We'll temporarily
// store data here before we read or write to FRAM.
T_TRIP_RECORD gt_trip_record;
// Also create a structure to hold the lat & long data when it
// comes in from the GPS.
T_LAT_LONG gt_gps_data;
// Part of message.h
#define MSG_BUFFER_LEN 80 // Long enough for all message types
#define MAX_MSG_LEN (MSG_BUFFER_LEN -1)
char gca_msg_buffer[MSG_BUFFER_LEN]; // Buffer to hold a msg read from recv fifo
// Characters
#define STX 0x02 // ASCII code for the Start of Text character
#define COMMA ','
#define PERIOD '.'
#define SLASH '/'
#define COLON ':'
#define LINE_FEED 0x0a
#define CARRIAGE_RETURN 0x0d
char gc_msg_state; // Current state of the Message parser state machine
#define MSG_BUFFER_LEN 80 // Long enough for NMEA (or other) messages
#define MAX_MSG_LEN (MSG_BUFFER_LEN -1)
char gca_msg_buffer[MSG_BUFFER_LEN]; // Buffer to hold a msg read from recv fifo
char *gcp_msg_buffer_ptr;
char gc_msg_buffer_count; // Number of bytes in a message
short gb_got_message;
short gb_got_A_field;
short gb_msg_header_type;
short gb_need_starting_position = TRUE;
typedef struct
{
char degrees; // 0 to 90 for latitude, or 0 to 180 for Longitude
char minutes; // 0 to 59
char tenths; // Tenths of a minute (stored as 0 to 9)
}T_DMS;
//--------------------------------------------------------------
// The following functions are part of message.c
//==============================================================
// If the Data Logger is hooked up to a GPS unit that sends NMEA GGA messages,
// then this function will be called. We need to extract the latitude and
// longitude information from the message and store it in a global variable
// that holds the current position. We will get the GGA message in this
// format:
//
//
// 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
// | | | | | | | | | | | | | | |
// xxGGA,hthmmss.ss,ddmm.hh,a,dddmm.hh,a,x,xx,x.x,xx.x,M,-xx.x,M,x.x,xxxx*hh
//
// The above line has 73 chars.
// Field Number:
// 1) Universal Time Coordinated (UTC)
// 2) Latitude (Note: Can optionally have 3 places to the right of the
// decimal)
// 3) N or S (North or South)
// 4) Longitude (Note: Can optionally have 3 places to the right of the
// decimal)
// 5) E or W (East or West)
// 6) GPS Quality Indicator,
// 0 - fix not available,
// 1 - GPS fix,
// 2 - Differential GPS fix
// 7-14) These fields are of no interest to us
// 15) Checksum
// We are only interested in fields 2 thru 6. We can find fields by counting
// commas.
void process_gga_message(void)
{
char i;
char *ptr; // ptr to message string
char num_fields;
char field_len;
char result;
#define MAX_STR_LEN 9
char temp_buffer[MAX_STR_LEN +1];
T_DMS t_position;
// See if there is a checksum present. If so, then verify it.
if(gca_msg_buffer[gc_msg_buffer_count -3] == '*')
{
if(is_checksum_good() == FALSE) // If the checksum is bad
{
putc(NAK); // then send a NAK char, and
return; // don't process the message.
}
gca_msg_buffer[gc_msg_buffer_count -3] = 0; // Delete checksum from string
}
// Get a local pointer to the message string.
ptr = gca_msg_buffer;
// Count the number of fields in the message by looking for commas.
num_fields = strccnt(ptr, COMMA);
// We need at least 6 fields to be able to get the data we want.
// So if we found at least 7 commas, then the 6th field will be there.
if(num_fields < 7) // If the fields aren't there, the message is bad
return;
// First, see if the lat-long information is valid.
// Advance to the GPS Quality Indicator field, which is after the 6th comma.
// There's no risk of returning a null pointer, because we've already
// determined that there are at least 6 commas in the string.
for(i = 0; i < 6; i++)
ptr = strchr16(ptr, COMMA) + 1; // Point to the data in the field
// If the message is not valid, then exit.
if(*ptr == '0') // Ascii 0 means "fix not available" which means "bad" (?)
return;
// Advance to the Latitude field.
// Again, there's no risk of getting a null pointer back from strchr16,
// because we know it has 6 commas, and we're not doing more searches
// than that, and we're resetting the pointer to the start of the string.
ptr = gca_msg_buffer; // Reset ptr to start of message
ptr = strchr16(ptr, COMMA) + 1 ; // Find start of UTC field
ptr = strchr16(ptr, COMMA) + 1; // Find start of latitude field
field_len = strlenc(ptr, COMMA); // Get the field length (not including the comma)
if(field_len > MAX_STR_LEN)
return;
memcpy(temp_buffer, ptr, field_len); // Copy the latitude field to a temp buffer
temp_buffer[field_len] = 0; // Make it be a string
// Convert the ascii Latitude field to binary values, and put the binary values
// into the position data structure. It's in the form: ddmm.hh (or ddmm.ttt)
// where hh is hundredths of a minute. (ttt = thousandths)
result = convert_latlong_field(temp_buffer, &t_position);
if(result == FALSE)
return;
if(t_position.degrees > 90) // Did degrees go past limit ? (should never happen)
t_position.degrees = 90; // If so, restore it to the limit
// Save the degrees and minutes in a global structure.
gt_gps_data.c_latitude_degrees = t_position.degrees;
gt_gps_data.c_latitude_minutes = t_position.minutes;
gt_gps_data.nib_latitude_tenths = t_position.tenths;
// Find the start of the latitude sign field (N or S).
ptr = strchr16(ptr, COMMA) + 1;
// Now get the sign.
if((*ptr & 0x5f) == 'S') // Convert to upper case. Is it South ?
gt_gps_data.c_latitude_minutes |= 0x80; // If so, set bit 7 in minutes
// Advance to the start of the Longitude field.
ptr = strchr16(ptr, COMMA) + 1;
field_len = strlenc(ptr, COMMA); // Get the field length
if(field_len > MAX_STR_LEN)
return;
memcpy(temp_buffer, ptr, field_len); // Copy the longitude field to a temp buffer
temp_buffer[field_len] = 0; // Make it be a string
// Decode the Longitude field. It's in the form: dddmm.hh (or dddmm.ttt)
// where hh is hundredths of a minute. (ttt = thousandths)
result = convert_latlong_field(temp_buffer, &t_position);
if(result == FALSE)
return;
if(t_position.degrees > 180) // Did degrees go past limit ? (should never happen)
t_position.degrees = 180; // If so, restore it to the limit
// Save the degrees and minutes in a local global structure.
gt_gps_data.c_longitude_degrees = t_position.degrees;
gt_gps_data.c_longitude_minutes = t_position.minutes;
gt_gps_data.nib_longitude_tenths = t_position.tenths;
// Find the start of the latitude sign field (E or W).
ptr = strchr16(ptr, COMMA) + 1;
// Now get the sign.
if((*ptr & 0x5f) == 'W') // Convert to upper case. Is it West ?
gt_gps_data.c_longitude_minutes |= 0x80; // If so, set bit 7 in minutes
// We only write the starting position to the trip record once,
// so we don't want to do it until the trip has officially started.
// Also, we clear the trip record before we start the trip,
// so if we wrote the starting position before the trip started,
// then it would be cleared. So that's why we need the following line.
if(gb_trip_started == TRUE)
{
if(gb_need_starting_position == TRUE)
{
gb_need_starting_position = FALSE;
memcpy((char *)>_trip_record.starting_position, (char *)>_gps_data, sizeof(T_LAT_LONG));
}
else
{
memcpy((char *)>_trip_record.ending_position, (char *)>_gps_data, sizeof(T_LAT_LONG));
}
}
// We got a good GPS message, so blink the logging LED.
start_logging_led_blinking();
}
//--------------------------------------------------------------
// This sub-routine decodes a NMEA latitude or longitude field. It returns
// the degrees, minutes, seconds as binary values.
// The field can be in this format: ddmm.hh, or dddmm.hh, and it could conceivably
// have leading zeros, such as 0dmm.hh or 00dmm.hhs. It could also have no leading
// zeros or spaces: dmm.hh. And, it could have 3 places to the right of the decimal,
// such as ddmm.ttt, where ttt is thousandths.
// The incoming field must be a valid string.
char convert_latlong_field(char *ptr, T_DMS *t_position)
{
#define MIN_FIELD_LEN 6 // Minimum should be dmm.hh (6 chars)
#define MAX_FIELD_LEN 9 // Maximum should be dddmm.ttt (9 chars)
char latlong[MAX_FIELD_LEN +1]; // +1 for string terminator
char field_len;
char decimal_point_pos;
char degrees;
char minutes;
char hundredths;
char tenths;
// First, find the length of the field.
field_len = strlen(ptr);
// Then check the length to see if it's OK.
if(field_len < MIN_FIELD_LEN) // If it's shorter than dmm.hh, then return False
return(FALSE);
if(field_len > MAX_FIELD_LEN) // If it's longer than dddmm.ttt, then return False
return(FALSE);
memcpy(latlong, ptr, field_len); // Copy the field to a local array
latlong[field_len] = 0; // Make it be a string
// Find the offset of the decimal point within the string.
decimal_point_pos = strlenc(latlong, PERIOD);
if(decimal_point_pos == 0xff) // Return False if we couldn't find the decimal point
return(FALSE);
// Now validate the decimal point position. Minimum is: dmm.hh Max is: dddmm.ttt
if(decimal_point_pos < 3)
return(FALSE);
if(decimal_point_pos > 5)
return(FALSE);
// If there are 3 digits to the right of the decimal, truncate it to be 2 digits.
latlong[decimal_point_pos +3] = 0; // Convert .ttt into .hh
hundredths = adtoi(&latlong[decimal_point_pos +1]); // Convert the hundredths (hh) to binary
latlong[decimal_point_pos] = 0; // Make the minutes field be a string
minutes = adtoi(&latlong[decimal_point_pos -2]); // Convert the minutes to binary
latlong[decimal_point_pos -2] = 0; // Make the degrees field be a string
degrees = adtoi(latlong);
// At this point, we've extracted the degrees, minutes, and hundredths from the string.
// Round off the hundredths to the nearest tenth of a minute. ie., .15 becomes .20
hundredths += 5;
if(hundredths >= 100) // Round-up required for minutes and degrees ?
{
hundredths = 0;
minutes++; // If so, inc the minutes
if(minutes > 59) // Did minutes go past upper limit ?
{
minutes = 0; // If so, then set minutes to 0
degrees++; // Then increment the degrees
}
}
// Note: Degrees could be from 0-90 or 0-180, depending on whether it's
// latitude or longitude. Since we can't tell here which it is, the
// calling routine must check for overflow in degrees. (which shouldn't happen).
// Now convert the hundredths (00 to 99) to tenths (0 to 9)
// This method of truncation probably produces more compact code than
// division by 10, followed by multiplying by 10.
tenths = 0;
while(hundredths >= 10)
{
hundredths -= 10;
tenths++;
}
t_position->degrees = degrees; // 0-90 or 0-180 (lat. or long.)
t_position->minutes = minutes; // 0-59
t_position->tenths = tenths; // 0-9 tenths of a minute
return(TRUE);
}
//--------------------------------------------------------------
// Verify a checksum.
// The msg buffer might contain this: "PXXXG*55"
// (The acutal msg doesn't have "XXX". I changed it
// to XXX before posting this code).
// The message buffer count is 8 in this example.
char is_checksum_good(void)
{
char checksum;
char calculated_checksum;
// Convert the checksum at the end of the message to binary.
checksum = axtoi(gca_msg_buffer + gc_msg_buffer_count -2 );
// Calculate the checksum from the start of the msg buffer, up to,
// but not including the *.
calculated_checksum = calculate_checksum(gca_msg_buffer, gc_msg_buffer_count - 3);
if(checksum == calculated_checksum)
return(TRUE);
else
return(FALSE);
}
//--------------------------------------------------------------
// Calculate a checksum on a string, by XORing all the characters together.
char calculate_checksum(char *ptr, char count)
{
char i, value;
value = 0;
for(i = 0; i < count; i++)
value ^= *ptr++;
return(value);
} |
[3-7-04 Edited to remove double-spacing]
___________________________
This message was ported from CCS's old forum
Original Post ID: 14148 |
|
|
stevenm86
Joined: 11 Jun 2006 Posts: 11
|
|
Posted: Mon Jun 12, 2006 4:19 pm |
|
|
Hello.
You dont need to store so much stuff. You can store lat or lon as a 32bit int. The int has a fractional part. Ie, it stores data in 1/10000000ths of a degree. Just pretend there's a decimal point in there. If you need to do trig on the coordinates, you will need to do some conversion. I would like to write trig functions for fractional numbers like this (fixed point) but that is later.
Here is code. It reads in GPGGA and GPGLL at the moment, others can be added easily. Lat / Lon are reported in degrees ana decimal minutes, but this code converts it to decimal degrees (with one extra decimal place for accuracy). The accuracy here is very good, practically you wont need anything else.
Code: |
#use RS232(Baud=4800,Xmit=PIN_D2,RCV=PIN_B3,BITS=8,STREAM=GPS)
#use RS232(Baud=4800,Xmit=PIN_D2,RCV=PIN_B3,BITS=8,INVERT,STREAM=HOSTPC)
byte buf[80];
int32 lat=0;
int32 lon=0;
byte t_hr,t_min,t_sec;
byte fixValid;
byte posFix;
byte satUsed;
float hdop;
float alt;
byte msgPtr;
byte readChar()
{
return buf[msgPtr++];
}
byte isSp(byte c)
{
if(c=='*' || c==',' || c=='\n' || c=='\r' || c=='.')
return 1;
return 0;
}
byte toNum(byte c)
{
return c-48;
}
byte read2Digits()
{
return 10*toNum(readChar())+toNum(readChar());
}
int32 read4Digits()
{
return read2Digits()*100+read2Digits();
}
int32 readMin()
{
int32 t;
t = MAKE32(read2Digits())*MAKE32(10000);
msgPtr++;
t = t + MAKE32(read2Digits())*MAKE32(100) + read2Digits();
return t;
}
void readLat()
{
lat=read2Digits() * 10000000;
lat = lat + (100*readMin())/6;
msgPtr++;
if(readChar() == 'S')
lat = -lat;
msgPtr++;
}
void readLon()
{
lon = ((100)*(toNum(readChar())) + (read2Digits()))*(10000000);
lon = lon + (100*readMin())/6;
msgPtr++;
if(readChar() == 'W')
lon = -lon;
msgPtr++;
}
void skipBlock()
{
while(buf[msgPtr++] != ',');
}
void readTime()
{
t_hr = read2Digits();
t_min = read2Digits();
t_sec = read2Digits();
msgPtr+=5;
}
float readFloat()
{
float r = 0;
while(buf[msgPtr] != '.')
r = r*10.0+toNum(buf[msgPtr++]);
msgPtr++;
r = r+0.1*toNum(readChar());
msgPtr++;
return r;
}
void readGGA()
{
printf("%cREADING GGA \n\r", 11);
msgPtr=6;
readTime();
readLat();
readLon();
posFix = toNum(readChar());
msgPtr++;
satUsed = read2Digits();
msgPtr++;
hdop = readFloat();
alt = readFloat();
}
void readGLL()
{
printf("%cREADING GLL \n\r", 11);
msgPtr=6;
readLat();
readLon();
readTime();
if(readChar() == 'A')
fixValid=1;
else
fixValid=0;
}
void readMessage()
{
byte b=0;
byte t;
while(getc(GPS) != '$');
do
{
t = getc(GPS);
buf[b++] = t;
} while(t != '\n' && b<80);
if(buf[3]=='G')
readGGA();
if(buf[4]=='L')
{
readGLL();
}
}
|
The printf debugging calls have some extra formatting because I have a tandy 100 as a debug console. You can safely modify all the printf stuff.
To use code- call readMessage() in main repeatedly in a loop. The global variabl;es such as lat/lon/alt/hdop will be kept updated.
Latitude / Longitude can be something like 391234567. This corresponds to 39.1234567 degrees. For trig operations, convert to float and divide by 10000000. You will get some percision loss here due to float conversion.
If you want to find a bearing (or heading) you'd need to subtract coordinates and then take the arctangent.. Subtract the coordinates in their fixed point form (no percision loss). The result will likely be small in magnitude, so you can convert THAT to a float for a minimum porcision loss and take arctan taht way.
Hope this helps |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|