load very big data stored in a computer file

[arl]

Hi for all,

I have a file with a raw data consisting in a very large string of 0's and 1`s.

This string is very large 98920 characters. My question is how is the better maner to include this data in a css program, ie, i want something like:

#include <18F452.h>
#fuses HS,NOWDT,NOPROTECT,NOLVP
#use delay (clock=20000000)

#include <STDLIBM.H>

char rawData[98920] = "1111111111010110011100010101010101010101010101010101010101010..........."
This array is too large so an error occur.

or

p = (char *)malloc(sizeof(char)*98920);
....
but how can i read the data from the computer file?



Another solution is load this data in my 24lc256, but how can I read the rawData that it is in a file in my computer?


Many thanks!.

[Douglas Kennedy]

I don't know what the data represents but if its an image or possibly a map .....have you thought of run length encoding RLE?
I did this a while back in which I took a NOAA raster chart and RLE encoded it into an EEPROM. Mostly because it was a chart I got really good compression. Decompressing takes a bit of time on the PIC but the user can wait a second or so for a map to display in this case on a 640x480 LCD. It was a GPS chartplotter application and when the track got close to going out of view the PIC took a second to access the newly needed part of the chart and redisplay it. In RLE instead of storing each 1 or 0 in a bit when there is a run of bits either 0 or 1 the type of bit and the run length is stored instead. Anyway you need to look at the PIC data sheets to understand the ROM and RAM you have at you disposal. After reading the data sheets you'll understand that 100,000 char on a PIC is as far away from reality as the original estimates of the Iraq occupation costs.

[arl]

thanks Douglas,

This rawData is a input information for a 1bit DAC.

RLE could be a great solution with some type of multitask buffering-reading decoded data to assure RT processing, anyway im trying to study another bitrates keeping sound quality vs RAM - ROM used.


Thanks

[sh1]

If it is only '0's and '1's, why don't you convert it to ninary format? instad of 98920 bytes, you'll have 98920 bits, 12365 bytes.

[Douglas Kennedy]

I often implement RLE as a hybrid thing. Example take a byte and reserve the first two bits to mean
bit 7 bit 6
1 1 RLE byte and there is a continuation to follow ( RLE >32)
1 0 RLE and no continuation
0 1 Raw byte
0 0 end of file marker
For a RLE byte bit 5 is the bit value of the run 0 or 1
bit 4..0 most significant 5 bits of the RLE count
or in the case of a raw byte 6 bits of the actual data EX 010111.

Unless the run length is greater than 6 no attempt is made to compress it

For what you are doing I'm not sure you'll get the compression to offset the complexity. For a nautical chart which is 90 percent background with pixels denoting shoreline showing up now and then the compression is often 100 to 1.