 |
 |
| View previous topic :: View next topic |
| Author |
Message |
kda406
Joined: 17 Sep 2003
Posts: 97
Location: Atlanta, GA, USA
|
| Efficient storage of text |
 Posted: Tue Jan 20, 2009 4:03 pm |
 |
|
I am looking for suggestions on storing large volumes of text. Here is a small example of the kinds of menus I have to work with that always burn up precious program memory:
| Code: | c = fgetc(TERM);
switch (c) {
case 'h': // Help
fprintf(TERM,"! - Factory default settings"); AB12Term_CRLF();
fprintf(TERM,"TAB - Toggle temperature monitor update interval"); AB12Term_CRLF();
fprintf(TERM,"$ - Terminal Mode"); AB12Term_CRLF();
fprintf(TERM,"@ - BIOS Defaults"); AB12Term_CRLF();
fprintf(TERM,"C - Clear 1W devices"); AB12Term_CRLF();
fprintf(TERM,"E - Dump uC EEPROM"); AB12Term_CRLF();
fprintf(TERM,"F - Find 1W Sensors"); AB12Term_CRLF();
fprintf(TERM,"H - Hardware Population"); AB12Term_CRLF();
fprintf(TERM,"L - List 1W devices"); AB12Term_CRLF();
fprintf(TERM,"N - get s/n"); AB12Term_CRLF();
fprintf(TERM,"R - Dump RTC RAM/EEPROM"); AB12Term_CRLF();
fprintf(TERM,"T - get temps"); AB12Term_CRLF();
fprintf(TERM,"a - dC/dt Array"); AB12Term_CRLF();
fprintf(TERM,"d - show delays/incidents"); AB12Term_CRLF();
fprintf(TERM,"g - get settings (dip switches)"); AB12Term_CRLF();
fprintf(TERM,"l - Read logs"); AB12Term_CRLF();
fprintf(TERM,"q - ETs & Cycles"); AB12Term_CRLF();
fprintf(TERM,"r - Reset (same as reset button)"); AB12Term_CRLF();
fprintf(TERM,"t - toggle test mode"); AB12Term_CRLF();
fprintf(TERM,"x - Toggle expert mode"); AB12Term_CRLF();
fprintf(TERM,"{ - Override up stage timer"); AB12Term_CRLF();
fprintf(TERM,"} - Override assist timer"); AB12Term_CRLF();
AB12Term_CRLF();
break; |
I use the function AB12Term_CRLF(); to try to save memory: | Code: | #separate
void AB12Term_CRLF(void) { fprintf(TERM,"\n\r"); } |
Please feel free to suggest better ways to store these large volumes of data inside the program memory.
Thanks,
Kyle |
|
 |
PCM programmer
Joined: 06 Sep 2003
Posts: 21708
|
|
| Posted: Tue Jan 20, 2009 4:10 pm |
|
|
| Post your PIC and compiler version so we know the capabilities. |
|
|
RLScott
Joined: 10 Jul 2007
Posts: 465
|
| Re: Efficient storage of text |
| Posted: Tue Jan 20, 2009 5:11 pm |
|
|
Use 6 bits per character instead of 8. Replace all your instances of
| Code: |
fprintf(TERM,"---some string---");
|
with something like
| Code: |
PrintExpandTerm(---some compacted data---);
|
For your coding, use the last 6 bits of the ASCII representation of each character. To define bit 6, reserve a few special characters to act as shift characters. For example, for the string "Hello!", define:
| Code: |
BYTE CONST HelloCompacted[6]= {
0b00100010, // 'H' and 2 bits of 'e'
0b01011011, // rest of 'e' and 4 bits of 'l'
0b00101100, // rest if 'l' and all of of next 'l'
0b10111111, // 'o' and 2 bits of '~' (a reserved shift character)
0b11101000, // rest of '~' and 4 bits of '!'
0b10000000}; // rest of '!' and 6 bits of zeros (string terminator)
|
Then for your decoding and printing function, you can write:
| Code: |
void PrintExpandTerm(BYTE CONST *compacted)
{
BYTE Bit6 = 0b01000000;
while(1)
{
//..get next 6 bits from *compacted into x6
if(x6==0) break;
if(x6==('~'&0x3f)) Bit6 = 0b00000000;
else if(x6==('}'&0x3f)) Bit6 = 0b01000000;
else fputc(TERM, x6 | Bit6);
}
}
|
I have omitted the code for stripping out 6 bits at a time from a string of bytes because that can get quite involved, but you get the idea. Depending on how many special characters you are willing to reserve, you could define things like:
'{' = shift to Bit6=0 for just one character (temporary shift)
You will notice that with only 6 bits to represent each character, '~', '}', '|', and '{' are equivalent to '?', '>', '=', and '<'. So if you want to recognize a reserved character in either context (with respect to bit 6) then you will have to find reserved characters that are reserved in both forms. This may be a problem with '?' since that is a character you might really want to use. However '_' and '^' correspond to ASCII control characters RS and US, so they might be better choices for reserved characters.
The benefit of such a compression is only 3/4, and not even that because some overhead is used to shift between the upper and lower halves of the ASCII table. But if the number of strings that you have is large enough, it might justify the cost of such a decompression algorithm. Note that in my implementation, '@' is also a reserved character because 000000 is the string terminator.
_________________
Robert Scott
Real-Time Specialties
Embedded Systems Consulting |
|
|
treitmey
Joined: 23 Jan 2004
Posts: 1094
Location: Appleton,WI USA
|
|
| Posted: Tue Jan 20, 2009 5:57 pm |
|
|
I think there was a program to aid in this for the PIC16Fxx
look up stringtorom2.exe |
|
|
FvM
Joined: 27 Aug 2008
Posts: 2337
Location: Germany
|
|
| Posted: Tue Jan 20, 2009 6:05 pm |
|
|
| The largest waste of program memory is by the repeated fprintf() statements and function calls. The menu can be printed from a list of null-terminated strings as well, with \0\0 as block terminator. |
|
|
mayler
Joined: 13 Nov 2008
Posts: 10
|
|
| Posted: Wed Jan 21, 2009 6:49 am |
|
|
How fvm said you can make a array of strings or You can make all calls in one line of fprintf, like that:
fprintf(TERM,"! - Factory default settings\r\n$ - Terminal Mode\r\n (other strings here)"); |
|
|
kda406
Joined: 17 Sep 2003
Posts: 97
Location: Atlanta, GA, USA
|
|
| Posted: Wed Jan 21, 2009 8:33 am |
|
|
Thanks for some intriguing replies.
I am using 4.076, although I have 4.085 and will upgrade soon. Typical processor is a PIC18F8722, with 128MB of program space. There is a lot of user interface text in each of my projects.
Robert, the compression is interesting, and I will consider it.
Treitmey, do you know if stringtorom2.exe works with PIC18 devices?
FvM, I used to put strings in long single print statements. Sometimes this would crash the compiler and sometimes it would make miscellaneous non existent errors pop up all over the program code (this was really a crash too). To avoid the stack overflows or whatever was happening behind the scenes on the compiler, I eventually had to drop down to using no more than about 80 characters or so in length per line as a 'rule of thumb'.
Fvm or Mayler, can one of you post a bit of sample code about using a list/array of null terminated strings? I have tried something like this in the past and it seemed to make the code bigger, not smaller. Perhaps I was doing it incorrectly. If you could provide a simple example of how to best implement this, it would be most appreciated.
Thanks very much to all involved,
Kyle |
|
|
treitmey
Joined: 23 Jan 2004
Posts: 1094
Location: Appleton,WI USA
|
|
| Posted: Wed Jan 21, 2009 9:35 am |
|
|
google found the link for post string2rom2.exe by burnsy
http://www.ccsinfo.com/forum/viewtopic.php?p=34218
His link is dead, but I could re post exe.
This was more geared toward storing ascii in the 16Fxxx 14bit rom area.
| Quote: | When interfacing the program memory block, the
EEDATA and EEDATH registers form a two-byte word
that holds the 14-bit data for read/write |
Not really applicable to 18Fxxx. But similar to technique described above, where data bit count is reduced using a type of encoding. |
|
|
kda406
Joined: 17 Sep 2003
Posts: 97
Location: Atlanta, GA, USA
|
|
| Posted: Wed Jan 21, 2009 10:33 am |
|
|
For education, I took yesterdays' example, and compiled it in a test program. The example in the first post uses 2514 bytes of program memory. Then I took the example from the beginning of the post and turned it into the following command: | Code: | | fprintf(TERM,"! - Factory default settings\n\rTAB - Toggle temperature monitor update interval\n\r$ - Terminal Mode\n\r@ - BIOS Defaults\n\rC - Clear 1W devices\n\rE - Dump uC EEPROM\n\rF - Find 1W Sensors\n\rH - Hardware Population\n\rL - List 1W devices\n\rN - get s/n\n\rR - Dump RTC RAM/EEPROM\n\rT - get temps\n\ra - dC/dt Array\n\rd - show delays/incidents\n\rg - get settings (dip switches)\n\rl - Read logs\n\rq - ETs & Cycles\n\rr - Reset (same as reset button)\n\rt - toggle test mode\n\rx - Toggle expert mode\n\r{ - Override up stage timer\n\r} - Override assist timer\n\r"); |
As you can see, I did the whole thing in a single line. This change dropped program size to 1648 bytes. Excellent!
Wait nope, not so fast...
The 2514 byte version prints to the terminal fine, and then waits for user input. The 1648 byte program (with only this one change) now prints: | Code: | > !Esß*ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿøeéÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿö ÷\ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
u$Ppÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿi
ààÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
˜ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿõHtß•\ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
nÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿø n¾ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0e øYÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
ÿ<snip>
|
and after a few minutes of watching the nonsense, it never did stop and wait for user input. The program was crashed to the point of requiring a hardware reset.
So it seems better on the surface (less ROM) and says it compiles successfully, but if it crashes (hard) when trying to print the information, it is no good. I have to go with bloat over instability.
I am still curious for a code sample from FvM or Mayler.
Thanks,
Kyle |
|
|
FvM
Joined: 27 Aug 2008
Posts: 2337
Location: Germany
|
|
| Posted: Wed Jan 21, 2009 5:10 pm |
|
|
My previous suggestion of string lists works with general C pointer techniques, but unfortunately not with CCS C ROM
constants, at least not in default #device CONST=ROM mode. The handbook and also readme.txt are mentioning
alternative syntax for indexed strings, but I didn't manage to get any of this ever working. I guess, it's a feature to come.
Also long printf constants apparently don't work, but the below code does:
| Code: | const char menu[] =
"! - Factory default settings\n\r"
"TAB - Toggle temperature monitor update interval\n\r"
"$ - Terminal Mode\n\r"
"@ - BIOS Defaults\n\r"
"C - Clear 1W devices\n\r"
"E - Dump uC EEPROM\n\r"
"F - Find 1W Sensors\n\r"
"H - Hardware Population\n\r"
"L - List 1W devices\n\r"
"N - get s/n\n\r"
"R - Dump RTC RAM/EEPROM\n\r"
"T - get temps\n\r"
"a - dC/dt Array\n\r"
"d - show delays/incidents\n\r"
"g - get settings (dip switches)\n\r"
"l - Read logs\n\rq - ETs & Cycles\n\r"
"r - Reset (same as reset button)\n\r"
"t - toggle test mode\n\r"
"x - Toggle expert mode\n\r"
"{ - Override up stage timer\n\r"
"} - Override assist timer\n\r";
printf(menu); |
|
|
|
mayler
Joined: 13 Nov 2008
Posts: 10
|
|
| Posted: Wed Jan 21, 2009 8:00 pm |
|
|
You can try with an array of strings...
| Code: |
int i;
const char menu[22][*] ={
"! - Factory default settings",
"TAB - Toggle temperature monitor update interval",
"$ - Terminal Mode",
"@ - BIOS Defaults",
"C - Clear 1W devices",
"E - Dump uC EEPROM",
"F - Find 1W Sensors",
"H - Hardware Population",
"L - List 1W devices",
"N - get s/n",
"R - Dump RTC RAM/EEPROM",
"T - get temps",
"a - dC/dt Array",
"d - show delays/incidents",
"g - get settings (dip switches)",
"l - Read logs",
"q - ETs & Cycles",
"r - Reset (same as reset button)",
"t - toggle test mode",
"x - Toggle expert mode",
"{ - Override up stage timer",
"} - Override assist timer"}
for(i=0;i<22;i++) printf("%s\n\r",menu[i]); |
To save a little memory, i put \n\r in printf code. |
|
|
kda406
Joined: 17 Sep 2003
Posts: 97
Location: Atlanta, GA, USA
|
|
| Posted: Thu Jan 22, 2009 9:52 am |
|
|
Both suggestions work. Here are the results:
Original way, 2514 bytes
Long line method, 1648 bytes, crashes
FvM's way, 1658 bytes
Mayler's way, 1790 bytes
My gut feeling was that Mayler's way would take less program memory, but the FvM method was significantly smaller. Both were a great savings over the way I was doing it.
THANKS so much for educating me on this. Hopefully others can learn from these examples too.
Kyle |
|
|
FvM
Joined: 27 Aug 2008
Posts: 2337
Location: Germany
|
|
| Posted: Thu Jan 22, 2009 10:32 am |
|
|
Interestingly, the array of strings method doesn't work with the string in the first place of printf() statement, e.g.
printf(menu[i]). It would simplify the print statement, if no additional formatting is required. Generally, array
of strings has some overhead for the index table to access individual strings, but it's very useful in some places,
I think. As written in the compiler handbook, the size can be omitted with the first array index.
| Code: | | const char menu[][*] ={ |
|
|
|
mayler
Joined: 13 Nov 2008
Posts: 10
|
|
| Posted: Thu Jan 22, 2009 1:53 pm |
|
|
| My method is interesting if you want to access one specific line at time. The [][*] in previous versions doesnt work... (4.049). The difference of 8% its the overhead ( very little), the for and the index in printf. If you make a printf without for ( printf("%s%s%s%s...")) i think the difference will shrink a little. But the FvM way is the most optimized method IMO. |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ 
Search
Register
Profile 
Log in to check your private messages 
Log in
