|
|
View previous topic :: View next topic |
Author |
Message |
allenhuffman
Joined: 17 Jun 2019 Posts: 552 Location: Des Moines, Iowa, USA
|
[Explained] write_program_memory < FLASH_WRITE_SIZE? |
Posted: Tue Dec 03, 2019 11:14 am |
|
|
Hoping someone knows the answer before I write a test program to figure this out:
Code: | FLASH_WRITE_SIZE
Smallest number of bytes that can be written to FLASH |
Does the CCS function write_program_memory() take care of writing less than FLASH_WRITE_SIZE?
i.e., if getenv("FLASH_WRITE_SIZE") returns 16 bytes as the minimum flash write size, and I try to do this:
Code: | unsigned int8 *data = { 1, 2, 3 };
write_program_memory (0x1000, data, 3); |
*Assuming* the flash is already erased, or that 0x1000 is on the FLASH_ERASE_SIZE boundry so the entire block is erase first, what will end up there?
Thanks! _________________ Allen C. Huffman, Sub-Etha Software (est. 1990) http://www.subethasoftware.com
Embedded C, Arduino, MSP430, ESP8266/32, BASIC Stamp and PIC24 programmer.
http://www.whywouldyouwanttodothat.com ?
Last edited by allenhuffman on Tue Dec 10, 2019 8:53 am; edited 1 time in total |
|
|
Ttelmah
Joined: 11 Mar 2010 Posts: 19520
|
|
Posted: Tue Dec 03, 2019 11:40 am |
|
|
The minimum that can be written is MIN_FLASH_WRITE, Hardware limitation.
However it doesn't do a read then write. You can only set bits to '0' from '1'.
If a bit needs to be set to '1' that it already '0', you will have to erase the
block first. Erased program memory holds '1'. |
|
|
allenhuffman
Joined: 17 Jun 2019 Posts: 552 Location: Des Moines, Iowa, USA
|
|
Posted: Tue Dec 03, 2019 12:38 pm |
|
|
Ttelmah wrote: | The minimum that can be written is MIN_FLASH_WRITE, Hardware limitation.
However it doesn't do a read then write. You can only set bits to '0' from '1'.
If a bit needs to be set to '1' that it already '0', you will have to erase the
block first. Erased program memory holds '1'. |
But what happens to the other bytes if I only pass 1 byte to the routine? Are they left alone?
Three scenarios - assume a 4K FLASH_ERASE_SIZE:
1)
0x1000 (4096) is on a 4K boundry, so:
Code: | unsigned int8 data[] = { 42 };
write_program_memory (0x1000, data, 1); |
...would format the 4K ERASE_SIZE from 0x1000 - 0x1FFF (clear all those bytes to 00).
Would it write that one byte at 0x1000?
2)
0x1000 (4096) is on a 4K boundry, so:
Code: | erase_program_memory (0x1000); // erase block at 0x1000
unsigned int8 data[] = { 42, 43, 44 };
write_program_memory (0x1005, data, 3); // write 3 bytes from 0x1005-0x1007 |
If I ensure that entire block is erased, then try to write a few bytes somewhere inside, but less than FLASH_WRITE_SIZE, what happens?
3)
Code: | erase_program_memory (0x1000); // erase block at 0x1000
unsigned int8 data1[] = { 42, 43, 44 };
write_program_memory (0x1005, data1, 3); // write 3 bytes from 0x1005-
unsigned int8 data2[] = { 45, 46, 47 };
write_program_memory (0x1008, data2, 3); // write 3 bytes from 0x1008-0x100a |
I ask because when we parse our HEX file, there can be a few remaining bytes at the end of a section:
Code: | :08400000102204000000000082
:104400000000E0004220AF00040037007B3E0900BE |
Above, the first line is just 8 bytes to be placed at 0x4000 (0x2000 on the PIC24).
Do I need to pad that out to FLASH_WRITE_SIZE to make it work, or does the CCS call do that for me? _________________ Allen C. Huffman, Sub-Etha Software (est. 1990) http://www.subethasoftware.com
Embedded C, Arduino, MSP430, ESP8266/32, BASIC Stamp and PIC24 programmer.
http://www.whywouldyouwanttodothat.com ? |
|
|
Ttelmah
Joined: 11 Mar 2010 Posts: 19520
|
|
Posted: Tue Dec 03, 2019 1:02 pm |
|
|
Yes, but the remaining bytes in the min size, would have indeterminate
values. They'd actually be whatever the write latches contained before
the instruction. |
|
|
allenhuffman
Joined: 17 Jun 2019 Posts: 552 Location: Des Moines, Iowa, USA
|
|
Posted: Tue Dec 03, 2019 1:15 pm |
|
|
Ttelmah wrote: | Yes, but the remaining bytes in the min size, would have indeterminate
values. They'd actually be whatever the write latches contained before
the instruction. |
Gotcha, thanks.
I write my code initially on a PC, and create wrapper functions to mirror the behavior on the hardware. In this case, I am replicating how erase/read/write_program_memory() behave.
This lets me do some intensive testing and throw all kinds of values at the routines that would be cumbersome to do on the actual hardware. _________________ Allen C. Huffman, Sub-Etha Software (est. 1990) http://www.subethasoftware.com
Embedded C, Arduino, MSP430, ESP8266/32, BASIC Stamp and PIC24 programmer.
http://www.whywouldyouwanttodothat.com ? |
|
|
allenhuffman
Joined: 17 Jun 2019 Posts: 552 Location: Des Moines, Iowa, USA
|
|
Posted: Tue Dec 03, 2019 1:17 pm |
|
|
Hey, do you know how erase_program_memory() behaves if you give it an address that is not a block start?
i.e.,
Code: | erase_program_memory (0x4); |
Would that just silently fail, or would it erase the entire FLASH_ERASE_SIZE block that 0x4 is in? _________________ Allen C. Huffman, Sub-Etha Software (est. 1990) http://www.subethasoftware.com
Embedded C, Arduino, MSP430, ESP8266/32, BASIC Stamp and PIC24 programmer.
http://www.whywouldyouwanttodothat.com ? |
|
|
Ttelmah
Joined: 11 Mar 2010 Posts: 19520
|
|
Posted: Tue Dec 03, 2019 1:58 pm |
|
|
Read the data sheet....
If you trigger an erase, it erases the entire block, and ignores the low bits
of the address loaded into the address registers.
So it'll erase from the start of the block containing the address to the end. |
|
|
allenhuffman
Joined: 17 Jun 2019 Posts: 552 Location: Des Moines, Iowa, USA
|
|
Posted: Tue Dec 03, 2019 2:30 pm |
|
|
Ttelmah wrote: | Read the data sheet....
If you trigger an erase, it erases the entire block, and ignores the low bits
of the address loaded into the address registers.
So it'll erase from the start of the block containing the address to the end. |
So the answer is the CSI wrapper erase_program_memory() is a direct pass through to the chip. Good.
While read_program_memory() and write_program_memory() use actual code to read or write larger blocks of data to the chip.
From testing on a 24FJ256GA106 I see that the write_ of fewer than FLASH_WRITE_SIZE appears to leave the bytes alone. On real hardware, I erase at 0x8000 (a multiple of the FLASH_ERASE_SIZE), then read the entire FLASH_WRITE_SIZE into a buffer and hex dump it, then write just 16 bytes, then read back the FLASH_WRITE_SIZE and dump it.
Code: | FLASH_ERASE_SIZE: 2048
FLASH_WRITE_SIZE: 256
erase 0x8000
read 256 bytes from 0x8000...
0x8000: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8010: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8020: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8030: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8040: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8050: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8060: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8070: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8080: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8090: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80a0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80b0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80c0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80d0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80e0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80f0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
write 16 bytes to 0x8000...
0x0000: 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f 50 51
read 256 bytes from 0x8000...
0x8000: 42 43 44 00 46 47 48 00 4a 4b 4c 00 4e 4f 50 00
0x8010: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8020: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8030: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8040: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8050: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8060: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8070: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8080: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x8090: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80a0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80b0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80c0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80d0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80e0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x80f0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00 |
I can only test on the four or five PIC24 variants we use, so do you know if this behavior is consistent on most? _________________ Allen C. Huffman, Sub-Etha Software (est. 1990) http://www.subethasoftware.com
Embedded C, Arduino, MSP430, ESP8266/32, BASIC Stamp and PIC24 programmer.
http://www.whywouldyouwanttodothat.com ? |
|
|
Ttelmah
Joined: 11 Mar 2010 Posts: 19520
|
|
Posted: Tue Dec 03, 2019 2:39 pm |
|
|
Yes, totally.
This is actually how the virtual EEPROM driver works.
You use a small structure each of which contains the address that if being
stored, and the value to store. Every time you write a value to the virtual
EEPROM, it walks through the page till it finds the first unused location,
then writes the structure defining the new byte. If you change a value
it changes the address to a value signifying 'erased', and adds a new block
with the new value.
When it gets to the end of the page doing this it retrieves all values not
flagged as 'erased', or 'unused', and writes these into the next page then
erases the page originally being used. |
|
|
allenhuffman
Joined: 17 Jun 2019 Posts: 552 Location: Des Moines, Iowa, USA
|
|
Posted: Tue Dec 03, 2019 3:27 pm |
|
|
I just tried a more complex test on real hardware where I basically did something like this (data is a buffer of values I change it time; hex dump after each statement shows what it tried to program):
Code: | write_program_memory (0x2000, data, 1):
10
write_program_memory (0x2001, data, 2):
11 11
write_program_memory (0x2003, data, 3):
12 12 12
write_program_memory (0x2006, data, 4):
13 13 13 13
write_program_memory (0x200a, data, 5):
14 14 14 14 14
write_program_memory (0x200f, data, 6):
15 15 15 15 15 15
write_program_memory (0x2015, data, 7):
16 16 16 16 16 16 16 |
But I see when I try to do a bunch of different size writes I don't get what I'd hoped, so it does seem I need to keep our internal buffer around like our legacy code did. I was kinda hoping the CCS API was doing extra work (like they do to split up larger read/writes).
Code: | read 256 bytes from 0x2000...
0x2000: ff ff ff 00 ff ff ff 00 ff ff ff 00 13 13 13 00
0x2010: ff ff ff 00 14 14 14 00 ff ff ff 00 15 15 15 00
0x2020: ff ff ff 00 ff ff ff 00 16 16 16 00 ff ff ff 00
0x2030: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x2040: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
…
0x20e0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x20f0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00 |
_________________ Allen C. Huffman, Sub-Etha Software (est. 1990) http://www.subethasoftware.com
Embedded C, Arduino, MSP430, ESP8266/32, BASIC Stamp and PIC24 programmer.
http://www.whywouldyouwanttodothat.com ? |
|
|
allenhuffman
Joined: 17 Jun 2019 Posts: 552 Location: Des Moines, Iowa, USA
|
|
Posted: Tue Dec 03, 2019 4:50 pm |
|
|
I implemented a PIC24 flash programming routine similar to the one I wrote for an external flash part. Now I can write any values and it handles it. It will read the flash block first, then overlay the user writes into that buffer, then when done (or moving to a different block), it will write the whole block out.
The end result is I can write "proper" code that always starts on a FLASH_ERASE_SIZE boundry (to erase it first) and fill it up with FLASH_WRITE_SIZE blocks, or I can just write 5 bytes in the middle and it will read the old data, overlay my new data, and write it out when I am done.
Here is a simple example that does a series of flashProgram() writes from data from the HEX file. i.e, if there is a block in the HEX file at 0x4000, it ends up in the PIC24 flash at 0x2000.
Each step, "data" is initialized with a series of bytes of the same value, incremented at each write step:
Code: | FLASH_ERASE_SIZE: 2048
FLASH_WRITE_SIZE: 256
flashProgram (0x4000, data, 16):
New HEX block: 0x004000 -> PIC address 0x002000
--------------------------------------------------------
Copy data into block buffer (0x2000, data, 16):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
flashProgram (0x4010, data, 16):
Copy data into block buffer (0x2010, data, 16):
01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01
flashProgram (0x4020, data, 16):
Copy data into block buffer (0x2020, data, 16):
02 02 02 02 02 02 02 02 02 02 02 02 02 02 02 02
flashProgram (0x4030, data, 33):
Copy data into block buffer (0x2030, data, 33):
03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 03 0
3 03 03 03 03
flashProgram (0x4051, data, 16):
Copy data into block buffer (0x2051, data, 16):
04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04
flashProgram (0x4061, data, 15):
Copy data into block buffer (0x2061, data, 15):
05 05 05 05 05 05 05 05 05 05 05 05 05 05 05
Starting new block. Write out current block to address 0x002000 (2048 bytes)
*** Loading up the block to show that stuff got in it... ***
read 256 bytes from 0x2000...
0x2000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x2010: 01 01 01 00 01 01 01 00 01 01 01 00 01 01 01 00
0x2020: 02 02 02 00 02 02 02 00 02 02 02 00 02 02 02 00
0x2030: 03 03 03 00 03 03 03 00 03 03 03 00 03 03 03 00
0x2040: 03 03 03 00 03 03 03 00 03 03 03 00 03 03 03 00
0x2050: 03 04 04 00 04 04 04 00 04 04 04 00 04 04 04 00
0x2060: 04 05 05 00 05 05 05 00 05 05 05 00 05 05 05 00
0x2070: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x2080: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x2090: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x20a0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x20b0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x20c0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x20d0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x20e0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
0x20f0: ff ff ff 00 ff ff ff 00 ff ff ff 00 ff ff ff 00
|
Provided my employer is okay with me sharing, I'll post this when I get it complete. _________________ Allen C. Huffman, Sub-Etha Software (est. 1990) http://www.subethasoftware.com
Embedded C, Arduino, MSP430, ESP8266/32, BASIC Stamp and PIC24 programmer.
http://www.whywouldyouwanttodothat.com ? |
|
|
Ttelmah
Joined: 11 Mar 2010 Posts: 19520
|
|
Posted: Wed Dec 04, 2019 3:41 am |
|
|
If you had read my post a while ago, I referred you to 'MIN_FLASH_WRITE'.
This is the minimum size data that the CCS functions will actually write.
Not the block size of the hardware page, but the minimum the function can
transfer.
You initial tests were trying to write less than this, which is why you got
the unexpected results.
Data must also be in a multiple of this size.:
Quote: |
The write_program_memory() function can only write multiples of this size to the FLASH. Additionally, the start address passed to the write_program_memory() function must be multiples of this value divided by two.
|
For most PIC24's, this value is 4. On some it is 8.
For your tests shown, only the 4 byte write would have met this requirement.
You only need a buffer this size to actually make it work. |
|
|
allenhuffman
Joined: 17 Jun 2019 Posts: 552 Location: Des Moines, Iowa, USA
|
|
Posted: Wed Dec 04, 2019 8:55 am |
|
|
Ttelmah wrote: | If you had read my post a while ago, I referred you to 'MIN_FLASH_WRITE'.
This is the minimum size data that the CCS functions will actually write.
Not the block size of the hardware page, but the minimum the function can
transfer.
You initial tests were trying to write less than this, which is why you got
the unexpected results. |
Yes, I understand that. I was trying to better understand what happens when you don't.
Our existing code does all writes using the minimum write size in a loop. When I found you could easily read/write_program_memory() and send 4K or 16K just fine, I wondered if the CCS API was already doing what I just did.
The documentation is quite lacking in any details. Thanks to tips from you and others, I am not frequently looking at the .lst file to see what goes on.
My goal here was to make the hardware API independent, so the firmware updater doesn't have to know any of this. I wanted to be able to parse a HEX file that could put 2 bytes at any location, and have that work. Currently, our firmware process has to query the board and get back things like FLASH_ERASE_SIZE and FLASH_WRITE_SIZE and then send data in chunks that match. I plan to update it so I can pack as much in an I2C message (for speed) and have the firmware do the right thing.
I'm also giving our update process an instant 25% speed boost by not sending every fourth byte which is always 00. _________________ Allen C. Huffman, Sub-Etha Software (est. 1990) http://www.subethasoftware.com
Embedded C, Arduino, MSP430, ESP8266/32, BASIC Stamp and PIC24 programmer.
http://www.whywouldyouwanttodothat.com ? |
|
|
jeremiah
Joined: 20 Jul 2010 Posts: 1349
|
|
Posted: Wed Dec 04, 2019 4:28 pm |
|
|
allenhuffman wrote: |
Yes, I understand that. I was trying to better understand what happens when you don't.
Our existing code does all writes using the minimum write size in a loop. When I found you could easily read/write_program_memory() and send 4K or 16K just fine, I wondered if the CCS API was already doing what I just did.
The documentation is quite lacking in any details. Thanks to tips from you and others, I am not frequently looking at the .lst file to see what goes on.
|
I think the problem you'll run into here is that these things are "implementation defined" meaning the compiler can (and will if my experience is anything) change how these work in future updates. You cannot rely on them long term and designing around a short term solution isn't a good idea. Using getevn() along with those constants should give you a decent API to make it hardware independent. You should have your software decide what to do when data that is too small is sent instead of relying on what a write_program_memory() does with bad inputs. If they don't define it in their manual, I would not rely on it for a portable solution.
It most likely isn't terrible to read in some data, compare the length to FLASH_WRITE_SIZE and then decide if you want to pad it or return an error or add it to a buffer you build up or do something totally different. They are gonna do each version of write_program_memory() as streamlined for a particular chip as possible and that may mean that it acts differently on one chip vs another if the data is less than required. And if CCS comes up with a better optimization later, they may decided to change it up completely based on the new optimization.
Sorry if that sounds overly negative...not meaning to be. But I would definitely suggest sticking to documented functionality when possible. |
|
|
allenhuffman
Joined: 17 Jun 2019 Posts: 552 Location: Des Moines, Iowa, USA
|
|
Posted: Thu Dec 05, 2019 9:31 am |
|
|
jeremiah wrote: | I think the problem you'll run into here is that these things are "implementation defined" meaning the compiler can (and will if my experience is anything) change how these work in future updates. You cannot rely on them long term and designing around a short term solution isn't a good idea. Using getevn() along with those constants should give you a decent API to make it hardware independent. You should have your software decide what to do when data that is too small is sent instead of relying on what a write_program_memory() does with bad inputs. If they don't define it in their manual, I would not rely on it for a portable solution. |
Totally agree with you. This is why I wrote my routines, to ensure the same behavior on all our boards.
I used to teach an RTOS class back in the 1990s. Our compiler was a strict-ANSI compiler. I was often hearing from customers with problems with it. I'd run their "problem" past the compiler team, and I bet 9 out of 10 times they would reply with a page in the ANSI spec showing it as "undefined."
My routine lets me do this:
programFlash (0x1234, data, 3);
...and it handles it without problem on all the variants of the PIC24 we use, from the same source. It will read the block first, apply the user data, and hold on to it. It won't write the block back until the user tells it to, or moves on to a different block, so it's faster than doing a bunch of separate FLASH_WRITE_SIZE blocks. (Downside: If you really just wanted to erase and write 5 bytes, it's less efficient. I added a flag for that, but am not including any of that in the production code since we don't need it.)
I have code, which we won't be using, that checks if it's a block boundry and a FLASH_ERASE_SIZE, then can skips to the write_program_memory() with nothing else needed.
I've also added an option to skip the 25% of the HEX file that is 00s and reconstruct on the PIC size, which gave an instant 25% speed improvement. _________________ Allen C. Huffman, Sub-Etha Software (est. 1990) http://www.subethasoftware.com
Embedded C, Arduino, MSP430, ESP8266/32, BASIC Stamp and PIC24 programmer.
http://www.whywouldyouwanttodothat.com ? |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|