CCS fat.c does it work or not?

[Douglas Kennedy]

There is no internal date for versioning inside CCS fat.c but the external date is 5/15/2013 9:32pm.
First a disclosure
FAT is fully owned by Microsoft and there is no attempt on my part to infringe on this ownership or encourage others to do so. I don't own a $250,000 license from MicroSoft and I am not selling code commercially.

Now that that's out the way
I'm experimenting for my personal curiosity with the CCS driver fat.c

I have a 3.3v device 24FJ256Ga110 at 16 Mhz. The SPI is fine on the logic analyzer and will do reads and writes of byes to the card. Block size is 512.
I believe the hardware is OK.

Before digging too deeply into CCS fat.c I'm asking if anyone ever got this 5/15/2013 CCS code to work or not either with FAT16 or FAT32?

[Ttelmah]

Haven't used the recent code.
However the older code (about a year ago), did work, with the following limitations:

1) SD, not SDHC.
2) Card has to be formatted 'simply'. Basic single partition on a new card (no offsets). I found the easiest way to get a card formatted like this was to format it in my old camera.

The patent stuff is not quite what you think. A couple of other companies, Dec, and IBM also developed FAT systems, and released the data about this. Microsoft got a patent on FAT16, but this was later overturned on the grounds of 'prior disclosure'. On FAT32 Four patents were applied for, but three were refused. The one they have that remains, covers VFAT, and LFN's. The CCS driver does not support these. The only lawsuits about FAT, have all been related to VFAT and LFN's. Microsoft also published a document "EFI FAT32 specification". the basis of release in this is that the description _is_ available for general use. The parts it excludes are VFAT, and LFN's....
So Microsoft have released 'FAT32' for use, but not the stuff supporting long file names.
There was also a patent on extended attributes from IBM, but this has expired.

So, provided you don't use long file names, the file systems are free for use.

Best Wishes

[dyeatman]

With a 2GB card under Ver 5.012 it works on a 18F46K22 with no problems.
I was able to format the card, create and delete files.

It is a memory hog though. Had to switch from a 45K22 to a 46K22
to have enough RAM.
_________________
Google and Forum Search are some of your best tools!!!!

[Douglas Kennedy]

With a 2gb card formatted in a sony camera it is crashing with the dir command. Probably is corrupting memory in

[ckielstra]

I don't have the V5 compiler, but in Aug 2013 Andrewg posted two bug fixes. He was going to submit these fixes to CCS but they are not present in the v4.141 version and I wouldn't be surprised to see them missing from the V5 compilers as well.
1) A fix for the CCS library not supporting cards with a Master Boot Record: http://www.ccsinfo.com/forum/viewtopic.php?t=43402
2) A fix for the CCS "fat.c" functions "get_next_addr" and "get_prev_addr" not correctly handling cluster transitions: http://www.ccsinfo.com/forum/viewtopic.php?t=43417
3) A fix for get_short_file_name: http://www.ccsinfo.com/forum/viewtopic.php?t=43417 (see 2nd post)

[Douglas Kennedy]

Thanks for the heads up on the MBR issue.

I was expecting this would be fixed. Almost every formatted card will have a MBR and an offset to the partition record. Anyway it wasn't fixed. CCS is expecting an unusual situation where the partition info is where the mbr would be.
The CCS code for a standard card grabs garbage info like sector sizes of zero and later does a divide by zero with it. With no checks the code will very often crash.
Now I thought maybe they wanted the first time for the CCS user to format the card with the CCS format command but it also crashes with a standard formatted card.
Now if CCS chooses not to read an MBR then they should at least put a warning comment in the code to say it requires a card with a specific format.
It is true there is still only one partition ( CCS only supports one partition ( no issue)) but most formats will still create an MBR

[Douglas Kennedy]

First Aug 2013 Andrewg's bug fixes are in
fat_init in fat.c needs modifying in several places to apply the offset to fixed references
Ex.
original
ec += mmcsd_read_data(11, 2,&Bytes_Per_Sector);
replacement
ec += mmcsd_read_data(g_mmcsdPartitionOffset+11, 2,&Bytes_Per_Sector);

Note the fixes expect an active partition code 0x80... now a format will not always set an active partition... seems like 0x1be is considered as if active by windows by default

The CCS code seems to have issues....little works but slowly it is getting to show some small results
Here is a trace for a 2 gb card

INITIALIZING FAT

r1=0x01
got r1 with bit0=1
r1=0x01
r1=0x00
card type=0xE1
offset=0x1BE
offset+11=0x00
oÎfset+10=0x00
offset+9=0x00
offset+8=0xF9
g_mmcsdPartitionOffset=0x0001F200

CSD

CSD_STRUCTURE: 0
TAAC: 26
NSAC: 0
TRAN_SPEED: 32
CCC: 5F5
READ_BL_LEN: A
READ_BL_PARTIAL: 1
WRITE_BLK_MISALIGN: 0
READ_BLK_MISALIGN: 0
DSR_IMP: 0
C_SIZE: F24
VDD_R_CURR_MIN: 7
VDD_R_CURR_MAX: 6
VDD_W_CURR_MIN: 7
VDD_W_CURR_MAX: 6
C_SIZE_MULT: 7
ERASE_BLK_EN: 1
SECTOR_SIZE: 1F
WP_GRP_SIZE: 7F
WP_GRP_ENABLE: 1
R2W_FACTOR: 4
WRITE_BL_LEN: A
WRITE_BL_PARTIAL: 0
FILE_FORMAT_GRP: 0
COPY: 1
 ERM_WRITE_PROTECT: 0
TMP_WRITE_PROTECT: 0
FILE_FORMAT: 0
CRC: CB
CID

Bytes_Per_Sector=512
Sectors_Per_Cluster=1
Reserved_Sectorsr=4530
FATs=2
Small_Sectors=0
Sectors_Per_FAT=30503
Hidden_Sectors=249
Large_Sectors=3969799
Next_Free_Clust=5

Bytes_Per_Cluster=512
FAT_Length=15617536
FAT_Start=25600
Root_Dir=91136

Data_Start=91136
/> ddiirr ''//''
--/--

At least dir isn't crashing the PIC now that the parameters are being read correctly via the MBR
I'm surprised to hear others have this working out of the box.
There may be a compiler bug with the 24PICs no firm evidence so far but it sure is flaky. Ex CID trace isn't working yet
and CD change directory will crash the code

[Ttelmah]

Slowly, sounds interesting.

Obvious thoughts:
1) You have got the required pullups on the bus?.
2) Is it possibly using software SPI?. The newer code can use software SPI, and if your chip supports relocatable peripherals, it would not be using hardware, unless you have selected the pins first?. If using software, it won't change clock rate, and will stay at the 400KHz initialisation rate.

I'd suspect '2' is the problem.

Andrew's 'key fixes' have been around a lot longer that Ckielstra thinks. Andrew posted some of these at least three years ago. The ones for get_next_addr, and get_prev_addr, I used. I am astonished that CCS has not incorporated these or an equivalent!...

Best Wishes

[Douglas Kennedy]

I was using this code

[Ttelmah]

There are three different reasons for pull-ups.

The CS line into the SD card _must_ be high when power is applied, or the card is not guaranteed to initialise correctly. Since the PIC pins wake up as inputs, a pull-up is required.

Then all the lines that are 'inputs' to the PIC, need pull-ups or they are floating when the card is removed.

Finally larger value pull-ups are used to fractionally improve the rising edges. on the data lines and clock line. These though listed in the SD documentation are pretty 'optional', given the quite good drive capabilities of the PIC.....

The WP, and CD lines, are just switches. Assuming you only need to detect a card is present, and _not_ write protected, then wire CD to ground, and have a pull-up on WP. Leave the 'COM' disconnected. Then when the WP signal is pulled down, there is a card ready to go. The WP to COM connection is 'closed' when the WP switch is closed (not write protected), while the CD to COM connection closes when a card is present. Hence by using these 'in series', you get 'card is present, and write enabled'.

The clock is set in two places. First at the top of the file in the bit just after loading stdint, where the speed should be set to 400KHz. This is the _required_ initialisation speed for SD. Then later in the routine in mmcsd_init(), where the faster speed is selected. By overriding the MMCSD_SPI_HW define, your code won't be setting the initial rate to 400KHz. Might well cause problems (most modern cards will initialise at more than 400KHz, but this is 'luck')....

Best Wishes

[ckielstra]

[Ttelmah]

Agreed.

I had the CCS code 'running', with the fixes for quite a while OK. Then wanted to add SDHC support, looked at it, and decided it was going to be easier, quicker, and basically cheaper to buy Andrew's library. With a few 'tweakettes', it ran straight away, and has worked ever since.

Far more people have problems caused by the 3.3v to 5v interfacing, than the library. It is very much worth verifying that you can read and write sectors right across the address range of the card, _before_ starting to try FAT.

I somewhere have an attempt I started at porting the Elm library. It was looking hopeful, but got 'filed', because time was more expensive than the Brush library...

Not quite 'all'. There are a few (very few) 'transitional' cards around, that are SD with capacities over 2GB. A couple of companies launched cards before the standard was generated. The annoying thing is if you have a camera that uses these, since they don't support SDHC.

Best Wishes

[Douglas Kennedy]

They often say a project is a journey not a destination
The issue with the CCS fat.c is the failure to type cast.
There are a number of smaller errors I'll post them soon after I fully test the features.
Below is patched working code that still could use more mathematical consistency

[Douglas Kennedy]

As a retired mathematician I once oversaw hundreds of coders so I have some awareness of coders
Many know coders love to use pointers but pointers often have issues and often don't form what mathematicians would call an algebra. Fat32 is a labeling of the storage space it doesn't impose a metric. Labels are mbr, partition record, fat table, cluster number, file name tile etc. There is no metric just labels and relationships. Something like mbr is (a) partition record is (b). a<b is true in this algebra without having to apply any metric ( bits bytes words blocks etc.) It is true distance relative to storage memory may be quantized into bytes, words, double words or 512 byte blocks but the algebra shouldn't care. Labels have relations to each other in the algebra (the mbr is before the partition record ) but the physical distance is imposed by the card geometry; the algebra is independent.
The algebra t is like drawing points on a line but not placing a grid. The points are just labels that have relations with each other. When a label needs a metric a transform table should be used based on the cards physical geometry. This places the math notationally above the card geometry. Like as if SD cards are a rubber strip with labels (mbr etc)...they differ by the way the strip is stretched between the labels and the maximum stretch possible ( capacity). The relationship between the labels is conserved no matter how the strip is stretched.

Pointers impose a specific metric 8 bit jump for a byte 16 bit jump 2 etc. In places the pointer is even cast down ....I'm living with them for now in this CCS code.


The weakness of pointers is that they can point to unlabeled points that are meaningless in the algebra (Ex. like some meaningless location within in the mbr... the CCS code doesn't do this but it is a good example of the risk pointers impose) and it is the source of the bugs.
CCS did a good job of isolating (abstracting) the low level SPI card driver. Coders often do well with hardware abstraction.
Now luckily object programming is imposing an algebra on coders. Objects have relations with each other and the compiler decides the physical location of the object in the metric of the memory space.

Anyway with several patches I can read the contents of my files from my SD card which is all I need and keeps me on firm ground w.r.t. Microsoft patents ( my file creation ,writing etc. is done only with Microsoft licensed products). I know non lawyers have offered broader opinions on this issue. This CCS code maybe useable for others but it is non only for me partially useable. I would advise not to use this CCS code if an alternative is available.