Extremely slow compilation on network drive

legion · Joined: 10 Oct 2007 Posts: 23

Working with files on a network drive (ReadyNAS Duo, Raid configured) the compiler takes several minutes compared to a couple of seconds to compile typical sources. I don't think such a huge difference is down to the access time over the network as the files involved are all just a few kB and I don't normally notice any difference loading much larger files off the NAS or my local HD.

I have the NAS mapped on WinXP as drive Z: so all the paths look sensible e.g.

Z:\\Work\PIC\Project1\main.c

Does anyone know the reason/fix for this issue? Obviously I can work off my local drive but the Network server gives me security and access options not available on my PC.

temtronic · Posted: Wed Oct 26, 2011 8:11 am

just thinking out loud..

Could there be a lot of other 'action(traffic)' on the server that you're not aware of ?
Any chance some 'buffers' aren't large enough to transfer the data in one chunk?

Does this happen 'randomly' or at predefined time windows? Server could be doing something(updates,defrag,etc.) in the background, chewing up machine cycles.

Can't see any 'security or access options' that are better on a server compared to a local PC. I do all my work on the 'local drive',then upload the modified files to the server's HD for backup.

Ttelmah · Joined: 11 Mar 2010 Posts: 19341

I must say, I have not seen this as a problem. Compiling on a virtual PC, running inside a Mac, using a network drive for the files.
Is this with the Windows IDE, or using command line compilation?. What is the 'path' statement set for on the PC?. Some of the stuff (config files etc.), is looked for in the PATH, before the compiler looks in the defined locations in the project. I'd suspect some sort of issue with this setting resulting in a timeout, followed by finding the real files.....

Best Wishes

legion · Joined: 10 Oct 2007 Posts: 23

legion · Joined: 10 Oct 2007 Posts: 23

Ttelmah, your setup sure makes mine seem trivial, so it ought to be resolvable!

My Path variable has "C:\PROGRA~1\PICC" in it, was that what you were expecting? I've used the Complier from the command line, from within MPLAB and ICEPIC and get the exact same result every time. (PCM 4.122)

RF_Developer · Joined: 07 Feb 2011 Posts: 839

I've got this problem too. I've found it takes about 1minute or more to compile code that takes 10secs tops on a local drive. I've found that the slow down is at it worst when the compiler is writing the symbol tree (.tre) and the listing (.lst) to the network drive. I've disable writing the tree by unchecking Options ... Project Options ... Output Files ... Call tree file . That helps in my case. I haven't found a way to turn off the listing, though I use that pretty often, unlike the call tree, which I've looked at perhaps only a couple of times over the past year. Ironically, I spent a lot of time getting the Microchip C18 compiler to even produce me a listing from MPLAB!

My reason for using network drives is for souce backup reasons. There's little point in all the output and diagnostic stuff going there too. Only once I've fully debugged is there any need for me to write hex files, and hex files ONLY to a network drive for distribution and archive purposes.

I also don't like clogging up the network nor the server with this sort of nuisance traffic. It doesn't need to be there. The source needs to be there (though I'd MUCH prefer to be using a decent source control system and compiling from working copies on a local drive) but the other stuff is basically junk with a short shelf life in the sense that it's only valid until the next compilation.

The real problem as I see it is that you cannot redirect the intermediate and output files to a local drive. All the files, source, intermediates and output have to be in one directory. I've tried setting up the project on a local drive and pointing it to source on a network drive, but the IDE doesn't realise that the two are on different drives and it tries to construct a relative path to the source files, with disasterous consequences ;-( not to mention verrrrry loooong yet totally invalid path names.

RF Developer

Ttelmah · Joined: 11 Mar 2010 Posts: 19341

Ah. So you have the compiler 'locally', and the data stored on the network?.
I installed the entire compiler on the network drive, so the path variable is talking to W: (not Z: for mine....).
I wonder if that is the difference. The compiler is on the same drive with the code (not the same directory).
I was thinking you might have had the PATH pointing to a local compiler directory that didn't actually exist.

Best Wishes

legion · Joined: 10 Oct 2007 Posts: 23

PCM programmer · Joined: 06 Sep 2003 Posts: 21708

MPLAB has the same problem. See the last post in this thread:
https://www.microchip.com/forums/tm.aspx?m=351076&mpage=2

legion · Joined: 10 Oct 2007 Posts: 23

Ttelmah · Joined: 11 Mar 2010 Posts: 19341

Interesting.....
It is better on the virtual PC in the Mac, than on a real one!.
It is also better with the compiler on the network drive.

Took a medium program, and compiled it locally on the virtual PC. 7.45 seconds 12000 lines. Same program, compiled it on the network machine with the compiler also on the network drive (as I normally run). 9.23 seconds. Then put the compiler locally, got slightly worse, to 12.43 seconds. Then ran up a W7 ultimate machine with similar hardware specs (I7, 12GB RAM), and tried this with the networked data 32.5 seconds!...
Ouch.
Also when browsing to find the file to compile, noticed that the actual update when you select a directory is slow from the Windows machine. Have never seen this from the Mac.
There is obviously something fundamentally wrong with the network handling, which is at least in part 'hidden' by using the virtual machine (it is the Mac that handles the networking here, the drive is seen as local by the virtual PC).

Best Wishes