Multi channel data acq & SD card storage using PICs

[ranjith_asp]

Dear all,

I am a newbie wrt interfacing 2 or more PICs. I need suggestions for choosing the PICs and other ICs, as per the requirement of my intended application.

My app req brief is as follows:
1. To acquire around 32 Analog Channels (0-5V range) - external ADC & MUX on the same PCB is envisaged.
2. Acquisition freq to be around 500Hz (min).
3. To store the data in SD card or Flash chips.
4. Data to be stored over a time of 48-72 hrs.
5. System will be not accessible to outside world and will be battery operated (Lithium ion based).
6. After said time is over, the system to be connected with a PC via a cable to download data (removal of SD card and using in card reader is to be avoided).

Kindly suggest me:
1. Whether a single PIC can acquire data, store data and transfer data to PC.
2. If more than one PIC is to be used, kindly suggest which PIC to be used for what purpose.
3. How to interface them. And so on.
4. For starting, I would like to simulate the system in Proteus 8, so suggestion of components available in Proteus library would be of great help to me.

Once I get expert suggestions on component selection, I would like to follow the road map as below:
Step 1: Simulate PIC with ADC to acquire 32 analog channels (store them accordingly to analyse in excel or something of that sort)
Step 2: Simulate PIC with SD card and make the PIC store the file in SD card - to be copied and analysed for results
Step 3: Simulate PIC interface to download data from SD to PC - data downloaded to be analysed for results

I would like to update the thread often to make this a benchmark thread for any newbie to become a novice in creating a system with all the features I have envisaged.

Many thanks in advance and wish me good luck..!!
_________________
Have a nice day..!!
Ranjith

[Ttelmah]

Seriously, start with intending to use Proteus/Isis. Don't....

Most of the people here who design electronics commercially have used/tried Proteus, and most have found that it will waste time. You will design a project, have Proteus tell you it can work, and then have to re-design it when you actually go to the real world. We had a time/motion company do an analysis of whether this package helped for the design phase, and their conclusion was that designs using it, on average took significantly longer than those that started without it. I do use Proteus as a PCB package, and will use it as an analog simulator, as an alternative to using older things like Spice, but have given up on using it with the PIC...

No one here will help you with Proteus.

Then be careful you are not wasting time 're-inventing the wheel'. There are several 'off the shelf' data-logger units that could do what you describe (though you miss critical things in your description like ADC resolution etc..).

What you describe, is easy for a single PIC. Something like the 18F26J50, can do this easily, and being 3.3v, is much easier to interface to the SD card. Remember if you are handling the SD card properly, you need to have a way to 'stop' the acquisition and complete all writes, before ejecting the card.
Add something like an DDC232 (remember this is a current converter, not voltage), and the protection on the inputs (vital), and you are basically done. The chip has USB to talk to the PC.

You miss mentioning that you may need to 'timestamp' the data. For most acquisition jobs, the time at which the sample is taken needs to be stored with the data. Then be surprised at just how long downloading this all to the PC will take.

Then separately, things like SD and flash, all take significant times to write, with pauses in this speed. Beware.

You'll need to be using an HC card (too much data to store on a standard capacity card). Means you need drivers for this (code library).

Then be prepared for the total power involved. Things like SD cards, take much more power to write, than just sitting idle, or reading. Operational current when writing, for even a 'low power' card, will typically be something like 150mA. Double this to cover the other electronics, and for 72Hours, you need something like a _minimum_ 22Ahr battery. Allow at least 50% more, if the temperature is low.

[guy]

Hi ranjith,
Ttelmah is a real pro and is raising important issues but he may have been in a bad mood things are not as bad as they seem...
You need 32 channels x 500 samples per channel = 16k samples per second. If you can do with the PIC's internal A/D you will get 10-12bit resolution and approx. 50-100Ksps which may be enough. 0-5V can be divided with accurate resistors to 0-3.3V (and calibrated if necessary).
16ksps * 2 bytes per sample is 32KB/sec, the write speed that your flash memory needs to support. As for timestamps, if the sampling rate is accurate and constant (for example exactly 16ksps) you may neglect the timestamp as it could be calculated using the sample number. Total memory would be 72 hours x 60min x 60sec x 16ksps x 2bytes per sample = ~8GB
Uploading 8GB to a PC would take 192 hours @ 115200 bps (suitable for zen masters...) so the way to go is definitely with USB. So try finding a PIC that would also support that.
If your sampled analog data is relatively constant or could somehow be compressed you may consider some kind of real time data compression. Sounds complicated but it could be just a few lines of code to significantly reduce data size, transfer rates, etc. Especially since during the recording the PIC is 90% idle.

I never used Proteus so I have nothing to add on this.
Your approach to help other newbies is remarkable. I'll help as much as I can (although we might both end up needing Ttelmah's support )

[Ttelmah]

With USB, the PIC18 using bulk mode can push perhaps 5-600K per second reliably over USB. For 6GB, about 2.5 hours to transfer to a PC.
Ouch.....

Much easier actually to use the SD card, and plug it into a card reader.

However look at some alternatives/comments:

1) Add a Vinculum to any PIC and write to a USB stick, rather than to SD. Result a simple USB stick that can be put into the PC, and no need for the PIC to do the work of handling the memory card.
2) Use something like a Raspberry PI for the acquisition. Same comment can write directly to a USB stick. Power consumption on the current releases (B+)is quite good.
3) If only 8 bit data is needed data involved halves.
4) 500s/sec*32 = 16000 samples per second acquisition and reading. Remembering the memory I/O involved to store the data, this needs to be 'off chip' to have much hope (or on a DSPIC, with DMA). Hence the suggestion to use an external SPI chip that will automatically sample the data at programmed intervals....

[guy]

[temtronic]

couple of points...

Even designing a PCB for 8 bit ADC is challenging, 10 or 12 bits needs a LOT of very,very careful, well thought layout and EMI reduction ! Trace thickness, grounds,bypass cap placement, all this and much more is required to achieve any level of sucess. In my former life I designed/built 32 channel/16 bit ADC 'frontend' for very 'noisy' optical emission spectrometers. NOT for the beginner!!

data collection. As has been pointed out a 'Viniculum based' USB flash drive is probably the best approach. Since you need to transfer to a PC(presumming a laptop ?), it's far faster to pull the stick, replace with new one, and go home to 'deal with the data'.IF you have the PIC store is dotCSV format,then uploading to excel is super easy !!

power. 3 days, lots of reliable power is needed. I'd be using a good 'car battery'.

Jay

[Ttelmah]

Agreed wholeheartedly.

The battery is arguable. The problem with Lithium Ion batteries is they need very careful supply designs to be reliable, but they can give up to about 5* the power density of the lead-acid. The Lead acid will win massively on cost.
Once you have seen a few Lithium batteries fail, you become more careful about how you use these. This was the big failure a couple of years ago on the Boeing 787. They went for them because of the better energy capacity.

[ranjith_asp]

Sorry everyone for my delayed response.
I carefully read everyone's reply, and I guess I can fairly understand the flags raised.

But since I have no working knowledge, I think it is best to try things out, on the basis of what you all experts have advised.

I am going to try out my Step-1: To acquire 32 analog channels and store it in PC
Analog range: 0-5V
ADC resolution: 8-bit would suffice (decided after investigating the sensors in my application).
Sampling rate: 500Hz per channel.
Microcontroller: PIC18F26J50 - 28 pin SPDIP (going for a breadboard implementation first )

I will wire the schematics in Proteus and will post it here for comments. Request you all to tell me if I am heading north

PS: Imagine the entire system is confined inside an enclosure with the box having some ports for communicating with the outside world. I do not want to open the box to remove the USB stick and copy the data. Instead, through the port, I either need high speed data downloading OR to connect the USB stick with necessary protection and cabling, so as it connects to both PIC or PC - not simultaneously. After data acquisition, I would turn off the PIC board, and will connect the USB stick through the port to my PC to "copy" data.
_________________
Have a nice day..!!
Ranjith