Intermittent bit shifts MCP23016

[JerryR]

Hello:

I have been having a hard time tracking down an intermittent bit shift occurring in a project using a Microchip MCP23106 port expander. I would appreciate the group's critique of my code. The shift occurs from a lower position to the next upper position (example: bit 0 will shift to bit 1). Most always occurs in GP1.

The code is pretty simple. I am using a PIC18F4520 and compiling using v3.249 PCWH. The RTCC interrupt completes in approximately 150us. The calling routine, among other things, implements a PWM on four heaters. There is another interrupt active (RDA).

Has anyone seen a similar problem? Is there any way to "harden" the code?

Thanks for the group's help.

I2C routines:

[Ttelmah]

Presumably you mean 23016, not 23106.
First, get rid of the division by 0.005 in the interrupt routine. Multiply by 200 instead. Division takes significantly more time than multiplication. You will probably halve your interrupt time, by this single change...
Then get rid of the delays between your I2C instructions. These are not needed.
You don't show your I2C setups. Are you running in 100KHz mode or 400KHz mode?. What pull up resistors are you using on the I2C?. How long is your bus?. How are the heaters driven?. How is the PIC grounded?. How is the PIC's supply derived?. What smoothing has it got?. What speed is the RS232 driven?. Have you tried without the serial interrupt enabled?.

Best Wishes

[PCM programmer]

As an experiment, disable interrupts while doing the i2c operations.
This will tell you quickly if the interrupts are causing the problem.
Example:

[JerryR]

PCM: Yes! I'll give that a try. It extremely hard to see this happen. The controller is at a remote site with no one there that can test...just complain

Ttelmah: Yes, I meant a MCP23016, sorry for the tpyo

To answer your questions:

[Ttelmah]

How many 23016's?.
By 'default', the bus should be at 100K. Now the spec for the pull-ups, depends on the total capacitance present on the bus. With 4K7, at 100K, you have a maximum specified capacitance of about 240pF. You should be inside this, if the PCB is reasonably designed, but you might want to try lower pull-ups (minimum is about 1.8K for a 5v system).
Are you really confidant about your supply noise figure?. Unless the system used to measure this has a frequency response well into the GHz region, you may well have high frequency components that are larger. You don't mention what the smoothing is on the PIC supply. You want at least one capacitor that has really good high frequency performance (ceramic for example), really close to the PIC. Also similar close to each of the I/O expanders. Do the I2C lines run close to anything else?.

Best Wishes

[JerryR]

Ttelmah:

Again, thanks for your input. Yes, I am going to try running the I2C bus at STD (FAST=100000) and see what happens.

To answer your other questions:

MCP23016 - three on our design.

Yes, we do use 4.7K pull-ups and I could try lower to "help" the edge integrity (faster rise time).

I like good bypassing. We are using a combination of a ceramic 0.1uF and a low-ESR tant. on the PIC ( on the edge of the device at Vcc ). I also use 0.1uF ceramics on each port expander Vcc pin place close to Vcc. I haven't looked at the power supplies with a spectrum analyser, but that may be a good next step. Using a 250MHz scope, I see only 9mV p-p currently in a low electrical noise environment. We are working in a high electrical noise environment. The problem has only manifested itself in actual use in the high noise environment.

Again, thanks for your assistance.

We don't use the I2C except fr the port expanders

[PCM programmer]

Also, as a test, try software i2c. Remove the FORCE_HW parameter.

[JerryR]

Thanks PCM. I'll include this and test it.

[JerryR]

Apparently the MCP23016 is extremely temperature sensitive. Holding a heat gun near the device causes the bit swapping malidy. The external timing circuit is not as sensitive to heat, however.

We have done a fair job isolating the effects of the heat gun as being heat rather than EMC (created by the heat gun).

Does any of this sound familiar to anyone out there?

Thanks!

[Ttelmah]

Are you really _sure_ it is not the external clock circuit. Have you actually programmed IARES to 1, and tested on TP. I'd be most suspicious of this. How close to 1Mz is your clock?.
Have used these chips on a couple of units, that have been put through full environmental testing (soaked at temperatures between -30C, and +70C, for several days, including fast cycling), without problems. Presumably yours are 'I' temperature marked (have never seen any that are not).

Best Wishes

[JerryR]

I know, it does sound strange. I've got a technician measuring TP and he's getting 1MHz +/- 10KHz at room temp and at heat gun temp (+40C). I sure would not have suspected a heat problem either. More later after more testing. Thanks for you continued interest!

[PCM programmer]

In the typical case, if you heat up a chip and it fails or becomes flakey,
there is a problem with:

1. The timing of the input signals to the chip may be wrong.
Look closely at the setup and hold times of the data to the clock.
Are they below the minimum requirements ? Look at the clock
frequency. Is it too high ?

2. There may be noise pulses on the board's Ground and/or Power
traces (tracks). When you heat up the chip, its input logic threshold
will get lower. This makes the chip more susceptible to noise.
If the Ground pin for the chip has noise pulses on it, you will often see
a problem as the chip heats up.


Any time that you have a flakey board and you can make it worse
by putting a heat gun on it, or make it better by freezing a chip with
chill spray, you should be looking at the things listed above.

If it's a small little chip that doesn't heat up by itself (normally) and its
not a defective batch, then I wouldn't jump on the idea that the chip
is the problem. (Unless you're heating it way out of spec). I would
look at the problem areas above.

[JerryR]

Problem solved. It turned out not to be a temp issue with the MCP23016, but a temp issue with a
T.I. 75176 RS485 driver / receiver close to the MCP23106. Communications to / from our controller from
a connected PC were being corrupted. Substituted equivalent LT or MAXIM devices for the TI, problem
went away at temperatures up to approximately 70C. TI devices began corrupting data at approximately 35C.
Entire lot of TI devices failed at lower temperatures.

Also hardened software with PCM and Ttelmah suggestions. THANKS GUYS! Also added a bit more robust
error detection on communications.

Again, thanks for the support group.

[treitmey]

I use that chip. The TI 75176. Can you tell be the Rev of silicon it was from?

I haven't seen the errors you describe,.. tho my protocol may resolve some
problems,.. But I may consider changing chips.

I'm going to write a simple test loop to see if I find anything similar.
I wonder if it has to do with its "thermal shutdown" at 150°C
tho it is characterized for operation from 0°C to 70°C.

[JerryR]

Treitmey: I didn’t mean to alarm everyone on TI’s 75176, this is probably an isolated batch. I’ll find a date code and send it to you. The temperature tolerance of the devices we tried was pathetic. The “flakiness” began around 30C-35C, no way the junction could be anywhere near 150C (or even 70C for that matter). We utilized two of these devices on a total of ten boards. All exhibit the same malady.

More later. Thanks