Filters (long)

[SteveS]

It seems that filtering comes up pretty often, so I thought I'd write up some real basics on digital filtering (please remember this is basic - there are exceptions to everything I say here):

Filters are generally used to remove unwanted frequencies from a signal and can be low pass (LPF-remove high frequencies - usually hf noise), high pass (HPF-remove low frequencies - usually DC), bandpass (BPF-remove high and low - you are looking for a particular frequency), or notch (remove a small band of frequencies - like 50/60 Hz line noise). I'm going to use LPF here as they are the most commonly needed.

Digital filters take discrete samples of a signal and use the new and past inputs and past outputs, each multiplied by various constants, and summed to produce a new output.

Digital filters can be further subdivided into two major groups: Infinite Impulse Response (IIR- also known as recursive) and Finite Impulse Response (FIR). IIR filters are usually designed from an analog filter with the desired characteristics. Theoretically they have infinite memory of past inputs (hence the name); in practice it isn't infinite, but they do recover slower from an impulse (spike) on the input than an FIR filter. However, they generally require less calculations to achieve a particular filtering level (order). FIR filters only use the new input and some number of past inputs to calculate the new output, so a spike is 'forgotten' once it is old enough to not be included in the calculation. FIR filters are very versatile but generally require more calculations than an equivalent IIR filter.

There are many software packages available to generate the equations for a filter based on your requirements. However, the PIC doesn't really have the ability to do very complex filters, so don't go overboard! I've listed a few basic filters below that are easy to implement and use and require little knowledge about filtering.

Be warned that many filters may cause overflow since they rely on adding up several values then dividing. So if you use an eight bit converter be wary of using eight bit math. 10 or 12 bit converters and 16 bit math should work in most situations.

==== Note: I use the following abreviations: ===

Out0 = new output
Out1 = last ouput
Out2 = next to last output, etc

In0 = newest (current) input
In1 = last input, etc

a, b, c etc, are constants

===== Low Pass Filter =====

A really simple low pass filter that approximates a single pole (R-C) analog filter:

Out0 = a * In0 + b * Out1
where a + b = 1.0 (a and b are fractions > 0 and < 1)

This is an IIR filter since it uses past outputs. Basically it averages the new input with last output. The smaller the 'a' term is, the slower the response of the filter. If you use constants divisible by a power of two it is a very fast calculation. A good starting point is to set a = 1/4 and b = 3/4:

Out0 = (In0 + Out1 + Out1 + Out1) / 4;


===== Averaging Filter =====

A simple FIR filter. This takes the average of several inputs to produce the output. In theory you would need past and future inputs but you can't, so you get a fixed time lag in this filter of half the number of samples used.

Again, if you use a power-of-two number of samples, the calculations are faster:

Out0 = (In0 + In1 + In2 + In3) / 4;

You will find you will probably need more than just four samples to get much filtering. You can also shape the filter by multiplying each input by different coefficents, but it complicates the calculations and can introduce worse truncation and/or overflow problems.


===== Median Filter =====

A type of FIR filter with very interesting characteristics. A median filter takes (generally) an odd number of inputs (5 - 7 is a good starting point), sorts them by value and takes the median (middle) value as the new output. This filter is extremely effective at removing noise spikes that last only a few samples. At the same time the filter will faithfully pass step changes in the signal. Other filters round off steps. The median is not as easy to code as other filters. The data must be sorted each time;, consequently it doesn't have the flow of the other filters. On the positive side, the median filter will not overflow.


===== Other filters =====

Just for completeness sake, let me add one of my favorites: the Lock-In Amplifier, or Demodulation filter (or other names). This is a special bandpass filter for modulated signals. If you can modulate the sensor at a fixed frequency, then you can use that same frequency to effectively create a very narrow bandpass filter. These filters can pick out signals that are completely lost in noise. Do a Google search for more info.


So, which to use?

- The IIR LPF is simple and effective and is good at removing general noise from a signal.
- The FIR can be tailored to most anything, but requires more code. It's great if you need true averages. It would also be good for something like a weigh scale, where the input data jumps around (with each new weighing), but needs to averaged during each weighing.
- The median filter is when the data gets a lot of short duration spike noise. Some sensors have outputs that behave this way.

Feel free to add to (or correct) this thread.

- SteveS

[SherpaDoug]

[SteveS]

corrected -thanks!

[SimpleAsPossible]

Out0 = (In0 + In1 + In2 + In3) << 2;
and
Out0 = (In0 + Out1 + Out1 + Out1) << 2;

should be

Out0 = (In0 + In1 + In2 + In3) >> 2;
and
Out0 = (In0 + Out1 + Out1 + Out1) >> 2;

to divide the sums by 4 rather than multiply.

Or, if you prefer a slightly clearer (to the non-C minded) version:
Out0 = (In0 + Out1 + Out1 + Out1) / 4;

[SteveS]

DOH!

I do that all the time. I also do this:

x>>2;

thinking

x= x>>2;

Fixed.

[Neutone]

In attempting to filter a 10 bit ADC and produce a 12 bit output signal I found I could get a good result using a different version of the "Low Pass" filter SteveS mentioned. With the elements of time and oversampeling the signal output really is more than 10 bits for a steady input. Even with oversampeling the output is not going to get much more than 2 added bits of resolution. Using a variable voltage input you can actually adjust the voltage to produce almost every 12-bit value. For the values of zero and full scale input the filter produces an error of 2 bits.

A simple version that runs very fast is this.

[Storic]

[Ken Johnson]

Can't take credit - I read this somewhere - but I use this digital exponential filter a lot:

Value += Filter * (newValue - Value);

All variables are floats (calculation is still pretty fast). You can adapt this to "int" types if time is of the essence.

Filter <= 1.0 (smaller Filter gives more filtering).

Works great!

Ken

[mthornton]

Value += FilterK * (newValue - Value);
where FilterK <1
this works great, but requires floating point variables & math
---------------------------------------------------------------------

so do this instead
multiply FilterK by 10 (I often use .9, so make =9)

Value = (10*Value+(9*(newValue - Value)))/10;

works fine on a 10 bit value using unsigned int16 variables

Also... Try playing with multipliers > 10
(works good x50). Not much higher or you overflow 16 bit unsigned values when working with 10 bit raw a/d newValue

[alan69]

Great thread thanks. Here's an code example for a median filter:

http://www.electronic-products-development.com/embedded-programming/filters/median-filter
_________________
Alan
www.electronic-products-development.com