Noise Is 3 Orders of Magnitude Greater Than A Wave Form

Bret apparently has decided he knows physics and signal processing better than all of us. I have no idea why he is posting since he claims we are all wrong. He seems to think the laws of physics do not apply to him.

Yes, certainly. *If* you have a suitable noise-free frequency band available, this is quite easy to do. No revolutionary techniques are required.

Your example assumes that you do indeed have such a band available, at about 100x the original signal frequency, and wide enough to accommodate the bandwidth of your signal.

In that case, the problem is easy to solve. Just modulate your signal onto a carrier somehow (you're using AM, but you could just as well use FM or some other technique), use a filter at the receiving end to separate the carrier from the noise, and then demodulate.

Although it might not seem that way, this *is* what you are effectively doing. You're just using a somewhat unconventional technique to do the filtering and demodulation. There may be simpler ways, e.g. instead of sampling the zero crossings to cancel the noise, just sample the peaks and troughs of the carrier to measure its amplitude.

You don't strictly need to know the exact frequency and phase of the carrier, although if you do happen to know it, you can take advantage of that to simplify any sample-oriented processing you want to do.

What others are talking about concerning averaging is what you need to do if you *don't* have a noise-free band available, and you've no choice but to deal with signal and noise together in the same band. In that case, the only way to reduce noise is to reduce the receiver's bandwith *somehow*, and averaging is one way to achieve that -- and knowing the frequency and phase of your carrier is a big help, because it lets you implement a synchronous detector.

But as I said, you're assuming that you *can* find a noise-free band, so you don't need averaging (except maybe over a few cycles of your carrier frequency, which is much higher than your signal frequency).

Note that synchronous waveform averaging is *not* a filter, and does not affect the bandwidth of the recovered signal in the least. You still get sharp transients, for example, if they were present in the original.

Synchronous waveform averaging will reject any signal that is not synchronous with the trigger, not just random noise.

Best regards,

Bob Masta DAQARTA v4.51 Data AcQuisition And Real-Time Analysis

Scope, Spectrum, Spectrogram, Sound Level Meter FREE Signal Generator Science with your sound card!

He seems to have some idea for a gedget that needs synchronous detection to do some measurement. But of course he'll never reveal what he's actually trying to do. And he won't do any research on how everybody else has been doing it for 60 years or so.

John

Not everything.

Great as long as you don't do that to the noise, too.

John

That's also true for the squared higher frequency sin wave method of subtracting noise. A conventional filter doesn't determine the amplitude of the noise and then subtract it. A conventional filter just attenuates noise above or below a certain bandwith.

If the noise is about the same frequency as the signal, then a conventional filter will not work.

Filters can toss information as well as noise.

The higher frequency sin^2 method doesn't require an average. In one application all it takes is one period of ["sampling" by] the high frequency wave to determine the noise as well as the signal.

Using regression and integrating over even part of the period of a smooth low frequency noise wave would yield very high precision, even if the "sampling" rate was fairly low.

Can you reduce smooth well behaved low frequency noise by 99.995%?

Bret Cahill

The situation is so common and the applications so widespread and the solution such a short distance "off trail" I was assuming they already had 10,000 dirt cheap off-the-shelf versions of it as they generally do in electronics.

That was the reason for the OP. Someone was supposed to answer, "TI sells exactly what you want in their 87WA3 series. Anyone can wire 'em up in 5 minutes. I used them for blah blah blah . . ."

I was expecting / hoping for more responses like the

guy's.

It was only later that I came up with the sin^2 method and that was presented mostly to clarify the OP, not to answer it.

Or the squared modulated signal.

For lack of better terms I used/abused the AM/FM analogy on the two basic solutions that led me here in the first place. The one I was calling "FM" circumvents the noise issue altogether and has an easy to enter niche market but the "AM" one has a noise issue but is much cheaper and has a much larger market.

So I had the FM/AM mindset too but true/conventional AM doesn't square the high frequency wave.

This is a critical difference.

So I have until July 2, 2010 to file a PPA.

Actually that would _not_ work.

My background isn't signal processing but now I'm starting to wonder if the hold up behind the larger solution might be traceable to a hold up in electronics.

Some California physicist suggested it was impossible to destroy information and recently S. Hawkings agreed. Shred and burn your notes then put the ashes on a rocket headed toward a black hole and

10^68 years from now they'll know what you were trying to hide. The same holds for DNA and your life experiences so everyone is in effect immortal.

That's one very good reason to believe efforts at reducing noise / getting information will be fruitful. I don't want to wait 10^67 years for it but then again, I'm not trying to destroy it either.

Bret Cahill

I'm still expecting some scholar to say this was done in the early

1880s.

Bret Cahill

Actually in 1918 by Edwin Armstrong.

Many thanks for the tip but phase lag is just a more sophisticated form of filtering which is valuable in many situations where the noise is all over the spectrum.

This is not filtering noise; it's measuring it then subtracting it.

Basically the signal is brought to zero to identify the noise. Then the noise is subtracted from whatever the receiver is putting out.

For an accurate signal measurement both the noise and the noise + signal output from the receiver must be known to a higher accuracy.

Integrating should yield that higher accuracy but it isn't always necessary as it can work over _one single higher frequency wave cycle_.

I ran it by a lawyer and he said it was completely legal.

Bret Cahill

Amplitude of the high(er) frequency wave is modulated by the signal curve.

Put sinxsin^2(10x) into

to see the difference with the common AM radio signal.

Bret Cahill

A lockin doesn't work by phase lag; it works by correlation.

That only works if the zero+measure thing is done at or above the noise's Nyquist frequency, and that is in turn only meaningful if the noise is bandlimited.

And you are able somehow to turn the signal on and off at that rate.

So all you need is a highpass filter. But the math algorithm you describe is about equivalent.

I thought you *were* a lawyer.

John

Who cares and it is irrelevant to what I posted.

But your method doesn't completely eliminate the noise either. You're approximating it by interpolation, and assuming that the result is "good enough".

There will always be some residual error -- and that error is just the same as the residual amount of noise that a conventional filter with equivalent passband characteristics would let through.

So can your method, since you're assuming you can interpolate between signal samples as well. To the extent that's not exactly right, you've lost information.

Just a word of caution about the general scheme of subtracting noise: I suspect that variations on this have been re-invented repeatedly, since it sounds like such a great idea. The problem is that when you actually go to implement it you find the big "gotcha": It totally depends upon perfect phase and amplitude matching. If you subtract a version of the noise with a slight time or phase or level shift, the performance goes downhill drastically, and can even be worse than the raw signal. In other words, it will be very hard to make this work as a robust system.

Best regards,

Bob Masta DAQARTA v4.51 Data AcQuisition And Real-Time Analysis

Scope, Spectrum, Spectrogram, Sound Level Meter FREE Signal Generator Science with your sound card!

It's surprising that a quick way of explaining why it won't work hasn't appeared as is generally the case in electronics.

Certainly some scholarly type somewhere has a comprehensive list of signal processing theories and equipment.

That's not an issue if a smooth noise has a period several times longer than the higher frequency wave. The higher frequency wave and the signal that modulates it would clearly appear w/o any synchronization.

Put 5sin(1.2x) + sinxsin^2(5x) into

The original signal sinx can be easily retrieved without knowing the phase or even the exact frequency of the modulated wave.

The obvious problem with the subtracting approach is the sensor must be much more precise to get a good measurement from the difference of two large numbers. If you wanted to be +/- 1% accurate and the noise was 100 times larger than the signal, then the sensor would need 4 sig fig accuracy.

The period of the noise is so long, however, the sensor could rezero itself just before and just after each signal measurement.

Can you eliminate 99.9975% of noise?

Bret Cahill

The noise frequency is limited to about +/- 50% - 75% of the signal frequency.

You'ld lose the signal with a simple filter.

The signal is transformed to something that has a lot of the characteristics of the original signal. For example, the integral is

1/2 the integral of the original curve.

The advantage is the transformed curve plots out the difference between the noise and the transformed signal. Then the noise is subtracted.

That is not a conventional filter.

Bret Cahill