Noise Is 3 Orders of Magnitude Greater Than A Wave Form

Everything is known about the transmitted wave, i.e., the shape & phase angle, except the amplitude.

All that is necessary is to recover is the amplitude of the wave. Can this be done when the noise is several orders of magnitude greater than the signal?

Bret Cahill

Reply to
Bret Cahill
Loading thread data ...

Narrowing the final detection bandwidth is the only hope. If the noise spectrum is white, narrow bandwidth through averaging works. If you cannot average, there is trouble. The old lockin amplifiers used a modulation or chopping signal and a long time constant in the final filter. I had signals at times that required an hour of integration to detect. It was slow but it worked.

Reply to
doug

I forgot to mention that the signal wave form can be anything that can be generated.

For example, if something similar to an AM radio signal, say, f(t) =3D (sint)(sin10t)(sin10t) was possible and the frequency of the noise was about the same as the sin(t) factor, then f(t) will plot the noise every time f(t) =3D 0.

In this case that would be ten times as often as the sin(t) factor.

The noise can then be subtracted to recover the wave form.

Bret Cahill

Reply to
Bret Cahill

I think what's happening is that you have

  1. Defined the noise to be bandlimited to radian frequencies below about 1

and

  1. Up-converted (modulated) the signal to have usable (recoverable) components around 10 times that frequency.

So (am I allowed to say 'duh'?) a simple highpass filtering operation will remove the noise from the upconverted signal. You moved the signal to where you knew there was no noise.

It's more interesting when the signal and the noise occupy the same bandwidth.

John

Reply to
John Larkin

ask comp.dsp

Reply to
HardySpicer

In principle, this can be done with synchronous averaging. I have a tutorial series starting here:

The basic idea is that you must be able to sample the signal synchronously with the sourve (transmitted) wave. That means you need some sort of trigger signal derived directly from the source. On each trigger, you acquire some number of samples, a long enough series for your needs. Here you will want the series long enough to encompass at least one waveform cycle, since you are looking for amplitude of the overall wave.

Let's say that you acquire 1024 samples per trigger. (It's generally OK to miss triggers, as long as you always start acquisition on a trigger.) Then you add those 1024 samples, one by one, into a 1024-bin accumulator. The accumulator will end up holding the average value of the waveform, assuming that the waveform is constant and the noise is not synchronous.

For every doubling of the number of samples you add into the accumulator, the S/N improves by 3 dB. The overall improvement is thus determined by how long you want to wait to accumulate enough samples.

This is the technique used to monitor "evoked potentials" in the brain. For example, the subject is presented with a series of repeating tone bursts of a given frequency, while scalp electrodes monitor brain activity. The source of the tone bursts is also the trigger for the averager. The brain response to any given burst is hopelessly buried in noise, since the scalp electrodes see all the brain activity, not just the auditory part. Only a tiny part of the total is due to the auditory response, but that part is in synchrony with the stimulus, while the rest of the brain in general is not. So after several thousand tone bursts, you can "see" the auditory response. This is used to test hearing in lab animals and infants who can't report what they hear.

Best regards,

Bob Masta DAQARTA v4.51 Data AcQuisition And Real-Time Analysis

formatting link
Scope, Spectrum, Spectrogram, Sound Level Meter FREE Signal Generator Science with your sound card!

Reply to
Bob Masta

The noise and the signal are both low and have about the same frequency, ~ x/2pi. There is little that can be done to change this situation. There is no time to wait more than several cycles for the result either.

To plot the noise. Every time sin 10t =3D 0, f(t) =3D zero, and the only thing left is the low frequency noise.

Then a high pass filter can smooth out the (sin10t)(sin10t) component so something like the original signal can still be recovered after the noise is subtracted out.

Alternatively traditional filters can be eliminated altogether. The signal + noise as well as the noise alone can be traced out from the high frequency signal.

The noise is then subtracted from the signal + noise to recover the signal.

Plot f(t) and it's easy to see the original signal sill exists, although in a somewhat discontinuous form.

It would be interesting if this has never been done before.

Bret Cahill

Reply to
Bret Cahill

That was the plan. Complete control over and knowledge of the source wave should provide some options to recover the signal, even if the noise is orders of magnitude more than than the source wave.

Kind of like the military shooting down a tank of hydrazine with a transponder on it and then claiming they took out a satellite.

The system will attenuate the low frequency wave more than a high frequency wave that isn't part of the low frequency wave.

Determining the amount of attenuation of the low frequency wave was, in fact, the goal.

The high frequency component, however, should attenuate much like the low frequency wave if it is tracing out the low frequency wave.

Maybe 5 - 7 cycles at most.

At first I thought a more sophisticated version of resonant frequency, i.e., lots of different frequency waves added together, might have been a solution but there wasn't enough time to get past the transient effects.

Reply to
Bret Cahill

I'm not saying your approach won't work in my case but your Fig. 1 isn't exactly my situation.

My noise + signal, if you could see which was which, would look like a large smooth curve with a much smaller amplitude sin curve with a similar period superimposed just above it and/or just below it.

My solution was to multiply a high frequency _always positive_ wave onto the original signal.

The new signal is discontinuous but it still looks and acts a lot like the original.

The difference is every time the high frequency wave was zero, the entire function would be zero and only the noise would remain. The noise is smooth so it could be accurately determined by numerical regression even if the high frequency signal wasn't all that high.

Once the noise is known it can be subtracted from the entire output from the receiver.

The noise needs to be known to at least 4 decimal place accuracy.

Bret Cahill

Reply to
Bret Cahill

It is the same situation just the spectrum of the noise is different. In all of this, the game is the same because the physics is the same. To get a signal to noise ratio or signal to interference ratio larger than one, you need a bandwidth where the noise is smaller than the signal. Averaging, which is applying a narrow bandpass filter helps most for random noise.

Just how similar a period? The closer the two are, the harder your job is.

Life might be simpler if you just multiplied your signal by a square wave whose value is 0 or 1.

What you are describing is not noise but interference.

That is a tall order.

Reply to
doug

Which is everything when you want to reduce noise.

The math is entirely different. A smooth low frequency noise curve can be sampled fewer times over longer intervals and the result can be very accurate, especially with regression.

Convert the signal to a higher frequency wave form that still has characteristics of the original signal.

Time =3D money. If the average takes more than a few cycles, the result won't be any good for other reasons.

Maybe 0.5 - 1.5.

Say the sensor is getting something like 100sin(1.3t) [the noise] + sint [the signal].

I need to know the signal to 0.25% accuracy.

Total control the signal should be worth _something_.

What's the advantage?

If it's not possible there's a completely different approach.

Bret Cahill

Reply to
Bret Cahill

The sensor receives the loud noise, say, 10sin(0.7x), plus the small signal, say, sin(x ). The signal, however, can be transformed into sin (x)sin^2(100x) so the sensor receives

(10sin(0.7x) + sin(x)sin^2(100x)) x from 2 to 3.4

This can be viewed by pasting the entire line into

formatting link

The blue area is the signal to be extracted from the noise. It's over the noise to the left of x =3D pi and under the noise to the right of pi.

The high frequency curve runs between the signal and the noise and maps out both curves to any precision depending on frequency and regression.

Bret Cahill

Reply to
Bret Cahill

You are assuming the ability to high-frequency modulate the signal before the noise is added to it. So you already know what the noiseless signal looks like.

John

Reply to
John Larkin

No, the same considerations apply. You need a bandwidth where the ratio of signal to noise meets your requirement.

No, what you are claiming above is for noise that has a vastly different spectral content than the signal.

That has no effect on the SNR since the noise gets converted too.

If the signal is really one thousandth of the noise, and the noise is random, you need to average a million traces to get an SNR of one. This is because the SNR increases as the square root of the number of traces averaged.

So you want a bandpass filter at frequency t which has skirts down

120db at 1.3t (assuming the noise signal is narrow band and that it is 1000 times the signal amplitude originally).

Knowing the frequency and phase is already factored in the discussion.

You get to look at the signal 50% of the time and the noise 50% of the time to get their relative amplitudes.

I would go after that.

Reply to
doug

What you are doing is changing the problem. You are now looking at a sin(100x) modulated by sin(x). This says the noise and the signal are independent of one another. If you can do this, make it 1000x and make you life easier.

Reply to
doug

As I pointed out above, I forgot to mention in the OP that the designer could change the signal to suit the problem.

Actually it's a sin^2(100x) which is always positive.

Sin100x will not work as the sign of the original signal must be preserved.

For example, try pasting

(10sin(0.7x) + sin(x)sin(100x)) x from 2 to 3.4

into

formatting link

How would you know the noise curve?

Bingo!

In some situations it may be difficult to use very high frequencies.

That's why regression was mentioned.

Bret Cahill

Reply to
Bret Cahill

As I said in the OP, "Everything is known about the transmitted wave, i.e., the shape & phase angle, except the amplitude."

This works for the same reason reading a newspaper in a foreign language is easy. You already know what they are going to say.

Bret Cahill

Reply to
Bret Cahill

The shape and the effect of the original signal, sin x, on the rest of the system cannot be preserved if the sign of the high frequency factor alternates each cycle.

And this is done by first identifying the noise curve.

The high frequency curve and, therefore, the entire f(t), will equal 0 on every cycle of the high frequency.

Since we know when the signal is zero we know the value of the noise at that time.

Not if you can identify the noise to 99.995% accuracy.

I didn't think the approach above was either fancy or "narrowing the bandwith."

Obviously, if the noise frequency is about the same as signal frequency it will necessary to end run the use of conventional filters altogether.

It would be interesting to learn if and when something similar was tried.

Bret Cahill

Reply to
Bret Cahill

Sin(100x) will work just fine. To detect sin(x) just multiply the signal by sin(100x) again or, to be careful multiply by sin(100x) and also by cos(100x). This does a quadrature detection and the magnitude of the two term is independent of the phase relative to sin(100x)

The point of using the high frequency is to remove your signal from the noise value.

The physics requirements are always the same. You need to have the signal power greater than the noise power in the detection bandwidth. Any fancy processing scheme is just trying to narrow the detection bandwidth.

Reply to
doug

Are you familiar with the way a lockin amplifier works? That sounds maybe like what you are doing. If you know everything about the signal but its amplitude, then you have or can construct a normalized (unity amplitude) version of it. That will positively correlate with the unknown-amplitude version of the signal but have zero correlation to random noise.

Things like IR absorption spectrometers commonly chop (square wave modulate) the light source and recover the signal with a synchronous rectifier. That washes out any noise picked up in the optical path or the detector. Things like this commonly dig signals out from 1000x the noise... but slowly.

If the noise is known to be bandlimited, it's a lot easier... almost cheating.

John

Reply to
John Larkin

PolyTech Forum website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.