Kalman filter tuning for hand tracking

Hi Piotr,

One interpretation of the process noise covariance (Q) is that it is a measure of how much you expect your positions and velocities to change.

If you really believe your model, then you probably mainly expect your positions to be incremented by the associated velocity, rather than some other influence. This means the size of the Q entries affecting the x and y positions can be small (though zero is almost certainly too small).

It's probably OK to make the assumption that the process noises on the positions and velocities are independent. This makes your Q matrix diagonal (so you only have, at most, four parameters to choose).

So: how much do you expect your velocities to change over the running of the filter?

Ciao,

Peter K.

/cut

No, I do not believe in my model too much ;). But its ok for me. In fact I want Kalman filter only to discriminate pixels that are too far from the predicted position.

Here I use a different approach. I assume that the process noise is due to accelerations ax, ay normally distributed around 0. So the process noise vector w is: | 0.5 deltaT.^2*ax| | 0.5 deltaT.^2*ay| | deltaT*ax | | deltaT*ay |

and Q =cov(ww^T);

So if i put C = | Var(ax) 0| | 0 Var(ay)| then Q= |0.25C, 0.5C| |0.5C , C |

I checked this for a generated sequence of points with known Var(ax) and Var(ay). I measured the difference between true and predicted values and the data covariance matched this model very well. But when I am trying to estimate Q using only updated state vector values, the covariance matrix does not fit this model at all. I tried also to simulate velocity measurements by taking the difference between the old position x(k-1/k-1) and new measured position z(k). But the results were still bad.

You can imagine that hand motion performing rapid gestures is not linearly predictible. So the process noise will present high covariance values. Its ok, I need only rough estimate of the hand position, but I'd rather not guess what the covariance is but estimate it from the training data.

regards

Piotr

OK.

So why not include the accelerations in the state vector, and make "jerk" the noise "driving" the system?

Do the innovations (errors) in your Kalman filter equations end up being white (i.e. uncorrelated) ? If not, then there is probably a problem somewhere.

Try the innovations and see what they tell you.

OK. The problem is that different gestures will give you different estimates of the process noise covariance. I could imagine a straight up-and-down gesture would give one value for the y variance and a very small value for the x variance, and vice versa for a simple left-right gesture.

Unless your gestures are all pretty similar, I don't think your training data will tell you much.

Ciao,

Peter K.

Hello Peter

For some reasons your last post has not appeared in my news browser, so I cannot reply it directly. But I've read this in google's archives.

I'd rather not introduce additional variables in my state vector. I assume that the system has a consant velocity and I am using accelerations as noise just to predict what Q should look like and see whether I can trust the results I get from the data.

If by innovation you mean z-H*x(k/k-1), its covariance is mainly diagonal. Its 2 x 2, because measurement z contains only positions, not velocities. If you are asking about corelation of w1 = x(k/k)-x(k/k-1), then position and velocity along one coordinate (e.g position x and vx) are corelated, but not in the same way as: w2 = xtrue - x(k/k-1) (here I know the true values because I check this method for a generated sequence). the latter, cov(w2) is more or less similar to:

Q= |0.25C, 0.5C| |0.5C , C |

but the former, cov(w1), is sth like:

[3*C1 2*C1] [2*C1 C1 ]

with C1 being sth like:

[ a 0] [ 0 b]

where a is generally greater than b. (More or less agrees with standard deviations I introduced on ax and ay to generate the process noise) So the correlation tendency seems to be inverted in some way. If I use the result as my initial Q, it does not converge, but oscilates with these proportions maintained.

Concerning gestures, I am aware, that various gestures produce different covariance. My intention is to work on sign language gestures, so I think it is possible to determine a covariance matrix which will embrace most signs. But as I mentioned, let Q determined from the predictions be large, its ok until it agrees with the process noise covariance found using true position values.

Regards

Piotr

Yes, UQ's news server has been a little dodgy lately.

OK. You seem to understand things; I was just trying to "rock the boat" a little to see what degrees of freedom we have.

Yes, that's the one. It should be white (i.e. its covariance should be diagonal)... so that means you shouldn't have a real problem with your

Yup, the innovations are the errors between the measurements (z) and your estimate (Hx(k|k-1)).

Yup, as they should be: your model makes x an integration (well, accumulation) of previous vxs, as well as a noise component.

Do you know the true velocities?

Is this saying something about x being a preferred direction? If a is larger than b in the top-right part of cov(w1), then that seems odd. Shouldn't they be (roughly) the same?

Hmm.

OK.

Sorry I haven't been much help; I can't see why the covariances above are coming out like that. Could it be some interaction between Q and your choice of R? I assume R (measurement noise covariance) is chosen as sigma^2 I?

Ciao,

Peter K.

glad to hear that :)

Yes.

In the true point generation process I used: ax = normrnd(0,4.5); // random point from normal zero mean , 4.5 sigma distribution ay = normrnd(0,1.7); (Matlab code). So a and b should correspond to 4.5^2 and 1.7^2, respectively

Yes. My measurement noise in the quantization of a floating point number to the nearest int. This in fact is a uniformly distributed noise, not gaussian, but i used its sigma^2 = 0.0833 to scale R.

I noticed that you have been answering for questions on KF for some time, so it must be one of your interests. If you would like to spare some time, you could repeat my experiment, and see if you get similar results. This might be interesting and all together it takes about 50 lines of code. Just to make sure that I do not make a big mistake, here is what I exactly do

1) generate x_true = [x,y,vx,vy] 2) add noise process of known parameters 3) predict x_true based on the previous x_update value 4) measure x,y 5) find x_update

and then compare cov(xtrue-x_pred) and cov(x_update-x_pred)

Regards

Piotr