Optimal control - LQR problem

Read it as a 'weak' control signal. If your control action is weak, you won't be able to contain variations in the (plant) output well, which means large output in this context. Strong control can maintain steady conditions better, but at the expense of increased control effort. In many practical applications this is a fairly minor consideration, other aspects like stability are more important. For situations like controlling spacecraft orientation with limited fuel supplies, it's a key factor.

Hello Bruce,

thank you for your reply. I see now that author was refering to control effort and not amplitudes of signals. I wonder when it is appropriate to consider LQR comparing to classical PID control (cascade, or feedforward)?

If R is is 0 then there is no cost function for control output. The control output will do what ever is required to reduce the error is quickly as possible. Should this happen the response will normally be very violent because the control output u(t) will go into saturation to drive the error to 0. This violent reaction is often not desired.

There must be a cost function for the control output, u'(t)*R*u(t) to keep the control output from going into saturation trying to minimize just the error in the state. This is effectively a tuning parameter because few applications really have a true cost function u'(t)*R*u(t). In this case R is modified until the desired response is obtained.

??? If R is 0 then the optimizing routine will calculate huge gains to minimize the error. This in turn will result in huge control outputs that will minimize the error. Some times the best way to learn is to write a small program that simulates a plant then then minimizes the cost function by trying different PID gains. Do several runs changing R after each run. I will see if I have an example some place.

Peter Nachtwey

I'm a bit narrowly focussed to answer that question. In my business (industrial control) processes are generally not well defined, and the cost of control effort is normally low. PID, sometimes incorporating additional functions as you mention, is the norm for regulatory loops. Dynamic multivariable predictive algorithms are also used.

Certainly it's fair to answer the question "which of these should I spend my time implementing".

If you model your system right you can force your LQR to _be_ a PID controller; in one sense you can think of the process of getting the LQR design to be just another way of choosing the coefficients for a controller.

LQR is mainly just another way to select PID gains. LQR has a couple problems. First LQR is a regular, it only tries to drive the PV to 0. Second, as Bruce mentioned, the cost function in most applications is low and not a concern. I have had only one person ask me about LQR in a motion control product and he was just trying to impress me because following the motion profile was most important.

Tim is right about the LQR controller can be a PID or a I-PVA as in my example below. The LQR work sheet started out as a minimun ITAE work sheet. The big difference is the cost function and the how PID gains are generated. The controller with the LQR gains can be used with feed forwards. LQR just provides yet another way to calculate the controller gains.

Here is an example ftp://ftp.deltacompsys.com/public/NG/Mathcad%20-%20LQR.pdf

Notice that the cost function takes into account not only errors but error rates and the second derivative. If Q only takes the error into account the underdamped system would oscillate as it approaches the SP. The Q array works well as it is. The vale of R can be made bigger to make the response slower or made smaller to make the response faster but with a chance of an osciallating response. I can make R=3D0 so there is not cost for the control effort and the control output will go in to saturation in a effort to reduce the error as fast as possible.

There are more efficient ways to calculate the gains. I may update the work sheet to reflect a more efficient way of calculating the gains.

Peter Nachtwey

AsimV notified me about using the err'*Q*err. I told him it really doesn't make any difference but when solving for the LQR gains it really is simpler just to use x'*q*x and assume the target is [0;0;0]. I modified my LQR worksheet to reflect that and added a pole plot at the bottom. ftp://ftp.deltacompsys.com/public/NG/Mathcad%20-%20LQR.pdf

AsimV also notified me about how long it takes to compute the LQR gains the way I did. I admit it. Using a optimizer is slow. I looked to see how my Digital COntrol System Analysis and Design book by Phillips and Nagle did it. They use a iterative technique that is similar to computing steady state Kalman gains.

ftp://ftp.deltacompsys.com/public/NG/Mathcad%20-%20LQR%2010pt2.pdf

This method in the book works but I had to make the Q and A array 4x4 so the method in the book will compute a integrator gain. Since the Q array is not the same in the two .pdf files the results are not the same. Also, the method in the book seems to assume that R is a scaler instead of a matrix. I can see where it might be important to have a more general solution that the optimizer provides so that R can be a matrix that will not only penalize control output but the rate of change in control output. This may be necessary if the feed back resolution is coarse.

The big advantage that the method in the book has is that is computes the gains MUCH faster. The optimizer technique takes minutes to compute the LQR gains but the method in the book takes a fraction of a second.

Also notice that I didn't really need a cost function for the control output when using the method in the book. It seems that having a cost for the derivatives keeps the control output tame enough.

Any comments?

Peter Nachtwey

Peter, that was exactly what I was trying to achieve last night, but I failed because I coudn't find a way to slove ARE (Algebraic Ricatti equation). Your formulas (in yur book) are very similar but there something I failed to derive... For example Optimal gain vector can be found in this way : K = R^(-1)*B'*P and Algebraic Ricatti equation is: A'P+PA-PBK+Q = 0 => A'*P

+P*(A-B*K)+Q = 0. I wonder what numerical technique is used to derive formulas you used, because they guaranteed to return optimal solution.Most theory atricles and materials I have found on the Internet used formulas I posted above. Is there more theory in your book, maybe some derivation? I tried last night to solve this in analytically but had no success. Now, when I have these relationships, I 'll try to fit them to theoretical description I posted on plctalk forum. K = inv( B' * P * B + R)*B' * P * A; P = A' *P * (A-B*K)+Q;

Cost function is simply incorporated in ARE, because ARE is derived from cost function. This way, solution to ARE is matrix P, and with P, one can calculate K. It seems that authors used very smart way to solve ARE that is used in the book. I must admit that most materials I have found are practically useless, they are really full of high level theory, while physical meaning is hidden.

Best regards, Asim Vodencarevic

The authors of the book I referenced in the pdf didn't use the Ricatti equation. They derived it by using the cost function and making substitutions. The main idea is the current value of the cost function should be 0 if the state is 0 or at least minimal. Then the calculations work backwards to find the point where the change in the cost function from time n to time n-1 doesn't increase. At that point the LQR gains are stable.

I wouldn't know. I have only this one book that covers the LQR topic and I didn't even bother to look at it seriously until you asked because I am not a big believer in LQR control. I did remember the Matlab example is the book looked simple but I was never interested in regulators more minimizing control effort. I want to follow motion profiles exactly. Anything was a diversion.

Yes, 9 pages of explanation and about 39 formulas

I have felt your pain. I have had to read a few different books on the same topic over and over and compare formulas and the way things are said. Sometimes one author would do a much better explanation than another. I have seen cases where two different books have different equations for calculating the same thing when I worked through the math I could verify one book is right and the other wrong. The difference was more than just two different ways to calculate the same thing. In other words, you can't really trust the text books all the time.

Peter Nachtwey

Peter, by searching more materials about LQR, I have found that performance index is defined as integral(x' * Q * x + u' * R * u)dt + x' * Qf * x, where matrix Qf represents so called terminal or final state cost. This is really weird. What form your book is using? Matlab has bulit in function to solve LQR problem. It is interesting that it uses following performance index: integral(x' * Q * x + u' * R

• u +2*x' * N *u)dt. I don't see any real practical use of this and it complicates calculations. I'm pretty much sure that cost of final state can be incorporated in x' * Q * x term, and solvoing simpler form: integral(x' * Q * x + u' * R * u)dt will do the trick. I'm sure there are people here who design and use LQR controller in practice. I'd like to hear their opinions. Thank you