1
y =
θ∗ φ
(4.3.47)
s + 1
where φ =
s u, 1 u
can be generated by filtering u. Because (4.3.47) is in
s+2
s+2
the form of parametric model (4.3.24), we can apply the results of this section to
obtain the adaptive law
˙ θ = Γ φ
1
= y −
( θ φ + n 2
s + 1
s) ,
ns = αφ φ
where α > 0 and θ = [ˆ b 1 , ˆ b 0] is the on-line estimate of θ∗. This example illustrates that the dimensionality of θ, φ may be reduced if some of the plant parameters are
known.
4.3.5
Gradient Method
Some of the earlier approaches to adaptive control in the early 1960s [20, 34,
96, 104, 115, 123, 175, 220] involved the use of simple optimization techniques
4.3. ADAPTIVE LAWS WITH NORMALIZATION
181
15
2
Aˆ
ϕ
1.5
ˆ
A
10
ϕ
1
5
0.5
0
0
0
10
20
30
40
0
10
20
30
40
sec
(a)
(b)
sec
20
Aˆ ( t ) sin ( 2 t + ϕ
ˆ ( t ) )
10
A sin ( 2 t + ϕ )
0
-10
-200
5
10
15
20
25
30
35
40 sec
(c)
Figure 4.7 Simulation results for Example 4.3.1.
such as the gradient or steepest descent method to minimize a certain perfor-
mance cost with respect to some adjustable parameters. These approaches
led to the development of a wide class of adaptive algorithms that had found
wide applications in industry. Despite their success in applications, the
schemes of the 1960s lost their popularity because of the lack of stability in
a global sense. As a result, starting from the late 1960s and early 1970s,
the schemes of the 1960s have been replaced by new schemes that are based
on Lyapunov theory. The gradient method, however, as a tool for designing
adaptive laws retained its popularity and has been widely used in discrete-
time [73] and, to a less extent, continuous-time adaptive systems. In contrast
to the schemes of the 1960s, the schemes of the 1970s and 1980s that are
based on gradient methods are shown to have global stability properties.
182
CHAPTER 4. ON-LINE PARAMETER ESTIMATION
What made the difference with the newer schemes were new formulations of
the parameter estimation problem and the selection of different cost func-
tions for minimization.
In this section, we use the gradient method and two different cost func-
tions to develop adaptive laws for estimating θ∗ in the parametric model
z = W ( s) θ∗ ψ
(4.3.24)
The use of the gradient method involves the development of an algebraic
estimation error equation that motivates the selection of an appropriate cost
function J( θ) that is convex over the space of θ( t), the estimate of θ∗ at time
t, for each time t. The function J( θ) is then minimized with respect to θ
for each time t by using the gradient method described in Appendix B. The
algebraic error equation is developed as follows:
Because θ∗ is constant, the parametric model (4.3.24) can be written in
the form
z = θ∗ φ
(4.3.48)
where φ = W ( s) ψ.
The parametric model (4.3.48) has been the most popular one in discrete
time adaptive control. At each time t, (4.3.48) is an algebraic equation where
the unknown θ∗ appears linearly. Because of the simplicity of (4.3.48), a wide
class of recursive adaptive laws may be developed.
Using (4.3.48) the estimate ˆ
z of z at time t is generated as
ˆ
z = θ φ
where θ( t) is the estimate of θ∗ at time t. The normalized estimation error
is then constructed as
z − ˆ
z
z − θ φ
=
=
(4.3.49)
m 2
m 2
where m 2 = 1 + n 2 s and ns is the normalizing signal designed so that
φ ∈ L
m
∞
(A1)
As in Section 4.3.4, typical choices for ns are n 2 s = φ φ, n 2 s = φ P φ for P = P
> 0, etc.
4.3. ADAPTIVE LAWS WITH NORMALIZATION
183
For analysis purposes we express
as a function of the parameter error
˜
θ = θ − θ∗, i.e., substituting for z in (4.3.49) we obtain
˜
θ φ
= −
(4.3.50)
m 2
Clearly the signal m = −˜
θ φ is a reasonable measure of the parameter
m
error ˜
θ because for any piecewise continuous signal vector φ (not necessarily
bounded), large m implies large ˜
θ. Several adaptive laws for θ can be
generated by using the gradient method to minimize a wide class of cost
functions of
with respect to θ. In this section we concentrate on two
different cost functions that attracted considerable interest in the adaptive
control community.
Instantaneous Cost Function
Let us consider the simple quadratic cost function
2 m 2
( z − θ φ)2
J( θ) =
=
(4.3.51)
2
2 m 2
motivated from (4.3.49), (4.3.50), that we like to minimize with respect to
θ. Because of the property (A1) of m, J( θ) is convex over the space of θ
at each time t; therefore, the minimization problem is well posed. Applying
the gradient method, the minimizing trajectory θ( t) is generated by the
differential equation
˙ θ = −Γ ∇J( θ)
where Γ = Γ > 0 is a scaling matrix that we refer to as the adaptive gain.
From (4.3.51) we have
( z − θ φ) φ
∇J( θ) = −
= − φ
m 2
and, therefore, the adaptive law for generating θ( t) is given by
˙ θ = Γ φ
(4.3.52)
We refer to (4.3.52) as the gradient algorithm.
184
CHAPTER 4. ON-LINE PARAMETER ESTIMATION
Remark 4.3.9 The adaptive law (4.3.52) has the same form as (4.3.35)
developed using the Lyapunov design approach. As shown in Section
4.3.4, (4.3.52) follows directly from the Lyapunov design method by
taking L( s) = W − 1( s).
Remark 4.3.10 The convexity of J( θ) (as explained in Appendix B) guar-
antees the existence of a single global minimum defined by ∇J( θ) = 0.
Solving ∇J( θ) = − φ = − z−θ φ φ = 0, i.e., φz = φφ θ, for θ will give
m 2
us the nonrecursive gradient algorithm
θ( t) = ( φφ ) − 1 φz
provided that φφ is nonsingular. For φ ∈ Rn× 1 and n > 1, φφ is
always singular, the following nonrecursive algorithm based on N data
points could be used:
N
− 1 N
θ( t) =
φ( ti) φ ( ti)
φ( ti) z( ti)
i=1
i=1
where ti ≤ t, i = 1 , . . . , N are the points in time where the measure-
ments of φ and z are taken.
Remark 4.3.11 The minimum of J( θ) corresponds to = 0, which implies
˙ θ = 0 and the end of adaptation. The proof that θ( t) will converge
to a trajectory that corresponds to
being small in some sense is
not directly guaranteed by the gradient method. A Lyapunov type
of analysis is used to establish such a result as shown in the proof of
Theorem 4.3.2 that follows.
Theorem 4.3.2 The adaptive law (4.3.52) guarantees that
(i)
, ns, θ, ˙ θ ∈ L∞
(ii)
, ns, ˙ θ ∈ L 2
independent of the boundedness of the signal vector φ and
(iii) if ns, φ ∈ L∞ and φ is PE, then θ( t) converges exponentially to θ∗
4.3. ADAPTIVE LAWS WITH NORMALIZATION
185
Proof Because θ∗ is constant, ˙˜
θ = ˙ θ and from (4.3.52) we have
˙˜ θ = Γ φ
(4.3.53)
We choose the Lyapunov-like function
˜
θ Γ − 1 ˜
θ
V (˜
θ) =
2
Then along the solution of (4.3.53), we have
˙
V = ˜
θ φ = − 2 m 2 ≤ 0
(4.3.54)
where the second equality is obtained by substituting ˜
θ φ = − m 2 from (4.3.50).
Hence, V, ˜
θ ∈ L∞, which, together with (4.3.50), implies that , m ∈ L∞. In
addition, we establish from the properties of V, ˙
V , by applying the same argument
as in the previous sections, that m ∈ L 2, which implies that , ns ∈ L 2. Now from
(4.3.53) we have
|φ|
| ˙˜
θ| = | ˙ θ| ≤ Γ | m|
(4.3.55)
m
which together with |φ| ∈ L
m
∞ and
m ∈ L 2
L∞ implies that ˙ θ ∈ L 2
L∞ and
the proof for (i) and (ii) is complete.
The proof for (iii) is long and more complicated and is given in Section 4.8. ✷
Remark 4.3.12 The property V (˜
θ) ≥ 0 and ˙
V ≤ 0 of the Lyapunov-like
function implies that lim t→∞ V (˜
θ( t)) = V∞. This, however, does not
imply that ˙
V ( t) goes to zero as t → ∞. Consequently, we cannot con-
clude that
or m go to zero as t → ∞, i.e., that the steepest descent
reaches the global minimum that corresponds to ∇J( θ) = − φ = 0.
If however, ˙ φ/m, ˙
m/m ∈ L∞, we can establish that d ( m) ∈ L
dt
∞,
which, together with m ∈ L 2, implies that ( t) m( t) → 0 as t → ∞.
Because m 2 = 1 + n 2 s we have ( t) → 0 as t → ∞ and from (4.3.55)
that ˙ θ( t) → 0 as t → ∞. Now |∇J( θ) | ≤ | φ| ≤ | m| |φ| , which implies m
that |∇J( θ( t)) | → 0 as t → ∞, i.e., θ( t) converges to a trajectory that corresponds to a global minimum of J( θ) asymptotically with time
˙
provided φ , ˙ m ∈ L
m m
∞.
Remark 4.3.13 Even though the form of the gradient algorithm (4.3.52)
is the same as that of the adaptive law (4.3.35) based on the SPR-
Lyapunov design approach, their properties are different. For example,
186
CHAPTER 4. ON-LINE PARAMETER ESTIMATION
(4.3.52) guarantees that ˙ θ ∈ L∞, whereas such property has not been
shown for (4.3.35).
The speed of convergence of the estimated parameters to their true values,
when ns, φ ∈ L∞ and φ is PE, is characterized in the proof of Theorem 4.3.2
(iii) in Section 4.8. It is shown that
˜
θ ( t)Γ − 1 ˜
θ( t) ≤ γn ˜
θ (0)Γ − 1 ˜
θ(0)
where 0 ≤ t ≤ nT 0, n is an integer and
2 α
γ = 1 − γ
0 T 0 λmin(Γ)
1 ,
γ 1 = 2 m 0 + β 4 T 20 λ 2 max(Γ)
where α 0 is the level of excitation of φ, T 0 > 0 is the size of the time interval
in the PE definition of φ, m 0 = sup t≥ 0 m 2( t) and β = sup t≥ 0 |φ( t) |. We established that 0 < γ < 1. The smaller the γ, i.e., the larger the γ 1, the
faster the parameter error converges to zero. The constants α 0 , T 0 , β and
possibly m 0 are all interdependent because they all depend on φ( t). It is,
therefore, not very clear how to choose φ( t), if we can, to increase the size
of γ 1.
Integral Cost Function
A cost function that attracted some interest in the literature of adaptive
systems [108] is the integral cost function
1
t
J( θ) =
e−β( t−τ) 2( t, τ ) m 2( τ ) dτ
(4.3.56)
2 0
where β > 0 is a design constant and
z( τ ) − θ ( t) φ( τ )
( t, τ ) =
,
( t, t) =
(4.3.57)
m 2( τ )
is the normalized estimation error at time τ based on the estimate θ( t) of θ∗
at time t ≥ τ . The design constant β acts as a forgetting factor, i.e., as time
t increases the effect of the old data at time τ < t is discarded exponentially.
The parameter θ( t) is to be chosen at each time t to minimize the integral
square of the error on all past data that are discounted exponentially.
4.3. ADAPTIVE LAWS WITH NORMALIZATION
187
Using (4.3.57), we express (4.3.56) in terms of the parameter θ, i.e.,
1
t
J( θ) =
e−β( t−τ) ( z( τ ) − θ ( t) φ( τ ))2 dτ
(4.3.58)
2 0
m 2( τ )
Clearly, J( θ) is convex over the space of θ for each time t and the application
of the gradient method for minimizing J( θ) w.r.t. θ yields
t
˙ θ = −Γ ∇J = Γ
e−β( t−τ) ( z( τ ) − θ ( t) φ( τ )) φ( τ ) dτ
(4.3.59)
0
m 2( τ )
where Γ = Γ > 0 is a scaling matrix that we refer to as the adaptive gain.
Equation (4.3.59) is implemented as
˙ θ = −Γ( R( t) θ + Q( t))
˙
φφ
R = −βR +
,
R(0) = 0
m 2
˙
zφ
Q = −βQ −
,
Q(0) = 0
(4.3.60)
m 2
where R ∈ Rn×n, Q ∈ Rn× 1. We refer to (4.3.59) or (4.3.60) as the integral
adaptive law. Its form is different from that of the previous adaptive laws
we developed. The properties of (4.3.60) are also different and are given by
the following theorem.
Theorem 4.3.3 The integral adaptive law (4.3.60) guarantees that
(i)
, ns, θ, ˙ θ ∈ L∞
(ii)
, ns, ˙ θ ∈ L 2
(iii) lim t→∞ | ˙ θ( t) | = 0
(iv) if ns, φ ∈ L∞ and φ is PE then θ( t) converges exponentially to θ∗.
Furthermore, for Γ = γI the rate of convergence can be made arbitrarily
large by increasing the value of the adaptive gain γ.
Proof Because φ ∈ L
m
∞, it follows that R, Q ∈ L∞ and, therefore, the differential
equation for θ behaves as a linear time-varying differential equation with a bounded
input. Substituting for z = φ θ∗ in the differential equation for Q we verify that
t
Q( t) = −
e−β(