future information on rd at the current time k, i.e., Rk :={rd(l); k<l≤N}. This introduction of
114
Discrete Time Systems
E means that the unknown part of the reference signal follows a stochastic process,
Rk
whose distribution is allowed to be unknown.
Now we formulate the following optimal fixed-preview tracking problems for the system (1)
and the performance index (2). In these problems, it is assumed that, at the current time k,
rd(l) is known for l ≤ min(N, k+h), where h is the preview length.
The Stochastic Optimal Fixed-Preview Tracking Problem by State Feedback:
Consider the system (1) and the performance index (2), and assume the conditions A1, A2
and A3. Then, find * d
u minimizing the performance index (2) where the control strategy * d
u
(k), 0 ≤ k ≤ N-1, is based on the information Rk+h:={rd(l); 0 ≤ l ≤ k+h} with 0 ≤ h ≤ N and the
state information Xk:={x(l); 0 ≤ l ≤ k}.
The Stochastic Optimal Fixed-Preview Tracking Problem by Output Feedback:
Consider the system (1) and the performance index (2), and assume the conditions A1, A2
and A3. Then, find * d
u minimizing the performance index (2) where the control strategy * d
u
(k), 0 ≤ k ≤ N-1, is based on the information Rk+h:={rd(l); 0 ≤ l ≤ k+h} with 0 ≤ h ≤ N and the
observed information Yk:={y(l); 0 ≤ l ≤ k}.
Notice that, on these problems, at the current time k to decide the control strategies, Rk+h can
include any noncausal information in the meaning of that it is allowed that the future
information of the reference signals {rd(l); k ≤ l ≤ k+h} is inputted to the feedback controllers.
3. Design of tracking controllers by state feedback
In this section we consider the state feedback problems.
Now we consider the coupled Riccati difference equations [Costa et al. (2005); Fragoso
(1989)]
Xi(k)=Ad,i’(k)Ei(X(k+1),k)Ad,i(k)+C1d,i‘C1d,i–F2,i‘T2,iF2,i(k), k=0, 1, ···
(3)
where E
M
i(X(k+1),k)= ∑
p
X
=
d, ( k)
1
ij
j
j+1(k+1), X(k)=(X1(k), ···, XM (k)),
=
+
+
2
T ,i (k) D12d,i ‘D12d,i B2d,i ‘Ei (X(k 1),k)B2d,i ,
R
=
+
2,i (k )
B2d,i ‘Ei (X(k 1),k)Ad,i ,
F (k)
1
−
= −
2,i
2
T , 2
R , (k
i
i
)
and the following scalar coupled difference equations.
α i (k)=Ei(α (k+1),k)+tr{Gd,iΧiGd,i ‘Ei(X(k+1),k)} (4)
where E
M
i( α (k+1),k)= ∑
p
α (k+1) and α (k)=(α (k), ... ,α (k)).
=
d, ( k),
1
ij
j
j
1
M
Remark 3.1 Note that these coupled Riccati difference equations (3) are the same as those for
the standard stochastic linear quadratic (LQ) optimization problem of linear discrete-time
Markovian jump systems without considering any exogeneous reference signals nor any
preview information [Costa et al. (2005); Fragoso (1989)]. Also notice that the form of the
equation (4) is different from [Costa et al. (2005); Fragoso (1989)] in the points that the
solution α (·) does not depend on any modes in [Costa et al. (2005)] and the noise matrix Gd
does not depend on any modes in [Fragoso (1989)].
Stochastic Optimal Tracking with Preview for Linear Discrete Time Markovian Jump Systems
115
We obtain the following necessary and sufficient conditions for the solvability of the
stochastic optimal fixed-preview tracking problem by state feedback and an optimal control
strategy for it.
Theorem 3.1 Consider the system (1) and the performance index (2). Suppose A1, A2 and
A3. Then the Stochastic Optimal Fixed-Preview Tracking Problem by State Feedback for (1)
and (2) is solvable if and only if there exist matrices Xi(k)≥O and scalar functions α i (k), i=1,
···,M, satisfying the conditions Xi(N)=C1d,i'(N)C1d,i(N) and α i (N)=0 such that the coupled
Riccati equations (3) and the coupled scalar equations (4) hold over [0,N]. Moreover an
optimal control strategy for the tracking problem (1) and (2) is given by
*
θ
d
u (k)=F2,i(k)x(k)+Du,i(k)rd(k)+Dθu,i(k)Ei( c (k+1),k) for i=1, ···,M
where Du,i(k)=-
1
θ
2,
T− i (k)B2d,i‘Ei(X(k+1),k)B3d,i and Dθu,i(k)=- 1
2,
T− i (k)B2d,i‘. i (k), i=1, ···,M,
k∈ [0,N] satisfies
θ (k) = Ad, i’(k)E (θ (k + 1),k) + Bd,
i
i
i
(k) dr (k)
, (5)
θ
=
i (N)
C1d,i‘D13d,i d
r (N)
where E
M
i( θ (k+1),k)= ∑
p
θ (k+1) and θ (k)=(θ (k), ···, θ (k)),
=
d, ( k)
1
ij
j
j
1
M
Ad, i ’(k) = A −
d,i
Dθu,i’ 2
T ,i 2
F ,i (k),
Bd, i (k) = A
+
−
+
d
,i ’ Ei (X(k 1),k)B3d,i
2
F ,i ’ 2
T ,iDu,i (k) C1d,i ‘D13d,i
and θ
θ
θ
θ
c , i (k) is the 'causal' part of i (·) at time k. This c , i is the expected value of i over Rk and given by
θ (l) = Ad, i’(l)E (θ (l + 1),l) + Bd,
+ ≤ ≤ +
,
i
i
c i
c
(l) dr (l), k 1 l k h,
θ
+ +
=
+ ≤
−
(6)
c, i (k
h 1)
0 if k h N 1
θ
+ +
=
+ =
c, i (k
h 1) C1d,i ‘D13d,i d
r (N) , k h N
where E
M
i( θ
θ
θ
θ
θ
c (k+1),k)= ∑
p
(k+1) and (k)=(
(k) , ···,
(k)).
=
d, ( k)
1
ij
j
c, j
c
c,1
c, M
Moreover, the optimal value of the performance index is
*
J
=
Χ + α
+
θ
dN (x0, u
d
d
,r ) tr{ Q
}
i
i
i (0)
E{ E {2
‘x }}
0
0
0
R
0
i
0
0
N −1
(7)
1/2
−
2
+E{ ∑ E {| T
θ
+
+ J
R
2, m( k)D
k E
(
k 1 ,k)| }}
r
θ u,m(k) ( ) m(k) c (
)
d ( d )
k
k=0
where θ −
θ
θ
c, m( k) (k)= m( k) (k)- c, m( k) (k), k ∈ [0,N],
E
M
i( θ −
θ −
θ −
θ −
θ −
c (k+1),k)= ∑
p
(k+1),
(k)=(
(k), ···,
(k)) and
=
d, ( k)
1
ij
j
c, j
c
c ,1
c, M
N −1
⎧⎪
J
=
+ ⎨ ∑
−
θ
+
d (r )
E{ E {|D
(N)r (N) 2|
E
T
R
R
m k
θ
N
}
1/2
2
d
13d,m(N)
d
}
E
{ | 2, ( )D
k E
k
1 ,k |
u,m(k) ( ) m(k) ( (
) )
k
⎪⎩ k=0
-2E
θ
m(k)(
(‘k+1),k)D
‘ T
D
k r k
θu,m(k) 2, m( k) u,m(k) ( ) d ( )
+ 2E
θ
m(k)(
(‘k+1),k)B3d,m(k)(k) dr (k)+Jd,k,m(k)( dr) }},
116
Discrete Time Systems
J
=
⎡−
+
+
d,k ,m(k) ( d
r )
d
r (‘k) D
⎣
u,m(k) ‘ 2
T , m( k)Du,m(k) (k) B3d,m(k)‘Em(k)
(X(k 1),k)B3d,m(k)
+D
⎤
13d,m(k) ‘D13d,m(k) ⎦ d
r (k).
(Proof) See the appendix 1.
Remark 3.2 Note that each dynamics (6) of θ c, i , which composes the compensator
introducing the preview information, is coupled with the others. It corresponds to the
characteristic that the Riccati difference equations (3) are coupled with each other, which
give the necessary and sufficient conditions for the solvability of the stochastic optimal
tracking problem by state feedback.
Next we consider the following two extreme cases according to the information structures
(preview lengths) of rd:
i. Stochastic Optimal Tracking of Causal {rd(·)}:
In this case, {rd(k)} is measured on-line, i.e., at time k, rd(l) is known only for l≤k.
ii. Stochastic Optimal Tracking of Noncausal {rd(·)}:
In this case, the signal {rd(k)} is assumed to be known a priori for the whole time interval
k∈ [0,N].
Utilizing the optimal control strategy for the stochastic optimal tracking problem in
Theorem 3.1, we present the solutions to these two extreme cases.
Corollary 3.1 Consider the system (1) and the performance index (2). Suppose A1, A2 and
A3. Then each of the stochastic optimal tracking problems for (1) and (2) is solvable by state
feedback if and only if there exist matrices Xi(k) ≥O and scalar functions α i (k), i=1, ···,M,
satisfying the conditions Xi(N)=C1d,i'(N)C1d,i(N) and α i (N)=0 such that the coupled Riccati
difference equations (3) and the coupled scalar equations (4) hold over [0,N]. Moreover, the
following results hold using the three types of gains
Kd,x,i(k)=F2,i(k), Krd,i(k)=Du,i(k) and Kd,θ,i(k)=Dθu,i(k) for i=1, ···,M.
i. The control law for the Stochastic Optimal Tracking of Causal {rd(·)} is
ud,s1(k)=Kd,x,i(k)x(k)+Krd,i(k)rd(k) for i=1, ···,M
and the value of the performance index is
JdN(x0, ud,s1, rd)=tr{
Χ }+ α (0)+E{ E {2θ ‘x0}}
0
i
Q
0
i
0
i
0
R
0
i
N −1
+E{ ∑ E {| 1/2
T
D
J ( r
R
2, m( k)
θu,m(k)(k)Em(k)( θ (k +1),k) 2
| }}+ d d).
k
k=0
ii. The control law for the Stochastic Optimal Tracking of Noncausal {rd(·)} is
ud,s2(k)=Kd,x,i(k)x(k)+Krd,i(k)rd(k)+Kd,θ,i(k)Ei(θ (k +1),k) for i=1, ···,M
with θ i (·) given by (5) and the value of the performance index is
JdN(x0, ud,s2, rd)=tr{
Χ }+α (0)+2θ ‘μ0+ J (rd).
0
i
Q
0
i
0
i
0
i
d
(Proof)
i. In this causal case, the control law is not affected by the effects of any preview
information and so θ
θ
c (k)=0 for all k ∈ [0,N] since the each dynamics of c, i becomes
Stochastic Optimal Tracking with Preview for Linear Discrete Time Markovian Jump Systems
117
autonomous. As a result we obtain θ (k)=θ − c (k) for all k∈ [0,N]. Therefore we obtain
the value of the performance index JdN(x0, ud,s1, rd).
ii. In this noncausal case, h=N-k and (5) and (6) becomes identical. As a result we obtain
θ (k)=θ
θ −
c (k) for all k ∈ [0,N]. Therefore we obtain c (k)=0 for all k ∈ [0,N] and the value
of the performance index JdN(x0, ud,s2, rd). Notice that, in this case, we can obtain the
deterministic value of θ (0) using the information of {rd(·)} until the final time N and so
0
i
the term E{ E {2θ ‘x0}} in the right hand side of (7) reduces to 2θ ‘μ0. (Q.E.D.)
0
R
0
i
0
i
4. Output feedback case
In this section, we consider the output feedback problems.
We first assume the following conditions:
A4: Gd,m(k)(k)Hd,m(k)'(k)=O, Hd,m(k)(k)Hd,m(k)'(k)>O
By the transformation
ud, c (k):=u d(k)-Du,i(k)rd(k)-Dθu,i(k)Ei(θ c (k+1),k)
and the coupled difference equations (3) and (4), we can rewrite the performance index as
follows:
J
=
Χ + α
dN (x0, ud, c, d
r ) tr{ Q
}
i
i
i (0)
0
0
0
+E{ E {2‘θ x }
R
0
i
0
0
}
N −1
⎧
⎪
⎫⎪
+E⎨ ∑ E {| u
−
d c (k ) − F
(k)x(k)
2
,
θ
+
R
2,m(k)
-D
k E
(
k
1 ,k)|
}
θu,m(k) ( ) m(k) c (
)
⎬
2
T , m( k)
k
⎪⎩ k=0
⎪⎭
+ Jd ( d
r )
and the dynamics can be written as follows:
x(k+1)=Ad,m(k)(k)x(k)+Gd,m(k)(k)ωd(k)+B2d,m(k)(k) ud, c (k)+ rd, c (k)
where
rd, c (k)=B2d,m(k){Du,m(k)(k)rd(k)+Dθu,m(k)(k)Em(k)(θ c (k+1),k)}+B3d,m(k)(k)rd(k).
For this plant dynamics, consider the controller
ˆ x
+
=
+
e (k
1) A
(k) ˆ xe (k) B
(k) ud, c*
d,m(k)
2d,m(k)
(k)
(8)
+B
(k) rd, c k − M
k [y k − C
ˆ
3d,m(k)
( )
m(k) ( )
( )
2d,m(k) xe (k )]
ˆ x
=
= μ
=
e (0)
E{ E {x
, ud c k
F
k ˆ x
R
0}}
0
, * ( )
2,m(k) ( ) e (k )
0
where Mm(k) are the controller gains to decide later, using the solutions of another coupled
Riccati equations introduced below.
Define the error variable
e(k):=x(k)- ˆ xe (k)
118
Discrete Time Systems
and the error dynamics is as follows:
e(k+1)=Ad,m(k)(k)e(k)+Gd,m(k)(k)ωd(k)+Mm(k)(k)[y(k)-C2d,m(k) ˆ x (k)]
e
=[Ad,m(k)+Mm(k)C2d,m(k)](k)e(k)+[Gd,m(k)+Mm(k)Hd,m(k)](k)ωd(k)
Note that this error dynamics does not depend on the exogenous inputs ud nor rd. Our
objective is to design the controller gain Mm(k) which minimizes
JdN(x0, ud, c* , rd)=tr{
Χ }+α (0)+E{ E {2θ ‘x0}}
0
i
Q
0
i
0
i
0
R
0
i
N −1
+E{ ∑ E {|F
R
2,m(k)(k)e(k)
k
k=0
-D
2
θu,m(k)(k)Em(k)( θ −
c (k +1),k)|
J ( rd)
2
T
}}+
, m( k )
d
Notice that e(k) and Em(k)(θ − c (k +1),k) are mutually independent.
We decide the gain matrices Mi(k), i=1, ···,M by designing the LMMSE filter such that
N −1
∑
E{ E {|e(k) 2
| }} is minimized. Now we consider the following coupled Riccati
k=0
Rk
difference equations and the initial conditions.
+
= ∑
⎡
−
j
Y (k 1)
pd, A
ij ⎣ d,i ’ i
Y (k)Ad,i Ad,i i
Y (k)C2d,i ’(Hd,iHd,i’Πi (k)
i∈ J( k)
(9)
+C
Y (k)
1
?