D ◦ S
τ
2 ◦ S1, since S2 is time-invariant
=
D ◦ S,
τ
as required.
We now show that S is linear. Let x be any input signal and a any complex number. Then
S(ax)
=
(S2 ◦ S1)(ax)
=
S2(S1(ax))
=
S2(aS1(x)), since S1 is linear
=
aS2(S1(x)), since S2 is linear
=
aS(x).
Lastly, if x and y are two input signals, then
S(x + y)
=
(S2 ◦ S1)(x + y)
=
S2(S1(x + y))
=
S2(S1(x) + S1(y)), since S1 is linear
=
S2(S1(x)) + S2(S1(y)), since S2 is linear
=
S(x) + S(y).
This shows that S is linear.
We now compute the frequency response of S. Let H1(ω), H2(ω), H(ω) be the frequency
response of S1, S2, and S, respectively, at the frequency ω. Consider the complex expo-
nential input x given by
∀t ∈ R, x(t) = eiωt.
Then the signal y = S1(x) is a multiple of x, namely, y = H1(ω)x. In particular, y is a
(scaled) complex exponential, and so z = S2(y) is given by
z = H2(ω)y = H2(ω)H1(ω)x.
342
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
But since H(ω) is the frequency response of S at the frequency ω, we also have
z = S(x) = H(ω)x,
and so we obtain
∀ω ∈ R, H(ω) = H2(ω)H1(ω).
(8.34)
The frequency response of the cascade composition is the product of the frequency re-
sponses of the components. Exactly the same formula applies in the discrete-time case.
This is a remarkable result. First, suppose that the cascade connection of Figure 8.10 is
reversed, i.e. consider the system ˜
S = S1 ◦ S2. Then the frequency response of ˜
S is
˜
H(ω) = H1(ω)H2(ω) = H2(ω)H1(ω) = H(ω).
That is, ˜
S and S have the same frequency response! This implies, in fact, that S and ˜
S are
equivalent; they give the same output for the same input. Hence,
In any cascade connection of LTI systems, the order in which the systems are composed
does not matter.
8.5.2
Feedback connection
The feedback arrangement shown in figure 8.11 is fundamental to the design of control
systems. Typically, S1 is some physical system, and S2 is a controller that we design to
get the physical system to do our bidding. The overall system S is called a closed-loop
system. We first show that S is LTI if S1 and S2 are LTI, and we then calculate its frequency
response.
Suppose x is the input signal, and define the signals u, z and y as shown. The circle with
the plus sign represents the relationship u = x + z. The signals are then related by
y
=
S1(u)
=
S1(x + z)
=
S1(x) + S1(z), since S1 is linear
=
S1(x) + S1(S2(y)),
Note that this equation relates the input and output, but the output appears on both sides.
We can rewrite this as
y − S1(S2(y)) = S1(x).
(8.35)
Lee & Varaiya, Signals and Systems
343
8.5. FREQUENCY RESPONSE OF COMPOSITE SYSTEMS
Thus, given the input signal x, the output signal y is obtained by solving this equation. We
will assume that for any signal x, (8.35) has a unique solution y. Then, of course, y = S(x).
We can use (8.35) and methods similar to the ones we used for the cascade example to
show that S is LTI (see box on page 352).
We now compute the frequency response H(ω) of S at frequency ω. Let the frequency
response of S1 be H1, and of S2 be H2. Suppose the input signal is the complex exponential
∀t ∈ R, x(t) = eiωt.
For this input, we know that S1(x) = H1(ω)x and S2(x) = H2(ω)x. Since S is LTI, we
know that the output signal y is given by
y = H(ω)x.
Using this relation for y in (8.35) we get
H(ω)x − S1(S2(H(ω)x)) = H(ω)[x − S1(S2(x))], since S2, S1 are linear
=
H(ω)[x − H1(ω)H2(ω)x]
=
H(ω)[1 − H1(ω)H2(ω)]x
=
S1(x)
=
H1(ω)x, by (8.35),
from which we get the frequency response of the feedback system,
H1(ω)
H(ω) =
(8.38)
1 − H1(ω)H2(ω)
This relation is at the foundation of linear feedback control design.
Example 8.19: Consider a discrete-time feedback system as in Figure 8.11 where
S1 simply scales the input by 0.9. That is, for all discrete-time input signals u,
S1(u) = 0.9u. Suppose further that S2 is a one-sample delay. That is, S2 = D1.
From example 8.9, the frequency response of S2 is given by
∀ ω ∈ R, H2(ω) = e−iω.
344
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
The frequency response of S1 is (trivially) given by
∀ ω ∈ R, H1(ω) = 0.9.
Thus, the frequency response of the feedback composition is
∀
0.9
ω ∈ R,
H(ω) =
.
1 − 0.9e−iω
We can plot the magnitude of this using the following Matlab code, which (after
adjusting the labels) results in Figure 8.12.
omega = [0:pi/250:pi];
H = 0.9./(1 - 0.9.*exp(-i*omega));
plot(omega, abs(H));
Notice that at zero frequency, the gain is 9, and that it rapidly drops off at higher
frequencies. Thus, this system behaves as a lowpass filter.
Example 8.20:
We can use formula (8.38) to obtain the frequency response of
more complex composition, such as the one shown in Figure 8.13. To find the
frequency response of the composition S given on the left of the figure, we first
express as the composition on the right where the system S3 is given in the lower
part of the figure. The frequency response H of S can now be obtained from formula
(8.38) as
H1(ω)H3(ω)
H(ω) =
.
1 − H1(ω)H3(ω)
The same formula also gives the frequency response H3 of S3,
H2(ω)
H3(ω) =
,
1 − H2(ω)
which, upon substitution in the previous expression, yields,
H1(ω)H2(ω)
H(ω) =
.
1 − H2(ω) − H1(ω)H2(ω)
Lee & Varaiya, Signals and Systems
345
8.6. SUMMARY
8.6
Summary
Linear time-invariant systems have a particularly nice property: if the input is a sinusoid,
then the output is a sinusoid of the same frequency. Moreover, if the input is a sum of two
sinusoids, then the output will be a sum of two sinusoids with the same frequencies as the
input sinusoids. Each sinusoid is scaled and shifted in phase by the system. The scaling
and phase shift, as a function of frequency, is called the frequency response of the system.
It turns out that mathematically, this phenomenon is easiest to analyze using complex
exponentials instead of real-valued sinusoids. The reason for this is that the phase shift
and scaling together amount to simple multiplication of the complex exponential by a
complex constant. If an input is represented as a sum of complex exponentials (a form
of the Fourier series), then the output is simply the same Fourier series with each term of
the series scaled by a complex constant. These scaling constants, viewed as a function of
frequency, are the frequency response of the system.
Composing LTI system becomes particularly easy. The frequency response of the cascade
of two LTI systems is simply the product of the frequency responses of the individual
systems. This simple fact can be used to quickly ascertain the frequency response of
complicated compositions.
346
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
Probing Further: Relating DFS coefficients
We have two discrete-time Fourier series expansions (see (8.30) and (7.8)),
p−1
x(n) = ∑ Xkeikω0n
(8.31)
k=0
K
(p − 1)/2 if p is odd
x(n) = A0 + ∑ Ak cos(kω0n + φk), K =
p/2
if p is even
k=1
There is a relationship between the coefficients Ak, φk and Xk, but the relationship is
more complicated than in the continuous-time case, given by (8.28). To develop that
relationship, begin with the second of these expansions and write
K
Ak
x(n) = A0 + ∑
(ei(kω0n+φk) + e−i(kω0n+φk)).
2
k=1
Note that since ω0 = 2π/p, then for all integers n, eiω0pn = 1, so
e−i(kω0n+φk) = e−i(kω0n+φk)eiω0 pn = ei(ω0(p−k)n−φk).
Thus
K
A
K
k
Ak
x(n)
=
A0 + ∑
eiφk eikω0n+ +
e−iφk eiω0(p−k)n
2
∑ 2
k=1
k=1
K
A
p−1
k
Ap−m
=
A0 + ∑
eiφk eikω0n+ +
e−iφp−m eiω0mn,
2
∑
2
k=1
m=K
by change of variables. Comparing this against (8.31),
A0
if k = 0
A
X
keiφk /2
if k ∈ {1, · · · , K − 1}
k =
Akeiφk /2 + Ake−iφk/2 = Ak cos(φk) if k = K
Ap−ke−iφp−k/2
if k ∈ {K + 1, · · · , p − 1}
This relationship is more complicated than (8.28). Fortunately, it is rare that we need to
use both forms of the DFS, so we can usually just pick one of the two forms and work
only with its set of coefficients.
Lee & Varaiya, Signals and Systems
347
8.6. SUMMARY
K = 1
K = 3
1.0
K = 7
K = 32
0.5
0.0
-0.5
-1.0
-0
1
2
3
4
5
6
7
8
-3
Time in seconds
x10
(a)
0.6
0.5
0.4
0.3
0.2
0.1
-0.0
-4
-3
-2
-1
0
1
2
3
4
3
Frequency in Hz
x10
(b)
Figure 8.9: (a) Some finite Fourier series approximations to one cycle of a square
wave. The number of Fourier series terms that are included in the approximation
is 2K + 1, so K is the magnitude of the largest index the terms. (b) The magnitude
of the complex Fourier series coefficients shown as a function of frequency.
348
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
Probing Further: Formula for Fourier series coefficients
To see that (8.32) is valid, try substituting for x(t) its Fourier series expansion,
∞
x(t) = ∑ Xkeikω0t
k=−∞
∞
to get
1 Z p
Xm =
∑ Xkeikω0te−imω0tdt.
p 0 k=−∞
Exchange the integral and sum (assuming this is valid, see box on page 350) to get
1
∞
Z
p
Xm =
∑ Xk
eikω0t e−imω0t dt.
p k=−
0
∞
The exponentials can be combined to get
1
∞
Z
p
Xm =
∑ Xk
ei(k−m)ω0t dt.
p k=−
0
∞
In the summation, where k varies over all integers, there will be exactly one term of the
summation where k = m. In that term, the integral evaluates to p. For the rest of the
terms, k = m. Separating these two situations, we can write
p
1
∞
Z
Xm = Xm +
∑
Xk
ei(k−m)ω0t dt,
p k=−∞,k=m 0
where the first term Xm is the value of the term in the summation where k = m. For each
remaining term of the summation, the integral evaluates to zero, thus establishing our
result. To show that the integral evaluates to zero, let n = k − m, and note that n = 0.
Then
Z
p
Z
p
Z
p
einω0t dt =
cos(nω0t)dt + i
sin(nω0t)dt
0
0
0
Since ω0 = 2π/p, these two integrals exactly span one or more complete cycles of the
cosine or sine, and hence integrate to zero.
Lee & Varaiya, Signals and Systems
349
8.6. SUMMARY
Probing Further: Exchanging integrals and summations
The demonstration of the validity of the formula for the Fourier series coefficients in the
box on page 349 relies on being able to exchange an integral and an infinite summation.
The infinite summation can be given as a limit of a sequence of functions
N
xN(t) = ∑ Xkeikω0t.
k=−N
Thus, we wish to exchange the integral and limit in
1 Z p
Xm =
( lim xN(t))dt.
p 0
N→∞
A sufficient condition for being able to perform the exchange is that the limit converges
uniformly in the interval [0, p]. A sufficient condition for uniform convergence is that x
is continuous and that its first derivative is piecewise continuous.
See R. G. Bartle, The Elements of Real Analysis, Second Edition, John Wiley & Sons,
1976, p. 241.
x
y
z
S 1
S 2
S
Figure 8.10: The cascade connection of the two LTI systems is the system S =
S2 ◦ S1. The frequency response is related by ∀ω, H(ω) = H1(ω)H2(ω).
350
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
x
u
y
S
+
1
z
S 2
S
Figure 8.11: The feedback connection of the two LTI systems S1, S2 is the LTI
system S.
The frequency response is related by ∀ω ∈ R, H(ω) = H1(ω)/[1 −
H1(ω)H2(ω)].
Lee & Varaiya, Signals and Systems
351
8.6. SUMMARY
Probing Further: Feedback systems are LTI
To show that S in Figure 8.11 is time-invariant we must show that for any τ ∈ R,
S(D (x)) = D (S(x)) = D (y),
(8.36)
τ
τ
τ
that is, we must show that D (y) is the output of S when the input is D (x). Now the
τ
τ
left-hand side of (8.35) with y replaced by D (y) is
τ
D (y) − S
(y)))
=
D (y) − D (S
τ
1(S2(Dτ
τ
τ
1(S2(y)),
since S1 and S2 are time-invariant
=
D (y − S
is linear,
τ
1(S2(y))), since Dτ
=
D (S
τ
1(x)), by (8.35)
=
S1(D (x)), since S
τ
1 is time-invariant
so that D (y) is indeed the solution of (8.35) when the input is D (x). This proves (8.36).
τ
τ
Linearity is shown similarly. Let a be any complex number. To show that ay is the output
when the input is ax, we evaluate the left-hand side of (8.35) at ay,
ay − S1(S2(ay)) = ay − aS1(S2(y)), since S2 and S1 are linear
=
a[y − S1(S2(y))]
=
aS1(x), by (8.35)
=
S1(ax), since S1 is linear
which shows that S(ax) = aS(x).
Now suppose w is another input and z = S(w) is the corresponding output. I.e.,
z − S1(S2(z)) = S1(w).
(8.37)
We evaluate the left-hand side of (8.35) at y + z, using that S2 and S1 are linear,
(y + z) − S1(S2(y + z)) = [y − S1(S2(y))] + [z − S1(S2(z)],
=
S1(x) + S1(w), by (8.35) and (8.37)
=
S1(x + w), since S1 is linear
and so S(x + w) = y + z = S(x) + S(z).
352
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
9
8
7
6
5
4
3
amplitude response
2
1
00
0.5
1
1.5
2
2.5
3
3.5
frequency (radians/sample)
Figure 8.12: The magnitude response of the feedback composition of example
Lee & Varaiya, Signals and Systems
353
8.6. SUMMARY
x
y
u
x
y
S
+
+
1
S 2
S
+
1
S 3
S
S
+
S 2
S 3
Figure 8.13: The composition on the left can be expressed as the one on the
right, where S3 is the system at the bottom.
354
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
Exercises
Each problem is annotated with the letter E, T, C which stands for exercise, requires some
thought, requires some conceptualization. Problems labeled E are usually mechanical,
those labeled T require a plan of attack, those labeled C usually have more than one
defensible answer.
1. E Find A ∈ C so that
∀ t ∈ Reals, Aeiωt + A∗e−iωt = cos(ωt + π/4),
where A∗ is the complex conjugate of A.
2. E Plot the function s : R → R given by
∀ x ∈ R, s(x) = Im{e(−x+i2πx)}.
You are free to choose a reasonable interval for your plot, but be sure it includes
x = 0.
3. E This exercise explores the fact that a delay in a sinewave causes a phase shift.
That is, for any real numbers τ and ω, there is a phase shift φ ∈ R such that for all
t ∈ R,
sin(ω(t − τ)) = sin(ωt − φ).
Give φ in terms of τ, ω. What are the units of τ, ω, φ? Hint: The argument to the
sin function has units of radians.
4. E Let x : R → R. Show that x is periodic with period p if and only if Dp(x) = x.
Now show that if S is a time-invariant system and x is a periodic signal, then S(x)
is also periodic with period p.
5. E Analogously to D in (8.1) and D
τ
M in Section 8.1.3, define formally the following
variants:
(a) A shift operator Sv,h that shifts an image v units vertically and h units horizon-
tally, where v ∈ R and h ∈ R.
(b) A shift operator Sm,n that shifts a discrete image m units vertically and n units
horizontally, where m ∈ Z and n ∈ Z.
Lee & Varaiya, Signals and Systems
355
EXERCISES
6. E Consider a discrete-time system D : [Z → R] → [Z → R], where if y = D(x) then
∀ n ∈ Z, y(n) = x(n − 1).
(a) Is D linear? Justify your answer.
(b) Is D time-invariant? Justify your answer.
7. E Consider a continuous-time system TimeScale : [R → R] → [R → R], where if
y = TimeScale(x) then
∀ t ∈ R, y(t) = x(2t).
(a) Is TimeScale linear? Justify your answer.
(b) Is TimeScale time-invariant? Justify your answer.
8. E Consider the continuous-time signal x where
∀ t ∈ R, x(t) = 1 + cos(πt) + cos(2πt).
Suppose that x is the input to an LTI system with frequency response given by
∀
eiω
if |ω| < 4 radians/second
ω ∈ R,
H(ω) =
0
otherwise
Find the output y of the system.
9. T Suppose that the continuous-time signal x : R → R is periodic with period p. Let
the fundamental frequency be ω0 = 2π/p. Suppose that the Fourier series coeffi-
cients for this signal are known constants A0, A1, A2, · · · and φ1, φ2, · · · . Give the
Fourier series coefficients A , A , A , · · · and
, ,··· for each of the following
0
1
2