0
!
x
z
D!
S
0
0
0
!
x
y
S
D!
Figure 8.2: Time invariance implies that the top and bottom systems produce the
same output signal for the same input signal.
This is interpreted as follows:
A system S is time invariant if, for any input x that produces output y, a delayed input
D (x) produces output D (y).
τ
τ
Similarly, the discrete-time M-sample delay is written DM. The signal y = DM(x) is given
by
∀ n ∈ Z, y(n) = x(n − M).
(8.3)
A discrete-time system S is time invariant if
∀ M,x, S(DM(x)) = DM(S(x)).
(8.4)
Example 8.2: Consider a discrete-time system S,
S : [Z → R] → [Z → R].
Suppose that any input x produces output y where
∀ n ∈ Z, y(n) = x(n) + 0.9x(n − 1).
(8.5)
This system is time-invariant. To show this, consider a delayed input ˆ
x = DM(x),
for some integer M. That is,
∀ n, ˆx(n) = x(n − M).
(8.6)
Lee & Varaiya, Signals and Systems
315
8.1. LTI SYSTEMS
Suppose that this input produces output ˆ
y = S( ˆ
x). Then by the relation (8.5) be-
tween an input and an output,
∀ n, ˆy(n) = ˆx(n) + 0.9 ˆx(n − 1).
Substituting (8.6), we see that
∀ n, ˆy(n) = x(n − M) + 0.9x(n − M − 1) = y(n − M).
Since ˆ
y = DM(y), the system is time-invariant.
Example 8.3: Consider the system DelayAndSquare, or DS for short,
DS : [R → R] → [R → R].
Suppose that any input x produces output y where
∀ t ∈ R, y(t) = (x(t − 1))2.
(8.7)
This system is time-invariant. To show this, consider a delayed input ˆ
x = D (x), for
τ
some real number τ. That is,
∀ t, ˆx(t) = x(t − τ).
(8.8)
Suppose that this input produces output ˆ
y = S( ˆ
x). Then by the relation (8.7) be-
tween an input and an output,
∀ t, ˆy(t) = ( ˆx(t − 1))2.
Substituting (8.8), we see that
∀ t, ˆy(t) = (x(t − 1 − τ))2 = y(t − τ).
Since ˆ
y = D (y), the system is time-invariant.
τ
316
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
Example 8.4: Consider a system ReverseTime or RT,
RT : [R → R] → [R → R],
where any input x produces output y related by
∀ t ∈ R, y(t) = x(−t).
(8.9)
This system is not time-invariant. To show this, consider a delayed input ˆ
x = D (x),
τ
for some real number τ. That is,
∀ t, ˆx(t) = x(t − τ).
(8.10)
Suppose that this input produces output ˆ
y = S( ˆ
x). Then by the relation (8.9) be-
tween an input and an output,
∀ t, ˆy(t) = ˆx(−t).
Substituting (8.10), we see that
∀ t, ˆy(t) = x(−t − τ) = y(t − τ) = x(−t + τ).
Since ˆ
y = D (y), the system is not time-invariant.
τ
To be completely convinced that these two signals are different in general, consider
a particular signal x such that ∀t, x(t) = t, and take τ = 1. Then ˆy(0) = −1, but
(D (y))(0) = 1.
τ
Time invariance is a mathematical fiction. No electronic system is time invariant in the
strict sense. For one thing, such a system is turned on at some point in time. Clearly,
its behavior before it is turned on is not the same as its behavior after it is turned on.
Nevertheless, it proves to be a very convenient mathematical fiction, and it is a reasonable
approximation for many systems if their behavior is constant over a relatively long period
of time (relative to whatever phenomenon we are studying). For example, your audio
amplifier is not a time-invariant system. Its behavior changes drastically when you turn it
on or off, and changes less drastically when you raise or lower the volume. However, for
Lee & Varaiya, Signals and Systems
317
8.1. LTI SYSTEMS
the duration of a compact disc, if you leave the volume fixed, the system can be reasonably
approximated as being time-invariant.
Some systems have a similar property even though they operate on signals whose domain
is not time. For example, the domain of an image is a region of a plane. The output
of an image processing system may not depend significantly on where in the plane the
input image is placed. Shifting the input image will only shift the output image by the
same amount. This property which generalizes time invariance and holds for some image
processing systems, is called shift invariance (see problem 5).
8.1.2
Linearity
Consider the set of signals whose range is R or C . Such signals are real-valued functions
or complex-valued functions. Since real-valued functions are a subset of complex-valued
functions, we only need to talk about complex-valued functions. It does not matter (for
now) whether they are continuous-time signals or discrete-time signals. The domain could
be R or Z.
Suppose x is a complex-valued function and a is a complex constant. Then we can define
a new complex-valued function ax such that for all t in the domain of x,
(ax)(t) = a(x(t)).
In other words, the new function, which we call ax, is simply scaled by the constant a.
Similarly, given two complex-valued functions x1 and x2 with the same domain and range,
we can define a new function (x1 + x2) such that for all t in the domain,
(x1 + x2)(t) = x1(t) + x2(t).
Consider the set of all systems that map complex-valued functions to complex-valued
functions. Such systems are called complex systems. Again, it does not matter (for now)
whether they are discrete-time systems or continuous-time systems. Suppose that S is a
complex system. S is said to be linear if for all a ∈ C and for all complex signals x,
S(ax) = aS(x)
(8.11)
and for all complex signals x1 and x2 in the domain of S,
S(x1 + x2) = S(x1) + S(x2)
(8.12)
318
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
x
a
S
x
S
a
Figure 8.3: If S is linear, then these two systems are equivalent. The triangle
represents a system that scales a signal by some complex constant a.
The first of these—called the homogeneity property—says that if you scale the input, the
output is scaled. The second one—called the additivity property—says that if the input
is described as the sum of two component signals, then the output can be described as
the sum of two signals that would result from the components alone. Recall that linear
functions were introduced in Section 5.2. A linear system is one whose function relating
the output to the input is linear.
In pictures, the first property says that the two systems in Figure 8.3 are equivalent if S is
linear. Here, the triangle represents the scaling operation. The second property says that
the two systems in Figure 8.4 are equivalent.
Example 8.5:
Consider the same discrete-time system S of example 8.2, where
input x produces output y such that
∀ n ∈ Z, y(n) = x(n) + 0.9x(n − 1).
This system is linear. To show this, we must show that (8.11) and (8.12) hold.
Suppose input ˆ
x = ax produces output ˆ
y = S( ˆ
x). Then
∀ n, ˆy(n) = ˆx(n) + 0.9 ˆx(n − 1)
=
ax(n) + 0.9ax(n − 1)
=
a(x(n) + 0.9x(n − 1))
=
ay(n).
Lee & Varaiya, Signals and Systems
319
8.1. LTI SYSTEMS
x
S
y
x
S
y
S
Figure 8.4: If S is linear, then these two systems are equivalent.
Thus, S(ax) = aS(x), establishing (8.11).
To check (8.12), suppose input x1 produces output y1 and x2 produces y2. Let
x = x1 + x2 produce y. We must show that y = y1 + y2. We leave this as an exercise.
Example 8.6: In the continuous-time system DelayAndSquare or DS of example
8.3, an input x produces output y where
∀ t ∈ R, y(t) = (x(t − 1))2.
This system is not linear. To show this we must show that either (8.11) or (8.12)
does not hold for DS. We will show that (8.11) does not hold. To show this, consider
a scaled input ˆ
x = ax, for some complex number a. Suppose that this input produces
output ˆ
y = S( ˆ
x). Then,
∀ t, ˆy(t) = ( ˆx(t − 1))2.
Since ˆ
x = ax,
∀ t, ˆy(t) = (ax(t − 1))2 = a2(x(t − 1))2 = ay(t) = a(x(t − 1))2.
320
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
In particular, take for example a = 2, t = 0, and x such that ∀ t, x(t) = 1. These
values result in
ˆ
y(t) = 4 = ay(t) = 2.
Example 8.7: In the time-reversal system RT of example 8.4,
RT : [R → R] → [R → R],
any input x produces output y related by
∀ t ∈ R, y(t) = x(−t).
This system is linear. To show this we must show that (8.11) and (8.12) both hold.
We first show that (8.11) holds; that is, for all a ∈ R, x ∈ [R → R], and t ∈ R,
RT(ax)(t) = (aRT(x))(t).
But this is certainly true, since the left and right sides both evaluate to ax(−t). A
similar argument is used to show (8.12).
Linearity is a mathematical fiction. No electronic system is linear in the strict sense. A
system is designed to work with a range of input signals, and arbitrary scaling of the input
does not translate into arbitrary scaling of the output. If you provide an input to your
audio amplifier that is higher voltage than it is designed for, then it is not likely to just
produce louder sounds. Its input circuits will get overloaded and signal distortion will
result. Nonetheless, as a mathematical fiction, linearity is extremely convenient. It says
that we can decompose the inputs to a system and study the effect of the system on the
individual components.
8.1.3
Linearity and time-invariance
For time-domain systems, time-invariance is a useful (if fictional) property. For com-
plex (or real) systems, linearity is a useful (if fictional) property. For complex (or real)
Lee & Varaiya, Signals and Systems
321
8.1. LTI SYSTEMS
time-domain systems, the combination of these properties is extremely useful. Linear
time-invariant (LTI) systems turn out to have particularly simple behavior with sinusoidal
inputs.
Given a sinusoid at the input, the output of an LTI system will be a sinusoid with the
same frequency, but possibly with different phase and amplitude.
It then follows that
Given an input that is described as a sum of sinusoids of certain frequencies, the output
can be described as a sum of sinusoids with the same frequencies, but with (possible)
phase and amplitude changes at each frequency.
A straightforward way to show that LTI systems have these properties starts by consid-
ering complex exponentials (for a review of complex numbers, see Appendix B). A
continuous-time complex exponential is a signal x ∈ [R → C] where
∀ t ∈ R, x(t) = eiωt = cos(ωt) + isin(ωt).
Complex exponential functions have an interesting property that will prove useful to us.
Specifically,
∀ t ∈ R and τ ∈ R, x(t − τ) = eiω(t−τ) = e−iωτeiωt.
This follows from the multiplication property of exponentials,
eb+c = ebec.
Since D (x)(t) = x(t −
τ
τ), we have that for the complex exponential x,
D (x) = ax, where a = e−iωτ.
(8.13)
τ
In words, a delayed complex exponential is a scaled complex exponential, where the
scaling constant is the complex number a = e−iωτ.
We will now show that if the input to a continuous-time LTI system is eiωt , then the output
will be H(ω)eiωt , where H(ω) is a constant (not a function of time) that depends on the
frequency ω of the complex exponential. In other words, the output is only a scaled
version of the input.
When the output of a system is only a scaled version of the input, the input is called an
eigenfunction, which comes from the German word for “same.” The output is (almost)
the same as the input.
322
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
Complex exponentials are eigenfunctions of LTI systems, as we will now show. This is
the most important reason for the focus on complex exponentials in the study of signals
and systems. This single property underlies much of the discipline of signal processing,
and is used heavily in circuit analysis, communication systems, and control systems.
Given an LTI system S : [R → C] → [R → C], let x be an input signal where
∀ t ∈ R, x(t) = eiωt
Recall that S is time invariant if for all τ ∈ R,
S ◦ D = D ◦ S.
τ
τ
Thus
S(D (x)) = D (S(x)).
τ
τ
From (8.13),
S(D (x)) = S(ax)
τ
where a = e−iωτ, and from linearity,
S(ax) = aS(x)
so
aS(x) = D (S(x)).
(8.14)
τ
Let y = S(x) be the corresponding output signal. Substituting into (8.14) we get
ay = D (y).
τ
In other words,
∀ t,τ ∈ R, e−iωτy(t) = y(t − τ).
In particular, this is true for t = 0, so letting t = 0,
∀ τ ∈ R, y(−τ) = e−iωτy(0).
Changing variables, letting t = −τ , we note that this implies that
∀ t ∈ R, y(t) = eiωty(0).
Recall that y(0) is the output evaluated at 0 when the input is eiωt . It is a constant, in that
it does not depend on t, so this establishes that the output is a complex exponential, just
Lee & Varaiya, Signals and Systems
323
8.1. LTI SYSTEMS
like the input, except that it is scaled by y(0). However, y(0) does, in general, depend on
ω, so we define the function H : R → C by
∀ ω ∈ R, H(ω) = y(0) = (S(x))(0), where ∀ t ∈ R, x(t) = eiωt.
(8.15)
That is, H(ω) is the output at time zero when the input is a complex exponential with
frequency ω.
Using this notation, we write the output y as
∀ t ∈ R, y(t) = H(ω)eiωt
when the input is eiωt . Note that H(ω) is a function of ω ∈ R, the frequency of the input
complex exponential.
The function H : R → C is called the frequency response. It defines the response of the
LTI system to a complex exponential input at any given frequency. It gives the scaling
factor that the system imposes on that complex exponential.
For discrete-time systems, the situation is similar. By reasoning identical to that above,
for an LTI system, if the input is a discrete complex exponential,
∀ n ∈ Z, x(n) = eiωn
then the output is the same complex exponential scaled by a constant (a complex number
that does not depend on time),
∀ n ∈ Z, y(n) = H(ω)eiωn
H is once again called the frequency response, and since it is a function of ω, and is
possibly complex valued, it has the form H : R → C.
There is one key difference, however, between discrete-time systems and continuous-time
systems. Since n is an integer, notice that
eiωn = ei(ω+2π)n = ei(ω+4π)n,
and so on. That is, a discrete complex exponential with frequency ω is identical to a
discrete complex exponential with frequency ω + 2Kπ, for any integer K. The frequency
response, therefore, must be identical at these frequencies, since the inputs are identical.
That is
∀ω ∈ R, H(ω) = H(ω + 2Kπ)
for any integer K. That is, a discrete-time frequency response is periodic with period 2π.
324
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
8.2
Finding and using the frequency response
We have seen that if the input to an LTI system is a complex exponential signal x ∈ [R →
C] where
∀ t ∈ R, x(t) = eiωt = cos(ωt) + isin(ωt).
then the output is
∀t ∈ R, y(t) = H(ω)eiωt.
(8.16)
where H(ω) is a (possibly complex-valued) number that is a property of the system. H(ω)
is called the frequency response at frequency ω.
Example 8.8:
Consider a delay system S = DT , for some T ∈ R. It is an LTI
system, as is easy to verify by checking that (8.2), (8.11), and (8.12) are satisfied.
Suppose the input to the delay system is the complex exponential x given by
∀ t ∈ R, x(t) = eiωt.
Then the output y satisfies
∀ t ∈ R, y(t) = eiω(t−T) = e−iωT eiωt.
Comparing this to (8.16) we see that the frequency response of the delay is
H(ω) = e−iωT .
Example 8.9: Consider a discrete-time M-sample delay system S = DM. If y =
S(x) then y is given by
∀ n ∈ Z, y(n) = x(n − M).
(8.17)
This is an LTI system, as is easy to verify. We could find the frequency response
exactly the same way as in the previous example, but instead we use a slightly
different method. Since the system is LTI, we know that if the input is x such that
Lee & Varaiya, Signals and Systems
325
8.2. FINDING AND USING THE FREQUENCY RESPONSE
for all n ∈ Z, x(n) = eiωn, then the output is H(ω)eiωn, where H is the frequency
response. By plugging this input and output into (8.17), we get
H(ω)eiωn = eiω(n−M) = eiωne−iωM.
Divide both sides by eiωn to get
H(ω) = e−iωM.
The techniques in the previous examples can be used to find the frequency response of
more complicated systems. Simply replace the input x in a difference equation like (8.17)
with eiωn, and replace the output y with H(ω)eiωn, and then solve for H(ω).
Example 8.10: Consider a discrete-time, length two moving average, given by the
difference equation
∀ n ∈ Z, y(n) = (x(n) + x(n − 1))/2,
where x is the input and y is the output. When the input for all n is eiωn, this becomes
H(ω)eiωn = (eiωn + eiω(n−1))/2.
Solving for H(ω), we find that the frequency response is
H(ω) = (1 + e−iω)/2.
A similar approach can find the frequency response of a continuous-time system described
by a differential equation.
Example 8.11:
Consider a continuous-time system with input x and output y
related by the differential equation
∀
dy
t ∈ R,
RC
(t) + y(t) = x(t),
(8.18)
dt
326
Lee & Varaiya, Signals and Systems
8. FREQUENCY RESPONSE
R
+
+
x( t)
C
y( t)
-
-
Figure 8.5: The RC circuit of example 8.11.
where R and C are real-valued constants. This differential equation describes the
RC circuit of Figure 8.5, which consists of an R-ohm resistance in series with a C-
farad capacitor. The circuit has input voltage x, provided by a voltage source, shown
as a circle on the left. The output is the voltage y across the capacitor. Kirchhoff’s
voltage law gives the differential equation. It is easy to verify that this differntial
equation describes an LTI system.
We can determine the frequency response of this system by assuming that the input
x is given by ∀ t ∈ R, x(t) = eiωt, and finding the output. Since the system is LTI,
the output will be given by y(t) = H(ω)eiωt . Plugging these values for x and y into
(8.18) gives
RC(iω)H(ω)eiωt + H(ω)eiωt = eiωt ,
(8.19)
because
dy (t) = iωH(ω)eiωt.
dt
Dividing both sides of (8.19) by eiωt yields the frequency response of this circuit,
∀
1
ω ∈ R,
H(ω) =
.
1 + iRCω
Lee & Varaiya, Signals and Systems
327
8.2. FINDING AND USING THE FREQUENCY RESPONSE
8.2.1
Linear difference and differential equations
The procedure of example 8.10 can be used to write down by inspection the frequency
response of any high-order linear difference equation of the form, ∀ n ∈ Z,
a0y(n) + a1y(n −1)+· · ·+aNy(n−N) = b0x(n)+b1x(n−1)+· · ·+bMx(n−M). (8.20)
The coefficients of this difference equation, a0, · · · , aN and b0, · · · , bM, are real constants.
(They could also be complex.) This describes an LTI system, and a good way to recognize
that a discrete-time system is LTI is to write it in this form. If the input for all n is
x(n) = eiωn, then the output for all n is y(n) = H(ω)eiωn. Plugging these values of input
and output into (8.20) gives,
a0H(ω)eiωn + a1H(ω)eiω(n−1) + · · · + aNH(ω)eiω(n−N)
= b0eiωn + b1eiω(n−1) + · · · + bMeiω(n−M).
Recognizing that eiω(n−m) = e−imωeiωn, we can divide both sides by eiωn and solve for
H(ω) to get