As pointed out in the units on Expectation and Variance, the mathematical expectation E[X]=μX of a random variable X locates the center of mass for the induced distribution, and the expectation
measures the spread of the distribution about its center of mass. These quantities are also
known, respectively, as the mean (moment) of X and the second moment of X about the mean. Other moments
give added information. For example, the third moment about the mean 
gives
information about the skew, or asymetry, of the distribution about the mean. We investigate
further along these lines by examining the expectation of certain functions of X. Each of these
functions involves a parameter, in a manner that completely determines the distribution.
For reasons noted below, we refer to these as transforms. We consider three of
the most useful of these.
We define each of three transforms, determine some key properties, and use them to study various probability distributions associated with random variables. In the section on integral transforms, we show their relationship to well known integral transforms. These have been studied extensively and used in many other applications, which makes it possible to utilize the considerable literature on these transforms.
Definition. The moment generating functionMX for random variable X (i.e., for its distribution) is the function
The characteristic functionφX for random variable X is
The generating functiongX(s) for a nonnegative, integer-valued random variable X is
The generating function 
has meaning for more general random variables,
but its usefulness is greatest for nonnegative, integer-valued variables, and we limit our
consideration to that case.
The defining expressions display similarities which show useful relationships. We note two which are particularly useful.
Because of the latter relationship, we ordinarily use the moment generating function instead of the characteristic function to avoid writing the complex unit i. When desirable, we convert easily by the change of variable.
The integral transform character of these entities implies that there is essentially a one-to-one relationship between the transform and the distribution.
Moments
The name and some of the importance of the moment generating function arise from the fact that the derivatives of MX evaluateed at s=0 are the moments about the origin. Specifically
Since expectation is an integral and because of the regularity of the integrand, we may differentiate inside the integral with respect to the parameter.
Upon setting s=0, we have MX'(0)=E[X]. Repeated differentiation gives
the general result. The corresponding result for the characteristic function is
.
The density function is fX(t)=λe–λt for t≥0.
From this we obtain 
.
The generating function does not lend itself readily to computing moments, except that
For higher order moments, we may convert the generating function to the moment generating function by replacing s with es, then work with MX and its derivatives.
, so that
We convert to MX by replacing s with es to get 
. Then
so that
These results agree, of course, with those found by direct computation with the distribution.
Operational properties
We refer to the following as operational properties.
| (T1): If Z=aX+b, then
(13.15)
For the moment generating function, this pattern follows from
 (13.16)
Similar arguments hold for the other two.
 |
(T2): If the pair  is independent, then
(13.17)
For the moment generating function, esX and esY form an independent pair for
each value of the parameter s. By the product rule for expectation
 (13.18)
Similar arguments are used for the other two transforms. A partial converse for (T2) is as follows: |
(T3): If MX+Y(s)=MX(s)MY(s), then the pair  is uncorrelated.
To show this, we obtain two expressions for  , one by direct expansion and
use of linearity, and the other by taking the second derivative of the moment generating
function.
(13.19)  (13.20)
On setting s=0 and using the fact that MX(0)=MY(0)=1, we have
 (13.21)
which implies the equality E[XY]=E[X]E[Y].
 |
Note that we have not shown that being uncorrelated implies the product rule.
We utilize these properties in determining the moment generating and generating functions for several of our common distributions.
Some discrete distributions
Indicator function
Simple random variable
(primitive form) 
Binomial
. 
We use the product rule for sums of independent random variables and the generating
function for the indicator function.
Geometric(p). P(X=k)=pqk∀k≥0E[X]=q/p We use the formula for the geometric series to get
Negative binomial
If Ym is the number of the trial in a Bernoulli sequence on which the mth success occurs, and
Xm=Ym–m is the number of failures before the mth success, then
The power series expansion about t=0 shows that
Hence
Comparison with the moment generating function for the geometric distribution shows that Xm=Ym–m has the same distribution as the sum of m iid random variables, each geometric (p). This suggests that the sequence is characterized by independent, successive waiting times to success. This also shows that the expectation and variance of Xm are m times the expectation and variance for the geometric. Thus
Poisson(μ)
In Example 13.2, above, we establish gX(s)=eμ(s–1) and 
.
If {X,Y} is an independent pair, with X∼ Poisson (λ) and Y∼ Poisson (μ),
then Z=X+Y∼ Poisson (λ+μ). Follows from (T1) and product of exponentials.
Some absolutely continuous distributions
Uniform on 
Symmetric triangular
where MY is the moment generating function for Y∼ uniform 
and similarly
for MZ. Thus, X has the same distribution as the difference of two independent random
variables, each uniform on 
.
Exponential(λ)
In example 1, above, we show that 
.
Gamma
For α=n, a positive integer,
which shows that in this case X has the distribution of the sum of n independent random variables each exponential (λ).
Normal
.
The stan