Background

    We will now review some ideas from linear algebra. Proofs of the theorems are either left as exercises or can be found in any standard text on linear algebra.  We know how to solve n linear equations in n unknowns.  It was assumed that the determinant of the matrix was nonzero and hence that the solution was unique. In the case of a homogeneous system AX = 0, if  ,  the unique solution is the trivial solution X = 0.  If  ,  there exist nontrivial solutions to AX = 0. Suppose that  ,  and consider solutions to the homogeneous linear
system

A homogeneous system of equations always has the trivial solution .  Gaussian elimination can be used to obtain the reduced row echelon form which will be used to form a set of relationships between the variables, and a non-trivial solution.  

Example 1.  Find the nontrivial solutions to the homogeneous system  
        
Solution 1.

Background for Eigenvalues and Eigenvectors

Definition (Linearly Independent).  The vectors    are said to be linearly independent if the equation  

        

implies that .   If the vectors are not linearly independent they are said to be linearly dependent.  

Two vectors in are linearly independent if and only if they are not parallel.  Three vectors in   are linearly independent if and only if they do not lie in the same plane.

Definition (Linearly Dependent).  The vectors     are said to be linearly dependent if there exists a set of numbers not all zero, such that

         .

Theorem. The vectors    are linearly dependent if and only if at least one of them is a linear combination of the others.

    A desirable feature for a vector space is the ability to express each vector as s linear combination of vectors chosen from a small subset of vectors.  This motivates the next definition.

Definition (Basis).  Suppose that   is a set of  m  vectors in .  The set  S i s called a basis for    if for every vector    there exists a unique set
of scalars so that  X  can be expressed as the linear combination  

        

Theorem.  In  ,  any set of  n  linearly independent vectors forms a basis of  .  Each vector    is uniquely expressed as a linear combination of the basis vectors.

Theorem.  Let    be vectors in  .  

(i)    If  m>n,  then the vectors are linearly independent.  

(ii)    If  m=n,  then the vectors are linearly dependent if and only if   ,  where  .  

Proof  Eigenvalues and Eigenvectors  Eigenvalues and Eigenvectors  

    Applications of mathematics sometimes encounter the following questions:  What are the singularities of  ,  where    is a parameter?  What is the behavior of the sequence of vectors  ?  What are the geometric features of a linear transformation?  Solutions for problems in many different disciplines, such as economics, engineering, and physics, can involve ideas related to these equations. The theory of eigenvalues and eigenvectors is powerful enough to help solve these otherwise intractable problems.

    Let  A  be a square matrix of dimension  n × n  and let  X  be a vector of dimension  n.  The product  Y = AX  can be viewed as a linear transformation from  n-dimensional space into itself.  We want to find scalars    for which there exists a nonzero vector  X  such that

(1)          ;  

that is, the linear transformation  T(X) = AX  maps  X  onto the multiple  .  When this occurs, we call  X  an eigenvector that corresponds to the eigenvalue  , and together they form the eigenpair   for  A.  In general, the scalar    and vector  X  can involve complex numbers.  For simplicity, most of our illustrations will involve real calculations.  However, the techniques are easily extended to the complex case.  The n × n identity matrix  I  can be used to write equation (1) in the form

(2)        .  

The significance of equation (2) is that the product of the matrix    and the nonzero vector  X  is the zero vector!  The theorem of homogeneous linear system says that (2) has nontrivial solutions if and only if the matrix    is singular, that is,

(3)        .  

This determinant can be written in the form  

(4)        

Definition (Characteristic Polynomial).  When the determinant in (4) is expanded, it becomes a polynomial of degree n, which is called the characteristic polynomial   

(5)             

Exploration for p()

    There exist exactly n roots (not necessarily distinct) of a polynomial of degree n.  Each root    can be substituted into equation (3) to obtain an underdetermined system of equations that has a corresponding nontrivial solution vector X.  If     is real, a real eigenvector  X  can be constructed. For emphasis, we state the following definitions.

Definition (Eigenvalue).  If  A  is and  n × n  real matrix, then its  n  eigenvalues    are the real and complex roots of the characteristic polynomial  

        .

Definition (Eigenvector).  If     is an eigenvalue of  A  and the nonzero vector  V  has the property that

              

then  V  is called an eigenvector of  A  corresponding to the eigenvalue  .  Together, this eigenvalue and eigenvector V is called an eigenpair  .

    The characteristic polynomial   can be factored in the form

          
        
where    is called the multiplicity of the eigenvalue . The sum of the multiplicities of all eigenvalues is  n;  that is,

        .  

The next three results concern the existence of eigenvectors.

Theorem (Corresponding Eigenvectors).  Suppose that  A  is and  n × n  square matrix.    
(a)     For each distinct eigenvalue    there exists at least one eigenvector  V  corresponding to  .
(b)     If    has multiplicity  r,  then there exist at most  r  linearly independent eigenvectors    that correspond to  .

Theorem (Linearly Independent Eigenvectors).  Suppose that  A  is and  n × n  square matrix.  If the eigenvalues are distinct and    are the k eigenpairs, then   is a set of  k linearly independent vectors.

Theorem (Complete Set of Eigenvectors).  Suppose that  A  is and  n × n  square matrix.  If the eigenvalues of  A  are all distinct, then there exist  n  nearly independent eigenvectors  .  

Proof  Eigenvalues and Eigenvectors  Eigenvalues and Eigenvectors  

    Finding eigenpairs by hand computations is usually done in the following manner.  The eigenvalue    of multiplicity  r  is substituted into the equation

        .  

Then Gaussian elimination can be performed to obtain the row reduced echelon form, which will involve  n-k  equations in  n  unknowns, where  .  Hence there are  k  free variables to choose. The free variables can be selected in a judicious manner to produce  k  linearly independent solution vectors that correspond to  .

Computer Programs  Eigenvalues and Eigenvectors  Eigenvalues and Eigenvectors  

Example 2.  Find the eigenvalues and eigenvectors of the matrix  .
Solution 2.

Free Variables

    When the linear system is underdetermined, we needed to introduce free variables in the proper location. The following subroutine will rearrange the equations and introduce free variables in the location they are needed.  Then all that is needed to do is find the row reduced echelon form a second time.  This is done at the end of the next example.

Example 3.  Find the eigenvalues and eigenvectors of the matrix  .  
Solution 3.

Example 4.  Find the eigenvalues and eigenvectors of the matrix  .
Solution 4.

Example 5.  Find the eigenvalues and eigenvectors of the matrix  .
Solution 5.

Example 6.  Find the eigenvalues and eigenvectors of the matrix  .  
Solution 6.

Example 7.  Find the eigenvalues and eigenvectors of the matrix  .  
Solution 7.

Example 8.  Find the eigenvalues and eigenvectors of the matrix  .
Solution 8.

Example 9.   Find the eigenvalues and eigenvectors of the matrix  .
Solution 9.

Example 10. Find the eigenvalues of the matrix   .
Solution 10.

Example 11.  Find the eigenvalues of the following matrix  A.
Solution 11.

Research Experience for Undergraduates

Eigenvalues and Eigenvectors  Eigenvalues and Eigenvectors  Internet hyperlinks to web sites and a bibliography of articles.  

Download this Mathematica Notebook Eigenvalues and Eigenvectors

Return to Numerical Methods - Numerical Analysis

(c) John H. Mathews 2004