## Linear Maps between vector spaces: Basic Definitions

### Linear Maps

A linear map between two vectors spaces and is a function such that :

Notice that the addition of two linear maps and their multiplication by scalars produce linear map as well, which imply that the set of linear maps is also a linear vector space.
It is important not to confuse linear maps with affine maps. For example, the function defined such that is not a linear map but rather an affine map. is not a liner map since in general . On the other hand, the function defined such that is indeed a linear map.

#### Tensors

Linear maps between finite dimensional linear vector spaces can also be referred to as Tensors. Tensor analysis provides a natural and concise mathematical tool for the analysis of various engineering problems, in particular, solid mechanics. For a detailed description of tensors, refer to the Wolfram article on tensors.

According to wikipedia, the origin of the word “Tensors” dates back to the nineteenth century when it was introduced by Woldemar Voigt. It is likely that the word originated because one of the early linear operators introduced was the symmetric Cauchy stress matrix which functions to convert area vectors to force vectors. At the time, perhaps the scientists were interested in things that “stretch” and thus, the word “Tensor” from the Latin route “Tendere” came about.

### Kernel of Linear Maps

Let be a linear map between two vector spaces and . Then, the kernel of or is the set of all vectors that are mapped into the zero vector, i.e.:

For example, consider the linear map defined such that . Then, the kernel of this linear map consists of al the vectors in that are mapped to zero, i.e., the vectors whose components and satisfy:

There are infinitely many vectors that satisfy this condition. The set of all those vectors is given as:

### Matrix Representation of Linear Maps

The matrix representation of linear maps is the most convenient way to represent linear maps when orthonormal basis sets are chosen for the underlying vector spaces. Consider the linear map . Let and be the orthonormal basis sets for and respectively. Then, because of the linearity of the map, the map is indeed well defined by the components of the vectors . Since we can assume that it has components which can be denoted as follows:

Therefore, and its image under has the form:

Which in traditional matrix form admits the representation:

Notice that the vectors are the column vectors of the matrix representation .

#### Matrix Representation and Change of Basis

The components of the matrix representation of defined above depend on the choice of the orthonormal basis sets for each vector space. For the discussion in this section, we will restrict ourselves to square matrices, i.e., linear maps between vector spaces of the same dimension.
Let . Let be the chosen orthonormal basis set for both vector spaces and let be another orthonormal basis set and let be the matrix of coordinate transformation as defined in the Change of Basis section. The matrix representation of when is chosen as the basis set is denoted by . The relationship between and can be obtained as follows:
Let , denote . Let and denote the representation of and when is chosen as the coordinate system. Therefore in each coordinate system we have:

In addition, the relationship between the coordinates in the two coordinate systems is given by:

Therefore:

This is true for every , therefore:

In the following tool, you can choose the components of the matrix and the vector along with an angle of the counterclockwise rotation of the coordinate system. The tool then applies the transformation of coordinates from the coordinate system , to where , are vectors rotated by counterclockwise from , and . On the left hand side, the tool draws the vector in blue, the vector in red, the original coordinate system in black, and the vectors of the new coordinate system in dashed black. At the bottom of the left hand side drawing you will find the expressions for , , , , and using the basis set . On the right hand side, the tool draws the vectors in blue, in red, and the new coordinate system in black. At the bottom of the right hand side, you will find the expressions for , , and using the basis set .

Similarly, the following tool is for three dimensional Euclidean vector spaces. The new coordinate system is obtained by simultaneously applying a counterclockwise rotation , , and around the first, second, and third coordinate system axis, respectively.

### Tensor Product

Let and . The tensor product denoted by is a linear map defined such that :

In simple words, the tensor product defined above utilizes the linear dot product operation and a fixed vector to produce a real number using the expression , which is conveniently a linear function of . The resulting number is then multiplied by the vector .
Obviously, the tensor product of vectors belonging to vector spaces of dimensions higher than 1 are not invertible, in fact, the range of is one dimensional (why?)!
The following are some of the properties of the tensor product that can be deduced directly from the definition and the properties of the dot product operation, :

Another property is that if and are three orthonormal vectors, then:

It is important to note that the tensor product defined here is sometimes referred to as the dyadic product or the outer product of two vectors which is a particular type of the more general tensor product.

#### Matrix Representation of the Tensor Product

Let and consider the tensor product . Consider the orthonormal basis set . Then, the tensor product can be expressed in component form as follows:

Now, we have:

Which, can be represented in matrix form as follows:

#### Tensor Product Representation of Linear Maps

A linear map can be decomposed into the sum of multiple tensor products. For example, one can think of a linear map between three dimensional vector spaces, as the sum of three tensor products:

For this map to be invertible, each of the sets and has to be linearly independent (why?).

There is a direct relationship between the tensor product representation and the matrix representation as follows: let and let be an orthonormal basis set for both vector spaces, then, :

Therefore, any linear map can be represented as the sum of nine tensor product components

### The Set of Linear Maps

In these pages, the notation is used to denote the set of linear maps between and . i.e.,:

In addition, the short notation is used to denote the set of linear maps between and . i.e.,

We also freely use the set of matrices denoted to denote .

### The Algebraic Structure of the Set of Linear Maps

In addition to being a vector space, the elements of the sets of linear maps has an algebraic structure arising naturally from the composition operation. Let and , then, the composition map is also a linear map since:

Let . If and are the matrices associated with the linear maps and respectively, then, the components of the matrix associated with the linear map can be obtained from the equality: :

Therefore, the components can be calculated from the components of and as follows:

In otherwords, the component in the row and the column of is equal to the multiplication of the components in the row of by the components in the column of . Notice that the operation is well defined while the operation isn’t because of the difference in the dimensions of the above spaces.

However, if , and their respective associated matrices are then both composition maps are well defined. The first one is the composition map with its associated matrix while the second is the composition map and its associated matrix . In general, these two maps are not identical.

The identity map and its associated identity matrix is the identity element in the algebraic structure of

### Bijective (Invertible) Linear Maps:

In this section, we are concerned with the linear maps represented by square matrices and whether these linear maps (linear functions) are invertible or not. Recall from the Mathematical Preliminaries section that a function is invertible if such that . is denoted by . Let’s now consider the linear map (represented by a matrix ) , what are the conditions that guarantee the existence of such that where is the identity matrix? We will answer this question using a few statements:

Statement 1: Let be a linear map. Then .
This statement is simple to prove. First note that since is a linear map, then .
First, assume is injective. Since and since is injective therefore, 0 is the unique image of 0. Therefore, . For the opposite statement, assume that . We will argue by contradiction, i.e., assuming that is not injective. Therefore, with but . Since is linear we have . Therefore, which is a contradiction. Therefore is injective.

Statement 2: Let be a linear map. Then .
First assume that is invertible, therefore, is injective. Statement 1 asserts then that .
Assume now that . Therefore, from statement 1, is injective. We need to show that is surjective. Note that using Statement 1, and since an invertible map is also injective, then we just need to show that . This can be proven by picking a basis set for and showing that the set is linearly independent which right away implies that is surjective. Since is injective and is linearly independent we have:

Therefore:

Therefore, are linearly independent set of vectors in and in the range of . Therefore. such that

Therefore, is the preimage of . Therefore, is surjective.

Statement 3: Let be a linear map. Then the vectors forming the square matrix of are linearly independent.
First assume that are linearly independent vectors that form the row vectors of the linear map . We will argue by contradiction. Assume that and . Then, . However, since are linearly independent, they form a basis set and can be expressed in terms of all of them. Therefore . But is orthogonal to all of them, then . Therefore, and the map is bijective using statement 2.
For the opposite direction, assume that the map is bijective yet are linearly dependent. Since they are linearly dependent, therefore there is at least one vector that can be represented as a linear combination of the other vectors. Without loss of generality, assume that . Therefore

This shows that the range of has at most dimensions, therefore is not surjective, which is a contradiction.

Statement 3 asserts that a square matrix is invertible if and only if the rows are linearly independent. In the following section, we will present the determinant of a matrix as a measure of whether the rows are linearly independent or not.

#### Determinant:

The determinant of a matrix representation of a linear map is a real valued function of the components of a square matrix. The determinant is used to indicate whether the rows of the matrix are linearly dependent or not. If they are, then the determinant is equal to zero, otherwise, the determinant is not equal to zero. In the following, we will show the definition of the determinant function for , and for a general . We will also verify that the determinant of is equal to zero if and only if the row vectors of the matrix are linearly dependent for the cases and .

##### Determinant of :

Let such that

The determinant of is defined as:

Clearly, the vectors and are linearly dependent if and only if . The determinant of the matrix has a geometric meaning (See Figure 1). Consider the two unit vectors and . Let and . The area of the parallelogram formed by and is equal to the determinant of the matrix .
The following is true and :

where is the identity matrix.

Figure 1. Area transformation under

##### Determinant of :

Let such that

If , , and , then the determinant of is defined as:

I.e., the tripe product of , , and . From the results of the triple product, the vectors , , and are linearly dependent if and only if . The determinant of the matrix has a geometric meaning (See Figure 2). Consider the three unit vectors , , and . Let , , and . The determinant of is also equal to the triple product of , , and and gives the volume of the parallelepiped formed by , , and .

Additionally, and are linearly independent, it is straightforward to show the following:

In other words, the determinant gives the ratio between and where is the volume of the transformed parallelepiped between , , and and is the volume of the parallelepiped between , , and .
The alternator defined in Mathematical Preliminaries can be used to write the followign useful equality:

The following is true and :

where is the identity matrix.

Figure 2. Volume transformation under

##### Area Transformation in :

The following is a very important formula (often referred to as “Nanson’s Formula”) that relates the cross product of vectors in to the cross product of their images under a linear transformation. This formula is used to relate area vectors before mapping to area vectors after mapping.

##### Assertion:

Let . Let be an invertible matrix. Show the following relationship:

##### Proof:

Let be an arbitrary vector in . From the relationships above we have:

Therefore:

Since is arbitrary, it is straightforward to show that the vectors and are equal. And, since is invertible, so, is . Therefore:

Nanson’s formula is sometimes written as follows:

where

##### Determinant of :

The determinant of is defined using the recursive relationship:

where and is formed by eliminating the 1st row and column of the matrix . It can be shown that the rows of are linearly dependent.

### Eigenvalues and Eigenvectors

Let . is called an eigenvalue of the tensor if such that . In this case, is called an eigenvector of associated with the eigenvalue .

Notice that any nonzero multiplier of an eigenvector is again an eigenvector: If then is an eigenvector of . In addition, a nonzero linear combination of eigenvectors associated with the same eigenvalue is also an eigenvector:
If and then with and : is an eigenvector of .

#### Similar Matrices

Let . Let be an invertible tensor. The matrix representations of the tensors and are termed “similar matrices”.

Similar matrices have the same eigenvalues while their eigenvectors differ by a linear transformation as follows: If is an eigenvalue of with the associated eigenvector then:

Therefore, is an eigenvalue of and is the associated eigenvector. Similarly, if is an eigenvalue of with the associated eigenvector then:

Therefore, is an eigenvalue of and is the associated eigenvector. Therefore, similar matrices share the same eigenvalues.

#### The Eigenvalue and Eigenvector Problem

Given a tensor , we seek a nonzero vector and a real number such that: . This is equivalent to . In other words, the eigenvalue is a real number that makes the tensor not invertible while the eigenvector is a non-zero vector . Considering the matrix representation of the tensor , the eigenvalue is the solution to the following equation:

The above equation is called the characteristic equation of the matrix .
From the properties of the determinant function, the characteristic equation is an degree polynomial of the unknown where is the dimension of the underlying space.
In particular, , where are called the polynomial coefficients. Thus, the solution to the characteristic equation abides by the following facts from polynomial functions:
Polynomial roots: A polynomial has a root if divides , i.e., such that .
– The fundamental theorem of Algebra states that a polynomial of degree has complex roots that are not necessarily distinct.
– The Complex Conjugate Root Theorem states that If is a complex root of a polynomial with real coefficients, then the conjugate is also a complex root.

Therefore, the eigenvalues can either be real or complex numbers. If one eigenvalue is a real number, then there exists a vector with real valued components that is an eigenvector of the tensor. Otherwise, the only eigenvectors are complex eigenvectors which are elements of finite dimensional linear spaces over the field of complex numbers.

#### Graphical Representation of the Eigenvalues and Eigenvectors

The eigenvectors of a tensor are those vectors that do not change their direction upon transformation with the tensor but their length is rather magnified or reduced by a factor . Notice that an eigenvalue can be negative (i.e., the transformed vector can have an opposite direction). Additionally, an eigenvalue can have the value of 0. In that case, the eigenvector is an element of the kernel of the tensor.

The following example illustrates this concept. Choose four entries for the matrix and press evaluate.
The tool then draws 8 coloured vectors across the circle and their respective images across the ellipse. Use visual inspection to identify which vectors keep their original direction.
The tool also finds at most two eigenvectors (if they exist) and draws them in black along with their opposite directions. Use the tool to investigate the eigenvalues and eigenvectors of the following matrices:

After inspection, you should have noticed that every vector is an eigenvector for the identity matrix since , i.e., possesses one eigenvalue which is but all the vectors in are possible eigenvectors.
You should also have noticed that some matrices don’t have any real eigenvalues, i.e., none of the vectors keep their direction after transformation. This is the case for the matrix:

Additionally, the matrix:

has only one eigenvalue while any vector which is a multiplier of

keeps its direction after transformation through the matrix . You should also notice that some matrices will have negative eigenvalues. In that case, the corresponding eigenvector will be transformed into the direction opposite to its original direction. See for example, the matrix: