Linear Algebra

Vector Algebra

• $a^{T}a=|a{|}^2$ is a scalar. $a{a}^T$ is an $n\times n$ matrix.
• $|a|=\sqrt{a_1^2+\cdots +a_n^2}$

Matrix can be

• Symmetric

  • Positive-definite
  • Positive-semi-definitie
  • Negative-definite
  • Negative-semi-definite

• Diagonal: entries outside the main diagonal are all zero

  • Diagonalizable: $A$ is diagonalizable iff exists invertible matrix $P$ where $PA{P}^{-1}$ is a diagonal matrix
    • Inverse of a diagonalizable matrix is also diagonalizable
    • Orthogonally diagonalizable: $A$ is orthogonally diagonizable iff exists orthogonal matrix $P$ where $PA{P}^T=PA{P}^{-1}=I$

• Invertible vs Singular

  • Invertible: There exists an inverse = determinant is non-zero $|A|\ne 0$
    • The inverse of a symmetric matrix is also symmetric
  • Singular: There does not exist an inverse
  • $A{A}^{-1}=I$

• Orthogonal:$A$ is orthogonal iff $A^{-1}=A^T$

• Rank:= dimension of the vector space generated by its columns

• Identity Matrix $I$

• Determinant: scalar value that determines if the matrix has a determinant

Matrix Algebra Identities

• Non-commutative
  • Commutative only with scalars
• Distributive (w.r.t. matrix addition)
• Associative
  • Computational complexity depends on which you multipliy first: Matrix Chain Multiplication
• Transpose-distribution $(AbC{)}^T=C^{T}b{A}^T$
• Inverse-distribution $(AB{)}^{-1}=B^{-1}{A}^{-1}$
• Transpose-Inverse $(A^T{)}^{-1}=(A^{-1}{)}^T$ If $\Sigma =\text{diag}\{{\sigma }_1,\dots ,{\sigma }_n\}$ is a diagonal matrix:
• $tr\{\Sigma \}={\sigma }_1+{\sigma }_2+\cdots +{\sigma }_n$
• $|\Sigma |={\sigma }_1\times {\sigma }_2\times \cdots \times {\sigma }_n$