Linear algebra cheat-sheet
     Laurent Lessard
     University of Wisconsin–Madison
                                      Last updated: October 12, 2016
    Matrix basics
                                                                                  m×n
           A matrix is an array of numbers. A ∈ R                                          means that:
                               a           . . .     a 
                                11                     1n
                                     .       .          .
                      A= .                   ..        .    (mrows and n columns)
                                     .                  .
                                  am1 ...            amn
           Two matrices can be multiplied if inner dimensions agree:
                                                                                           n
                               C = A B where cij =Xaikbkj
                            (m×p)          (m×n)(n×p)                                    k=1
                                                                                T
           Transpose: The transpose operator A swaps rows and
                                            m×n                 T          n×m                 T
           columns. If A ∈ R                        then A ∈ R                     and (A )ij = Aji.
               •        T T
                   (A ) =A.
               •             T          T T
                   (AB) =B A .
                                                                                                                                    2
  Matrix basics (cont’d)
      Vector products. If x,y ∈ Rn are column vectors,
         •                          T
           The inner product is x y ∈ R (a.k.a. dot product)
         •                            T     n×n
           The outer product is xy      ∈R .
      These are just ordinary matrix multiplications!
                           n×n                                n×n
      Inverse. Let A ∈ R       (square). If there exists B ∈ R    with
      AB =I or BA = I (if one holds, then the other holds with the
                                                                  −1
      same B) then B is called the inverse of A, denoted B = A       .
      Some properties of the matrix inverse:
         •   −1
           A is unique if it exists.
         •    −1 −1
           (A )     =A.
         •    −1 T      T −1
           (A ) =(A ) .
         •       −1     −1 −1
           (AB)     =B A .
                                                                            3
  Vector norms
                      n
      A norm k·k : R → R is a function satisfying the properties:
         • kxk = 0 if and only if x = 0 (definiteness)
         • kcxk = |c|kxk for all c ∈ R (homogeneity)
         • kx +yk ≤ kxk+kyk (triangle inequality)
      Common examples of norms:                                 1 x2
         • kxk =|x |+···+|x | (the 1-norm)
               1     1            n
         • kxk =px2+···+x2 (the 2-norm)                                 x1
               2       1          n                       -1            1
         • kxk    =max        |x |    (max-norm)
               ∞         1≤i≤n  i
                                                               -1
      Properties of the 2-norm (Euclidean norm)              norm ball:
         • If you see kxk, think kxk (it’s the default)      {x | kxk = 1}
                                     2
         • xTx = kxk2
         • xTy ≤ kxkkyk (Cauchy-Schwarz inequality)
                                                                            4