LU decomposition

Thm 1:
N-rank regular matrix A has a unique factorization LU, where L is a N-rank regular matrix and U is a N-rank upper-triangular matrix.

When we think about the signs of eigenvalues in neural network literature, we only deal with Hessian, which is symmetric, so the signs of all eigenvalues uniquely determine the positive definiteness of the Hessian matrix. But this is not generally true for non-symmetric positive definite matrix.

One example of equivalent condition of a matrix being positive definite is the existence of a unique lower triangular matrix L with real and strictly positive diagonal entries s.t. M = LL holds. (M = LL is called Cholesky decomposition.)

Cholesky decomposition

Thm 2:
For a symmetric positive definite matrix M, there exists a decomposition s.t. M = LL^T

Pf. (sketch)
$$M = PDP^T \\ = PSS^TP^T \\ = B^TB \quad (B = (PS)^T) \\ = (QR)^T QR \quad (B = QR) \\ = R^T Q^{-1} Q R = R^T R = LL^T$$