Multi Variate Gaussian Distribution - Part 1

Introduction

In this article we will look at the multivariate gaussian distribution.

The mutivariate gaussian distribution is parameterized by mean vector $\boldsymbol\mu$ and covariance matrix $\boldsymbol\Sigma$. \begin{eqnarray*} & f_{\mathbf x}(x_1,\ldots,x_k) =\frac{1}{\sqrt{(2\pi)^k|\boldsymbol\Sigma|}} \exp\left(-\frac{1}{2}({\mathbf x}-{\boldsymbol\mu})^T{\boldsymbol\Sigma}^{-1}({\mathbf x}-{\boldsymbol\mu})\right), \\ & where\\ & \text{$\boldsymbol\mu$ is a Nx1 mean vector }\\ & \text{$\boldsymbol\Sigma$ is NxN matrix covariance matrix} \\ & \text{$|\boldsymbol\Sigma|$ is the determinant of $\boldsymbol\Sigma$} \\ \end{eqnarray*}
The mutivariate gaussian is also used as probability density function of the vector ${X}$
A naive way of implementation is to directly perform the computation as in the equation to get the result.
A Gaussian is defined
Since ill be using a lot of opencv conversion routines are defined for Opencv cv::Mat data structure to Eigen::MatrixXf data structure.
when loading the covarince matrix ,the determinant,inverse etc are also computed which would be later required for probability calculation
The probability computation is simply writing out the formula
The covariance matrix is postive semidefinate and symmteric
The Cholesky decomposition of a symmetric, positive definite matrix $\boldsymbol\Sigma$ decom- poses A into a product of a lower triangular matrix L and its transpose.
Thus the product \begin{eqnarray*} & \boldsymbol\Sigma=LL^T \\ & \boldsymbol\Sigma^{-1}=(LL^T)^{-1}=L^{-T}L^{-1} \\ & {\mathbf y}^T{\boldsymbol\Sigma}^{-1}{\mathbf y} \\ & {\mathbf y}^T{\boldsymbol\Sigma}^{-1}{\mathbf y} \\ & {\mathbf y}^T{L^{-T}L^{-1}}{\mathbf y} \\ & \left({L^{-1}\mathbf y}\right)^T\left({L^{-1}\mathbf y}\right) \end{eqnarray*}
thus we can only compute $\left({L^{-1}\mathbf y}\right)$ and take its tranpose
Thus we can perform the Cholskey factorization of the covariance matrix to find $\mathbf L$
Then we take the inverse of $\mathbf L $
Code
The code can be found at git repository https://github.com/pi19404/OpenVision in files ImgML/gaussian.hpp and ImgML.gaussian.cpp files.