Covariance

The covariance helps to measure how two variables varies with respect to one another. It measures the direction of the relationship between two variables.

$$COV(x,y)=\frac{1}{n-1}\sum _{i=1}^n(x_i-\mu (x))(y_i-\mu (y))$$

Covariance Matrix

$$C=\left[\begin{array}{cc}COV(x,x)& COV(x,y)\\ COV(y,x)& COV(y,y)\end{array}\right]=\left[\begin{array}{cc}VAR(x)& COV(x,y)\\ COV(y,x)& VAR(y)\end{array}\right]$$ $$\mathbf{A}=\left[\begin{array}{cc}{x}_1& y_1\\ x_2& y_2\\ ⋮& ⋮\\ x_n& y_n\end{array}\right]\mu =\left[\begin{array}{cc}{\mu }_x& {\mu }_y\\ {\mu }_x& {\mu }_y\\ ⋮& ⋮\\ {\mu }_x& {\mu }_y\end{array}\right]$$ $$\boxed{C=\frac{1}{n-1}(\mathbf{A}-\mu {)}^T(\mathbf{A}-\mu )}$$