{"id":3439,"date":"2015-04-12T01:40:09","date_gmt":"2015-04-12T06:40:09","guid":{"rendered":"http:\/\/www.ssc.wisc.edu\/~jfrees\/?page_id=3439"},"modified":"2015-04-19T14:32:08","modified_gmt":"2015-04-19T19:32:08","slug":"random-matrices","status":"publish","type":"page","link":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/regression\/basic-linear-regression\/2-9-technical-supplement-elements-of-matrix-algebra\/basic-operations\/random-matrices\/","title":{"rendered":"Random Matrices"},"content":{"rendered":"

Expectations.<\/strong> Consider a matrix of random variables \\begin{equation*} \\mathbf{U=}\\left( \\begin{array}{cccc} u_{11} & u_{12} & \\cdots & u_{1c} \\\\ u_{21} & u_{22} & \\cdots & u_{2c} \\\\ \\vdots & \\vdots & \\ddots & \\vdots \\\\ u_{n1} & u_{n2} & \\cdots & u_{nc} \\end{array} \\right). \\end{equation*} When we write the expectation of a matrix, this is short-hand for the matrix of expectations. Specifically, suppose that the joint probability function of \\({u_{11}, u_{12}, …, u_{1c}, …, u_{n1}, …, u_{nc}}\\) is available to define the expectation operator. Then we define \\begin{equation*} \\mathrm{E} ~ \\mathbf{U} = \\left( \\begin{array}{cccc} \\mathrm{E }u_{11} & \\mathrm{E }u_{12} & \\cdots & \\mathrm{E }u_{1c} \\\\ \\mathrm{E }u_{21} & \\mathrm{E }u_{22} & \\cdots & \\mathrm{E }u_{2c} \\\\ \\vdots & \\vdots & \\ddots & \\vdots \\\\ \\mathrm{E }u_{n1} & \\mathrm{E }u_{n2} & \\cdots & \\mathrm{E }u_{nc} \\end{array} \\right). \\end{equation*} As an important special case, consider the joint probability function for the random variables \\(y_1, \\ldots, y_n\\) and the corresponding expectations operator. Then \\begin{equation*} \\mathrm{E}~ \\mathbf{y=} \\mathrm{E } \\left( \\begin{array}{cccc} y_1 \\\\ \\vdots \\\\ y_n \\end{array} \\right) = \\left( \\begin{array}{cccc} \\mathrm{E }y_1 \\\\ \\vdots \\\\ \\mathrm{E }y_n \\end{array} \\right). \\end{equation*} By the linearity of expectations, for a non-random matrix A<\/strong> and vector B<\/strong>, we have E (A y<\/strong> + B<\/strong>) = A<\/strong> E y + B<\/strong>. <\/p>\n

Variances.<\/strong> We can also work with second moments of random vectors. The variance of a vector of random variables is called the variance-covariance matrix<\/em>. It is defined by \\begin{equation}\\label{E2:MatrixVar} \\mathrm{Var} ~ \\mathbf{y} = \\mathrm{E} ( (\\mathbf{y} – \\mathrm{E} \\mathbf{y})(\\mathbf{y} – \\mathrm{E} \\mathbf{y})^{\\prime} ). \\end{equation} That is, we can express \\begin{equation*} \\mathrm{Var}~\\mathbf{y=} \\mathrm{E } \\left( \\left( \\begin{array}{c} y_1 -\\mathrm{E } y_1 \\\\ \\vdots \\\\ y_n -\\mathrm{E } y_n \\end{array}\\right) \\left(\\begin{array}{ccc} y_1 – \\mathrm{E } y_1 & \\cdots & y_n – \\mathrm{E } y_n \\end{array}\\right) \\right) \\end{equation*} \\begin{equation*} = \\left( \\begin{array}{cccc} \\mathrm{Var}~y_1 & \\mathrm{Cov}(y_1, y_2) & \\cdots &\\mathrm{Cov}(y_1, y_n) \\\\ \\mathrm{Cov}(y_2, y_1) & \\mathrm{Var}~y_2 & \\cdots & \\mathrm{Cov}(y_2, y_n) \\\\ \\vdots & \\vdots & \\ddots & \\vdots\\\\ \\mathrm{Cov}(y_n, y_1) & \\mathrm{Cov}(y_n, y_2) & \\cdots & \\mathrm{Var}~y_n \\\\ \\end{array}\\right), \\end{equation*} because \\(\\mathrm{E} ( (y_i – \\mathrm{E} y_i)(y_j – \\mathrm{E} y_j) ) = \\mathrm{Cov}(y_i, y_j)\\) for \\(i \\neq j\\) and \\(\\mathrm{Cov}(y_i, y_i) = \\mathrm{Var}~y_i\\). In the case that \\(y_1, \\ldots, y_n\\) are mutually uncorrelated, we have that \\(\\mathrm{Cov}(y_i, y_j)=0\\) for \\(i \\neq j\\) and thus \\begin{equation*} \\mathrm{Var}~\\mathbf{y=} \\left( \\begin{array}{cccc} \\mathrm{Var}~y_1 & 0 & \\cdots & 0 \\\\ 0 & \\mathrm{Var}~y_2 & \\cdots & 0 \\\\ \\vdots & \\vdots & \\ddots & \\vdots\\\\ 0 & 0 & \\cdots & \\mathrm{Var}~y_n \\\\ \\end{array}\\right). \\end{equation*} Further, if the variances are identical so that \\(\\mathrm{Var}~y_i=\\sigma ^2\\), then we can write \\(\\mathrm{Var} ~\\mathbf{y} = \\sigma ^2 \\mathbf{I}\\), where I<\/strong> is the \\(n \\times n\\) identity matrix. For example, if \\(y_1, \\ldots, y_n\\) are i.i.d., then \\(\\mathrm{Var} ~\\mathbf{y} = \\sigma ^2 \\mathbf{I}\\). <\/p>\n

From equation (2.10), it can be shown that \\begin{equation}\\label{E2:MatrixVarCalc} \\mathrm{Var}\\left( \\mathbf{Ay +B} \\right) = \\mathrm{Var}\\left( \\mathbf{Ay} \\right) = \\mathbf{A} \\left( \\mathrm{Var}~\\mathbf{y} \\right) \\mathbf{A}^{\\prime}. \\end{equation} For example, if \\(\\mathbf{A} = (a_1, a_2, \\ldots,a_n)= \\mathbf{a}^{\\prime}\\) and B = 0<\/strong>, then equation (2.11) reduces to \\begin{equation*} \\mathrm{Var}\\left( \\sum_{i=1}^n a_i y_i \\right) = \\mathrm{Var} \\left( \\mathbf{a^{\\prime} y} \\right) = \\mathbf{a^{\\prime}} \\left( \\mathrm{Var} ~\\mathbf{y} \\right) \\mathbf{a} = (a_1, a_2, \\ldots,a_n) \\left( \\mathrm{Var} ~\\mathbf{y} \\right) \\left(\\begin{array}{c} a_1 \\\\ \\vdots \\\\ a_n \\end{array}\\right) \\end{equation*} \\begin{equation*} = \\sum_{i=1}^n a_i^2 \\mathrm{Var} ~y_i ~+~2 \\sum_{i=2}^n \\sum_{j=1}^{i-1} a_i a_j \\mathrm{Cov}(y_i, y_j). \\end{equation*} <\/p>\n

Definition – Multivariate Normal Distribution.<\/strong> A vector of random variables \\(\\mathbf{y} = \\left(y_1, \\ldots, y_n \\right)^{\\prime}\\) is said to be multivariate normal<\/em> if all linear combinations of the form \\(\\sum_{i=1}^n a_i y_i\\) are normally distributed. In this case, we write \\(\\mathbf{y}\\sim N (\\mathbf{\\boldsymbol \\mu}, \\mathbf{\\Sigma} )\\), where \\(\\mathbf{\\boldsymbol \\mu} = \\mathrm{E}~ \\mathbf{y} \\) is the expected value of y<\/strong> and \\(\\mathbf{\\Sigma}= \\mathrm{Var}~\\mathbf{y}\\) is the variance-covariance matrix of y<\/strong>. From the definition, we have that \\(\\mathbf{y}\\sim N (\\mathbf{\\boldsymbol \\mu}, \\mathbf{\\Sigma} )\\) implies that \\(\\mathbf{a^{\\prime}y}\\sim N (\\mathbf{a^{\\prime} \\boldsymbol \\mu}, \\mathbf{a^{\\prime}\\Sigma a} )\\). Thus, if \\(y_i\\) are i.i.d., then \\(\\sum_{i=1}^n a_i y_i\\) is distributed normally with mean \\(\\mu \\sum_{i=1}^n a_i \\) and variance \\(\\sigma ^2 \\sum_{i=1}^n a_i ^2\\). <\/p>\n

◄ Previous page<\/a><\/div>
Next page ►<\/a><\/div><\/p>\n","protected":false},"excerpt":{"rendered":"

Expectations. Consider a matrix of random variables \\begin{equation*} \\mathbf{U=}\\left( \\begin{array}{cccc} u_{11} & u_{12} & \\cdots & u_{1c} \\\\ u_{21} & u_{22} & \\cdots & u_{2c} \\\\ \\vdots & \\vdots & \\ddots & \\vdots \\\\ u_{n1} …<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":3423,"menu_order":5,"comment_status":"closed","ping_status":"closed","template":"","meta":{"jetpack_post_was_ever_published":false},"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/P8cLPd-Tt","acf":[],"_links":{"self":[{"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/pages\/3439"}],"collection":[{"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/comments?post=3439"}],"version-history":[{"count":1,"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/pages\/3439\/revisions"}],"predecessor-version":[{"id":3440,"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/pages\/3439\/revisions\/3440"}],"up":[{"embeddable":true,"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/pages\/3423"}],"wp:attachment":[{"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/media?parent=3439"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}