{"id":3436,"date":"2015-04-12T01:38:46","date_gmt":"2015-04-12T06:38:46","guid":{"rendered":"http:\/\/www.ssc.wisc.edu\/~jfrees\/?page_id=3436"},"modified":"2015-08-18T13:53:30","modified_gmt":"2015-08-18T18:53:30","slug":"matrix-inverses","status":"publish","type":"page","link":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/regression\/basic-linear-regression\/2-9-technical-supplement-elements-of-matrix-algebra\/basic-operations\/matrix-inverses\/","title":{"rendered":"Matrix Inverses"},"content":{"rendered":"

In matrix algebra, there is no concept of “division.” Instead, we extend the concept of “reciprocals” of real numbers. To begin, suppose that \\(\\mathbf{A}\\) is a square matrix of dimension \\(k \\times k\\) and let \\(\\mathbf{I}\\) be the \\(k\\times k\\) identity matrix. If there exists a \\(k \\times k\\) matrix \\(\\mathbf{B}\\) such that \\(\\mathbf{AB}=\\mathbf{I=BA}\\), then \\(\\mathbf{B}\\) is called the inverse <\/em>of \\(\\mathbf{A}\\) and is written
\n\\begin{equation*} \\mathbf{B}=\\mathbf{A}^{-1}. \\end{equation*} Now, not all square matrices have inverses. Further, even when an inverse exists, it may not be easy to compute by hand. One exception to this rule are diagonal matrices. Suppose that \\(\\mathbf{A}\\) is diagonal matrix of the form
\n\\begin{equation*} \\mathbf{A=}\\left( \\begin{array}{ccc} a_{11} & \\cdots & 0 \\\\ \\vdots & \\ddots & \\vdots \\\\ 0 & \\cdots & a_{kk} \\end{array} \\right). \\text{ Then }\\mathbf{A}^{-1}\\mathbf{=}\\left( \\begin{array}{ccc} \\frac{1}{a_{11}} & \\cdots & 0 \\\\ \\vdots & \\ddots & \\vdots \\\\ 0 & \\cdots & \\frac{1}{a_{kk}} \\end{array} \\right). \\end{equation*} For example,
\n\\begin{equation*} \\begin{array}{cccc} \\left( \\begin{array}{cc} 2 & 0 \\\\ 0 & -19 \\end{array} \\right) & \\left( \\begin{array}{cc} \\frac{1}{2} & 0 \\\\ 0 & -\\frac{1}{19} \\end{array} \\right) & = & \\left( \\begin{array}{cc} 1 & 0 \\\\ 0 & 1 \\end{array} \\right) \\\\ \\mathbf{A} & \\mathbf{A}^{-1} & = & \\mathbf{I} \\end{array} . \\end{equation*} In the case of a matrix of dimension \\(2\\times 2\\), the inversion procedure can be accomplished by hand easily even when the matrix is not diagonal. In the \\(2\\times 2\\) case, we suppose that if
\n\\begin{equation*} \\mathbf{A=}\\left( \\begin{array}{cc} a & b \\\\ c & d \\end{array} \\right), \\text{ then }\\mathbf{A}^{-1}\\mathbf{=}\\frac{1}{ad-bc}\\left( \\begin{array}{cc} d & -b \\\\ -c & a \\end{array} \\right) \\text{.} \\end{equation*} Thus, for example, if
\n\\begin{equation*} \\mathbf{A=}\\left( \\begin{array}{cc} 2 & 2 \\\\ 3 & 4 \\end{array} \\right) \\text{ then }\\mathbf{A}^{-1}\\mathbf{=}\\frac{1}{2(4)-2(3)} \\left( \\begin{array}{cc} 4 & -2 \\\\ -3 & 2 \\end{array} \\right) =\\left( \\begin{array}{cc} 2 & -1 \\\\ -3\/2 & 1 \\end{array} \\right) \\text{.} \\end{equation*} As a check, we have
\n\\begin{align*} \\mathbf{A\\mathbf{A}^{-1}}&=\\left( \\begin{array}{cc} 2 & 2 \\\\ 3 & 4 \\end{array} \\right) \\left( \\begin{array}{cc} 2 & -1 \\\\ -3\/2 & 1 \\end{array} \\right) =\\left( \\begin{array}{cc} 2(2)-2(3\/2) & 2(-1)+2(1) \\\\ 3(2)-4(3\/2) & 3(-1)+4(1) \\end{array} \\right) \\\\ &=\\left( \\begin{array}{cc} 1 & 0 \\\\ 0 & 1 \\end{array} \\right) =\\mathbf{I}\\text{.} \\end{align*} <\/p>\n

Basic Linear Regression Example of Matrix Inverses.<\/strong> With
\n\\begin{equation*} \\mathbf{X}^{\\prime }\\mathbf{X=}\\left( \\begin{array}{cc} n & \\sum\\limits_{i=1}^{n}x_i \\\\ \\sum\\limits_{i=1}^{n}x_i & \\sum\\limits_{i=1}^{n}x_i^2 \\end{array} \\right), \\end{equation*} we have
\n\\begin{equation*} \\left( \\mathbf{X}^{\\prime }\\mathbf{X}\\right) ^{-1}\\mathbf{=}\\frac{1}{n\\sum_{i=1}^{n}x_i^2-\\left( \\sum_{i=1}^{n}x_i\\right) ^2}\\left( \\begin{array}{cc} \\sum\\limits_{i=1}^{n}x_i^2 & -\\sum\\limits_{i=1}^{n}x_i \\\\ -\\sum\\limits_{i=1}^{n}x_i & n \\end{array} \\right). \\end{equation*} To simplify this expression, recall that \\(\\overline{x}=n^{-1} \\sum_{i=1}^{n}x_i\\). Thus,
\n\\begin{equation}\\label{E2:XPXInv} \\left( \\mathbf{X}^{\\prime }\\mathbf{X}\\right) ^{-1}\\mathbf{=}\\frac{1}{ \\sum_{i=1}^{n}x_i^2-n\\overline{x}^2}\\left( \\begin{array}{cc} n^{-1}\\sum\\limits_{i=1}^{n}x_i^2 & -\\overline{x} \\\\ -\\overline{x} & 1 \\end{array} \\right) . \\end{equation} <\/p>\n

Section 3.1 will discuss the relation \\(\\mathbf{b}=\\left( \\mathbf{X}^{\\prime}\\mathbf{X}\\right)^{-1} \\mathbf{X}^{\\prime}\\mathbf{y}\\). To illustrate the calculation, we have
\n\\begin{eqnarray*} \\mathbf{b} &=&\\left( \\mathbf{X}^{\\prime }\\mathbf{X}\\right) ^{-1}\\mathbf{X} ^{\\prime }\\mathbf{y=}\\frac{1}{\\sum_{i=1}^{n}x_i^2-n\\overline{x}^2} \\left( \\begin{array}{cc} n^{-1}\\sum\\limits_{i=1}^{n}x_i^2 & -\\overline{x} \\\\ -\\overline{x} & 1 \\end{array} \\right) \\left( \\begin{array}{c} \\sum\\limits_{i=1}^{n}y_i \\\\ \\sum\\limits_{i=1}^{n}x_iy_i \\end{array} \\right) \\\\ &=&\\frac{1}{\\sum_{i=1}^{n}x_i^2-n\\overline{x}^2}\\left( \\begin{array}{c} \\sum\\limits_{i=1}^{n}\\left( \\overline{y}x_i^2-\\overline{x} x_iy_i\\right) \\\\ \\sum\\limits_{i=1}^{n}x_iy_i-n\\overline{x}\\overline{y} \\end{array} \\right) =\\left( \\begin{array}{c} b_0 \\\\ b_1 \\end{array} \\right) . \\end{eqnarray*} From this expression, we may see
\n\\begin{equation*} b_1=\\frac{\\sum\\limits_{i=1}^{n}x_iy_i-n\\overline{x}\\overline{y}}{ \\sum\\limits_{i=1}^{n}x_i^2-n\\overline{x}^2} \\end{equation*} and
\n\\begin{equation*} b_0=\\frac{\\overline{y}\\sum\\limits_{i=1}^{n}x_i^2-\\overline{x} \\sum\\limits_{i=1}^{n}x_iy_i}{\\sum\\limits_{i=1}^{n}x_i^2-n\\overline{x} ^2}=\\frac{\\overline{y}\\left( \\sum\\limits_{i=1}^{n}x_i^2-n\\overline{x} ^2\\right) -\\overline{x}\\left( \\sum\\limits_{i=1}^{n}x_i y_i – n\\overline{x} \\overline{y}\\right) }{\\sum\\limits_{i=1}^{n}x_i^2-n\\overline{x}^2}= \\overline{y}-b_1\\overline{x}. \\end{equation*} These are the usual expressions for the slope \\(b_1\\) (Exercise 2A.8) and intercept \\(b_0\\). <\/p>\n

◄ Previous page<\/a><\/div>
Next page ►<\/a><\/div><\/p>\n","protected":false},"excerpt":{"rendered":"

In matrix algebra, there is no concept of “division.” Instead, we extend the concept of “reciprocals” of real numbers. To begin, suppose that \\(\\mathbf{A}\\) is a square matrix of dimension \\(k \\times k\\) and let …<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":3423,"menu_order":4,"comment_status":"closed","ping_status":"closed","template":"","meta":{"jetpack_post_was_ever_published":false},"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/P8cLPd-Tq","acf":[],"_links":{"self":[{"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/pages\/3436"}],"collection":[{"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/comments?post=3436"}],"version-history":[{"count":4,"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/pages\/3436\/revisions"}],"predecessor-version":[{"id":4915,"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/pages\/3436\/revisions\/4915"}],"up":[{"embeddable":true,"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/pages\/3423"}],"wp:attachment":[{"href":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/wp-json\/wp\/v2\/media?parent=3436"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}