{"id":3325,"date":"2015-04-11T23:11:07","date_gmt":"2015-04-12T04:11:07","guid":{"rendered":"http:\/\/www.ssc.wisc.edu\/~jfrees\/?page_id=3325"},"modified":"2015-08-17T13:48:35","modified_gmt":"2015-08-17T18:48:35","slug":"partitioning-the-variability","status":"publish","type":"page","link":"https:\/\/users.ssc.wisc.edu\/~ewfrees\/regression\/basic-linear-regression\/2-3-is-the-model-useful-some-basic-summary-measures\/partitioning-the-variability\/","title":{"rendered":"Partitioning the Variability"},"content":{"rendered":"

The squared deviations, \\(\\left( y_i-\\overline{y}\\right) ^2\\), provide a basis for measuring the spread of the data. If we wish to estimate the \\(i\\)th dependent variable without<\/em> knowledge of x<\/em>, then \\(\\overline{y}\\) is an appropriate estimate and \\(y_i- \\overline{y}\\) represents the deviation of the estimate. We use \\(Total~SS=\\sum_{i=1}^{n}\\left( y_i-\\overline{y}\\right) ^2\\), the total sum of squares, to represent the variation in all of the responses. <\/p>\n

Suppose now that we also have knowledge of x<\/em>, an explanatory variable. Using the fitted regression line, for each observation we can compute the corresponding\\ fitted value<\/em>, \\(\\widehat{y}_i = b_0 + b_1x_i\\). The fitted value is our estimate with<\/em> knowledge of the explanatory variable. As before, the difference between the response and the fitted value, \\(y_i- \\widehat{y}_i\\), represents the deviation of this estimate. We now have two “estimates” of \\(y_i\\), these are \\(\\widehat{y}_i\\) and \\(\\overline{y}\\). Presumably, if the regression line is useful, then \\( \\widehat{y}_i\\) is a more accurate measure than \\(\\overline{y}\\). To judge this usefulness, we algebraically decompose the total deviation as:
\n\\begin{equation}\\label{E2:deviationdecomp} \\begin{array}{ccccc} \\underbrace{y_i-\\overline{y}} & = & \\underbrace{y_i-\\widehat{y}_i} & + & \\underbrace{\\widehat{y}_i-\\overline{y}} \\\\ \\text{total} & {\\small =} & \\text{unexplained} & {\\small +} & \\text{explained} \\\\ \\text{deviation} & & \\text{deviation} & & \\text{deviation} \\end{array} \\end{equation} Interpret this equation as “the deviation without knowledge of x<\/em> equals the deviation with knowledge of x<\/em> plus the deviation explained by x<\/em>.” Figure 2.4 is a geometric display of this decomposition. In the figure, an observation above the line was chosen, yielding a positive deviation from the fitted regression line, to make the graph easier to read. A good exercise is to draw a rough sketch corresponding to Figure 2.4 with an observation below the fitted regression line. <\/p>\n

\"F2ANOVADecomp\"<\/a>
Figure 2.4 Geometric display of the deviation decomposition.<\/figcaption><\/figure>\n

R Code for Figure 2.4<\/strong><\/i><\/a> <\/h2>\n
\n
\r\nR-Code<\/strong>\r\npar(mar=c(2.2,2.1,.2,.2),cex=1.2)\r\nx <- seq(-4, 4, len=101)\r\ny <- x\r\nplot(x, y, type = \"l\", xlim=c(-3, 4), xaxt=\"n\", yaxt=\"n\", xlab=\"\", ylab=\"\")\r\naxis(1, at = c(-1, 1),lab = expression(bar(x), x))\r\naxis(2, at = c(-1, 1, 3),lab = expression(bar(y), hat(y), y), las=1)\r\nabline(-1, 0, lty = 2)\r\nsegments(-4, 1, 1, 1, lty=2)\r\nsegments(-4, 3, 1, 3, lty = 2)\r\nsegments(1, -4, 1, 3, lty = 2)\r\nsegments(-1, -4, -1, -1, lty = 2)\r\n\r\npoints(1, 3, cex=1.5, pch=19)\r\n\r\narrows(1.0, 1, 1.0, 3, code = 3, lty = 1, angle=15, length=0.12, lwd=2)\r\ntext(1.3, 2.2,   expression( y-hat(y)),cex=0.8) \r\n#text(.3,2.2,\"'unexplained' deviation\", cex=.8) \r\narrows(1.0, -1, 1.0, 1, code = 3, lty = 1, angle=15, length=0.12, lwd=2)\r\ntext(1.7, 0, expression(hat(y)-bar(y) == b[1](x-bar(x)) ), cex=0.8 )\r\n#text(2.1, -0.5, \" 'explained' deviation\", cex=0.8)\r\n#arrows(.9, -1, .9, 3, code = 3, lty = 1, angle=15, length=0.12, lwd=2)\r\n#text(.6, 1.2, cex=0.8  , expression( y-bar(y)))#\"total deviation\")\r\narrows(-1, -1.0, 1, -1.0, code = 3, lty = 1, angle=15, length=0.12, lwd = 2)\r\ntext(0, -1.3, expression( x-bar(x)), cex=0.8  )\r\ntext(3.5, 2.7, expression( hat(y)== b[0]+ b[1]*x), cex=0.8  )\r\n<\/pre>\n<\/div>\n
 Now, from the algebraic decomposition in equation (2.1), square each side of the equation and sum over all observations. After a little algebraic manipulation, this yields \\begin{equation}\\label{E2:ANOVADecomposition} \\sum_{i=1}^{n}\\left( y_i-\\overline{y}\\right) ^2=\\sum_{i=1}^{n}\\left( y_i-\\widehat{y}_i\\right) ^2+\\sum_{i=1}^{n}\\left( \\widehat{y}_i- \\overline{y}\\right) ^2. \\end{equation} We rewrite this as \\(Total~SS=Error~SS+Regression~SS\\) where \\(SS\\) stands for sum of squares. We interpret: <\/p>\n