4.10 Regression of x on y

The regression line of $y$ on $x$ is built to predict $y$ from a known value of $x$. Many situations need the reverse: estimating a father's height from his son's, or the body weight of an animal from the weight of one of its organs. When both variables vary naturally and either may be the one that is known, there are two regression lines, one for each direction of prediction.

The two lines answer different questions, so they generally have different equations. Understanding why they differ, and which one to use, prevents one of the most common errors in statistics.

Why a second regression line is needed

What each line minimises

The regression line of $y$ on $x$ is the line that makes the sum of the squares of the vertical distances from the points to the line as small as possible. Vertical distances are errors in $y$, so this line gives the best predictions of $y$ when $x$ is known.

To predict $x$ when $y$ is known, the errors that matter are errors in $x$, which are horizontal distances. The line that makes the sum of the squares of the horizontal distances as small as possible is a different line, the regression line of $x$ on $y$. Unless every point lies exactly on a straight line, the two lines are not the same.

The regression line of x on y is written with $x$ as the subject, $x = cy + d$, to show that it gives $x$ in terms of $y$. Here $c$ is the change in $x$ for each increase of one unit in $y$, on average, and $d$ is the predicted value of $x$ when $y = 0$.

Finding the line with a GDC

A GDC calculates the regression line of $x$ on $y$ with the same linear regression function as for $y$ on $x$, with the roles of the two lists swapped.

1. Enter the $x$-values in list 1 and the $y$-values in list 2, as before.

2. Run linear regression with list 2 as the "x-list" and list 1 as the "y-list". The calculator then treats $y$ as the independent variable.

3. The calculator displays the result as "$y = ax + b$", because it always uses those letters. Write it with the original variables: the gradient it gives is $c$ and the intercept is $d$, so the line is $x = cy + d$.

The value of $r$ is the same whichever way round the lists are entered.

The table gives the body weight, $x$ kg, and heart weight, $y$ g, of ten adult cats.

Data: ten of the 144 cats measured by R. A. Fisher, Biometrics, 1947.

Choosing the right line for a prediction

The rule depends only on which variable is known and which is to be estimated.

Rearranging one line to make the other variable the subject does not give the other line. The rearranged line minimises errors in the wrong direction, so its predictions are biased, and the weaker the correlation, the larger the error.

The heights of fathers and their adult sons, collected in the 1880s, show the effect clearly. The correlation is weak ($r = 0.392$), so the two regression lines cross at a wide angle at the mean point.

The correct answer to part 1 shows an effect that Galton called "regression towards the mean", and which gave regression its name. Very tall sons tend to have fathers who are tall, but closer to average than the sons themselves; very tall fathers, in the same way, tend to have sons closer to average. Because the correlation is far from perfect, each prediction is pulled towards the mean of the variable being predicted.

How the two lines are related

Both lines pass through the mean point

Each regression line passes through the mean point $(\bar{x}, \bar{y})$. Since this point lies on both lines, it is the point where they intersect. When the equations of both lines are known, $\bar{x}$ and $\bar{y}$ can be found by solving them as a pair of simultaneous equations, without a GDC.

The same fact gives questions in which a missing data value is found from the line of $x$ on $y$, exactly as with the line of $y$ on $x$.

When the two lines coincide

How far apart the two lines are depends on the strength of the correlation. If every point lies on one straight line ($r = 1$ or $r = -1$), the vertical and horizontal distances are all $0$ for that line, so it is the best line in both directions and the two regression lines are the same. As $|r|$ decreases, the lines open out like a pair of scissors about the mean point. When $r = 0$, the line of $y$ on $x$ is horizontal ($y = \bar{y}$) and the line of $x$ on $y$ is vertical ($x = \bar{x}$): knowing one variable then gives no information about the other, so the best estimate is simply its mean.

The reason can be seen from the gradients. Writing $s_x$ and $s_y$ for the standard deviations of the two variables, the gradients of the two lines are

These formulas are not needed for calculations, which are done on a GDC, but they explain three facts. Both gradients have the same sign as $r$. Their product is $ac = r^2$. And the rearranged line of $y$ on $x$ has gradient $\frac{1}{a}$ in the "$x =$" form, which equals $c$ only when $ac = 1$, that is, when $r^2 = 1$.

Exam-style questions on the two regression lines

Questions on this subtopic usually build on a regression question from 4.4: after finding the line of $y$ on $x$ and making a prediction, a later part gives a value of $y$ and asks for an estimate of $x$, which needs the second regression line.

The table gives the heights, in centimetres, of ten fathers, $x$, and their adult sons, $y$.

Data: ten of the families measured by F. Galton, 1886, converted to centimetres and rounded.

Self-check

• To do:

  I can explain why the regression line of $x$ on $y$ minimises horizontal distances, and why it differs from the line of $y$ on $x$.

• To do:

  I can find the regression line of $x$ on $y$ with a GDC and write it correctly in the form $x = cy + d$.

• To do:

  I can choose the correct regression line for a prediction and say why rearranging the other line is wrong.

• To do:

  I can find the mean point by solving the equations of the two regression lines simultaneously.

• To do:

  I can use the mean point and the line of $x$ on $y$ to find a missing data value.

• To do:

  I can state that the two lines coincide only when $r = \pm 1$, and describe how the angle between them changes as $|r|$ decreases.

1. (a)

   One of the students reads for 8 hours each week. Use an appropriate regression line to estimate this student's score in the test.

   [2]2 marks

   Markscheme (a)

   use of the regression line of $y$ on $x$ (M1)

   $y = 2.5 \times 8 + 45 = 65$ A1

2. (b)

   Another student scored 70 in the test. Use an appropriate regression line to estimate the number of hours this student reads each week.

   [2]2 marks

   Markscheme (b)

   use of the regression line of $x$ on $y$ (M1)

   $x = 0.3 \times 70 - 12 = 9$

3. (c)

   State whether Pearson's product-moment correlation coefficient for these data is positive or negative. Give a reason for your answer.

   [1]1 mark

   Markscheme (c)

   positive, since both regression lines have a positive gradient R1

Regression with the lists swapped (list 2 as the independent list) gives gradient $0.153360\ldots$ and intercept $1.20940\ldots$. Written with $x$ as the subject:

3. Heart weight is to be predicted from body weight, so use $y$ on $x$ with the unrounded values:

4. Body weight is to be predicted from heart weight, so use $x$ on $y$:

Both values ($3.0$ kg and $10$ g) lie inside the ranges of the data, so these are interpolations. The correlation is only moderate, so they are rough estimates.

(With the unrounded coefficients, $0.344809 \times 190 + 114.973 = 180.49$, so about $180$ cm.)

2. Rearranging gives $x = \dfrac{y - 97.4}{0.447}$, so

This is not appropriate. The line of $y$ on $x$ minimises errors in the son's height, not in the father's. Its estimate, 207 cm, is taller than every father in the data, while the correct estimate, about 180 cm, is only a little above the mean of 175.6 cm.

2. Now $x = 3$ is known and $y$ is to be estimated, so use the line of $y$ on $x$:

Then $y = 0.8 \times 10 + 8 = 16$.

Check in the second equation: $0.5 \times 16 + 2 = 10$. The mean point is $(10, 16)$.

To draw both lines on the same axes, write the second with $y$ as the subject: $x = 0.5y + 2$ gives $y = 2x - 4$.

The line passes through $(\bar{x}, \bar{y})$, so

The five $x$-values add up to $5 \times 6 = 30$:

which is the same line.

2. Here $a = 0.8$ and $c = 0.5$, so

Both gradients are positive, so $r$ is positive: $r = \sqrt{0.4} = 0.632$ (3 s.f.), a moderate positive correlation.

3. The father's height is known, so use $y$ on $x$:

4. Swap the lists (sons as the independent list): gradient $0.670240\ldots$, intercept $57.6312\ldots$, so

5. The son's height is known and the father's is to be estimated, so use $x$ on $y$:

(Rearranging the line of $y$ on $x$ would give $169.6$ cm, which earns no marks here.)

6. Both lines pass through the mean point. From the GDC (or by adding each row and dividing by 10):

Then $y = 2.2 \times 25 - 2 = 53$. The mean 5 km time is 25 minutes and the mean 10 km time is 53 minutes.

2. The 10 km time $y = 60$ is known, so use $x$ on $y$:

3. The 5 km time $x = 22$ is known, so use $y$ on $x$:

4. The line of $y$ on $x$ minimises errors in $y$ and is used to predict $y$ from $x$; the line of $x$ on $y$ minimises errors in $x$ and is used to predict $x$ from $y$.

Note: Award (M0)A0 for rearranging the line of $y$ on $x$ to obtain $x = 10$.

[2]2 marks