4.2 Presenting data

Frequency distributions

Frequency tables for discrete data

The number of times a value occurs is its frequency, written $f$. A frequency distribution lists each value with its frequency, and the total of the frequencies is the number of data values, $n = \sum f$.

For discrete data with only a few different values, each value gets its own row. Tally marks help to count raw data without missing any values.

Grouped data and class intervals

When data are continuous, or discrete with many different values, single values are grouped into classes. Each class is a class interval such as $50 \le t < 55$. Grouping makes a large data set manageable, but the individual values are lost: the table only says how many values fall in each class, not exactly where.

Classes must not overlap and must not leave gaps, so every value belongs to exactly one class. Inequalities make this clear: $50 \le t < 55$ and $55 \le t < 60$ put the value 55 in the second class only. Between 5 and 15 classes usually give a useful picture.

The table groups the waiting times between 272 eruptions of the Old Faithful geyser in Yellowstone National Park into classes of width 5 minutes.

Data: waiting times between 272 eruptions of Old Faithful, from Härdle, Smoothing Techniques (1991), the faithful data set in R.

Class boundaries for rounded data

When classes are written with whole numbers, such as "150–154 cm", the true class boundaries depend on how the data were rounded. Heights recorded to the nearest centimetre in the class 150–154 are true heights from 149.5 cm up to (but not including) 154.5 cm, so the boundaries are 149.5 and 154.5 and the class width is 5. Ages are different again: "15–19 years" in completed years means from the 15th birthday up to the 20th birthday, so the boundaries are 15 and 20.

Mid-interval values

Each class is represented by its mid-interval value, halfway between its boundaries. Mid-interval values are used to estimate the mean of grouped data in 4.3.

Histograms

Drawing a histogram

A histogram shows grouped continuous data. Each class is drawn as a bar whose edges sit on the class boundaries, so the bars touch: there are no gaps, because there are no gaps between the classes of a continuous variable. When all classes have the same width, the height of each bar is the frequency of the class.

To draw a histogram:

1. Draw a continuous horizontal scale for the variable, covering every class boundary, and label it with the variable and its units.

2. Draw a vertical scale for frequency, starting at 0.

3. For each class, draw a bar from the lower to the upper boundary with height equal to the frequency.

The Old Faithful histogram shows a feature that the table hides: the waiting times form two separate groups. A distribution with two peaks is called bimodal. Geologists link the two groups to the length of the previous eruption: a short eruption is usually followed by a short wait, and a long eruption by a long wait.

Discrete data are shown in a bar chart (or vertical line graph) instead, with a separate bar or line at each value and gaps between them, because no values lie in between.

Reading a histogram

The class with the tallest bar is the modal class (see 4.3). The number of values in a range of classes is the sum of their frequencies. The overall shape (one peak or two, symmetric or with a long tail) is described in the last part of this subtopic.

Cumulative frequency

Cumulative frequency tables

The cumulative frequency up to a value is the number of data values less than or equal to it (for grouped data, less than the upper class boundary). It is a running total of the frequencies.

For the Old Faithful data, each cumulative frequency is the previous one plus the frequency of the next class.

The last cumulative frequency always equals the total number of data values, which is a useful check.

Cumulative frequency graphs

A cumulative frequency graph plots each cumulative frequency against the upper class boundary of its class, because only at the end of a class is it certain that all the values in it have been counted. The graph starts at the lower boundary of the first class with cumulative frequency 0. The points are joined with a smooth curve or with straight line segments; joining them with straight segments assumes the values are spread evenly within each class.

The gradient of the graph shows how densely the data are packed: the curve is steep where many values fall in a short interval (75 to 85 minutes) and nearly flat where there are few (65 to 70 minutes). A cumulative frequency graph is always increasing or level, never decreasing.

Median, quartiles and percentiles

Ordered data are divided into four equal parts by three values. The median is the middle value. The lower quartile $Q_1$ has a quarter of the data below it, and the upper quartile $Q_3$ has three quarters below it.

More generally, a percentile gives the value below which a stated percentage of the data lie. The 10th percentile has 10% of the data below it; $Q_1$, the median and $Q_3$ are the 25th, 50th and 75th percentiles.

To estimate any of these from a cumulative frequency graph with total frequency $n$:

1. Find the cumulative frequency: $\frac{n}{4}$ for $Q_1$, $\frac{n}{2}$ for the median, $\frac{3n}{4}$ for $Q_3$, and $\frac{p}{100} \times n$ for the $p$th percentile.

2. Draw a horizontal line from this value on the vertical axis to the curve.

3. Draw a vertical line down to the horizontal axis and read off the value.

For grouped data the values are estimates, because the exact data values are not known. The precise positions used for raw data, such as the $\frac{n+1}{2}$th value, matter only for small data sets; for a graph, use $\frac{n}{2}$, $\frac{n}{4}$ and $\frac{3n}{4}$.

Range and interquartile range

Two measures of spread can be found from the graph. The range is the largest value minus the smallest; from grouped data it can only be estimated, as the upper boundary of the last class minus the lower boundary of the first. The interquartile range is the spread of the middle half of the data.

The IQR is not affected by a few extreme values, because it ignores the lowest and highest quarters of the data. The range depends only on the two most extreme values, so a single outlier can change it a great deal.

These estimates agree well with the exact values from the raw data ($Q_1 = 58$, median $76$, $Q_3 = 82$), because 272 values are spread fairly evenly within each class.

Reading the graph the other way

A cumulative frequency graph can also be read from the horizontal axis upwards. Starting at a value of the variable, go up to the curve and across to read how many data values lie below it. Dividing by $n$ and multiplying by 100 gives the percentage of the data below that value, which is the percentile it represents.

Box-and-whisker diagrams

The five-number summary

A data set can be summarised by five values: the minimum, $Q_1$, the median, $Q_3$ and the maximum. A box-and-whisker diagram (box plot) draws this five-number summary to scale.

For raw data, the quartiles used in this course follow the method of a GDC (see 4.3 for the details): order the data, find the median, then take $Q_1$ as the median of the values below it and $Q_3$ as the median of the values above it, leaving out the median itself when the number of values is odd.

Outliers on a box plot

An outlier is a value more than $1.5 \times \mathrm{IQR}$ beyond the nearer quartile. The two boundaries, sometimes called fences, are

Any value below the lower boundary or above the upper boundary is an outlier. On the box plot the whisker stops at the most extreme value that is not an outlier, and each outlier is marked separately with a cross. The boundaries themselves are not drawn.

For the river lengths, $\mathrm{IQR} = 688 - 310 = 378$ miles, so the upper boundary is $688 + 1.5 \times 378 = 1255$ miles and the lower boundary is $310 - 567 = -257$ miles. No river can be shorter than $-257$ miles, so there are no low outliers, while ten rivers are longer than 1255 miles. Whether such values should be kept is discussed in 4.1: these are genuine rivers, so they stay.

Shape and skewness

The shape of a distribution is described by its skewness. A symmetric distribution looks the same on both sides of its centre. A positively skewed distribution has a long tail to the right: most values are small, with a few much larger ones, as with incomes or the river lengths. A negatively skewed distribution has a long tail to the left: most values are large, with a few much smaller ones, as with the scores on an easy test.

On a box plot, skewness shows in the position of the median within the box and in the lengths of the whiskers:

Skewness in this course is described informally, in words; it is not calculated. A box plot cannot show a bimodal shape: the Old Faithful waiting times, with their two peaks, give an ordinary-looking box plot, which is why a histogram should be drawn as well when the shape matters.

Comparing distributions with box plots

Box plots drawn one above the other on the same scale make it easy to compare data sets. A comparison should cover two things, always in the context of the data:

• Centre: compare the medians, and say which group has typically higher values.

• Spread: compare the IQRs (and the ranges), and say which group is more variable or more consistent.

Shape and outliers can be mentioned as well. The parallel box plots compare the birth weights of babies whose mothers did or did not smoke during pregnancy.

The data show an association between smoking and lower birth weight, but on their own they do not prove that smoking causes it, since mothers who smoked may differ in other ways. Correlation and causation are discussed in 4.4.

Proportions from a box plot

Because each section of a box plot (lower whisker, the two halves of the box, upper whisker) holds a quarter of the data, a box plot also shows proportions. About 25% of the values lie above $Q_3$, 50% lie between $Q_1$ and $Q_3$, and so on. A value that falls between two of the five numbers can only be placed within a range of percentages.

Exam-style questions on presenting data

A race organiser recorded the times $t$ minutes taken by 80 runners to complete a 5 km race, as summarised in the table.

Self-check

• To do:

  I can construct frequency tables for discrete data and grouped frequency tables for continuous data.

• To do:

  I can find class boundaries, class widths and mid-interval values, including for rounded data and ages.

• To do:

  I can draw a histogram with equal class widths and describe the shape it shows.

• To do:

  I can construct a cumulative frequency table and graph, plotting at upper class boundaries.

• To do:

  I can estimate the median, quartiles, percentiles, range and IQR from a cumulative frequency graph.

• To do:

  I can find a five-number summary, identify outliers with the $1.5 \times \mathrm{IQR}$ rule and draw a box plot.

• To do:

  I can describe a distribution as symmetric, positively skewed or negatively skewed.

• To do:

  I can compare two distributions from box plots, commenting on centre and spread in context.

1. (a)

   Find the lower quartile and the upper quartile of the recorded heights.

   [2]2 marks

   Markscheme (a)

   $Q_1 = 5.35$ A1

   $Q_3 = 7.15$ A1

2. (b)

   Show that $0.6$ and $11.8$ are both outliers.

   [2]2 marks

   Markscheme (b)

   $\text{IQR} = 7.15 - 5.35 = 1.8$ (A1)

   $Q_1 - 1.5 \times 1.8 = 2.65$

3. (c)

   Every seedling was at least $3$ cm tall when it was planted at the start of the three weeks. For each of the two outliers, state whether it should be kept in or removed from the data, giving a reason.

   [2]2 marks

   Markscheme (c)

   $0.6$ should be removed: a seedling at least 3 cm tall cannot have a height of 0.6 cm three weeks later, so it must be an error in recording (e.g. a misplaced decimal point or a value in different units) R1

   $11.8$ should be kept: it is possible for one seedling to grow much more than the others, so it is a valid (genuine) data value R1

   Note: Award R1 for each correct decision with a valid reason. Do not award R1 for a decision without a reason.

The upper half is $19, 20, 21, 22, 24, 25, 38$, so $Q_3 = 22$.

Five-number summary: minimum 12, $Q_1 = 15$, median 18, $Q_3 = 22$, maximum 38.

(b) $\mathrm{IQR} = 22 - 15 = 7$ and $1.5 \times 7 = 10.5$.

Since $38 > 32.5$, the value 38 is an outlier. Every other value lies between 4.5 and 32.5, so there are no other outliers.

(c) The upper whisker ends at 25, the largest value that is not an outlier, and 38 is marked with a cross.

$0.6 < 2.65$ and $11.8 > 9.85$, so both are outliers AG