On the misconsumption of numbers.

Subtitle: how not to read a chart.

Numbers count. More than they should. We live in an age where if it can’t be quantified, it doesn’t matter. More and more we enslave ourselves to the beguiling false sense of control bestowed by numbers and their rationality. “Evidence” is “scientific” and hence objective and neutral.

But scientists aren’t. And how categories are chosen, questions are selected and framed, and instruments deployed makes a massive difference to what numbers come out at the end.

I won’t go into a huge long diatribe here about the discourses constructed by numbers and the numerification of events & phenomena that are far from quantifiable (mostly thanks to economists who wish to be scientists but are not).

Also: I’m a maths teacher and did a degree in pure mathematics–this is not a position adopted from a fear of numbers. Rather, what I really fear, is how a subject I love is abused for ideological ends, mostly by those who would claim they are not ideological.

But here is a little post on how to not read a numerical chart, and how to interrogate it. Some friends, who own the same type of dog that I do, posted this interesting table on a group chat:

If you’re a dog lover, it debunks a few little myths about the assumed linear relationship between human aging and dog aging. The standard rule of thumb is 1 dog year = 7 human years. So a dog of 5 is about 35 human years old. A dog of 12 is about 84 human years old.

Actually, this is a deeply emotional topic. Those of us with furry companions are constantly wracked by the asynchronous relationship between our life span and theirs. We are forced to watch some of our best friends age and die before our eyes. It’s testimony to the value they add to our lives that we continue to choose to do so, despite the impending loss and grief their demise will bring.

But here’s the rub with this little chart.

What is first striking is that the rate at which an infant dog ages is very rapid. Anyone who has raised a puppy will instinctively know this: they are near full size by about 6-8 months. Just like a 14 year old is near full size as a human.

The second is that, actually, in the upper ends of age, the linear model overestimates age. That’s right: your 12 year old hound is more between human age of 64-77, not 84 years old. Here’s that table as a chart for easier visualization. I’ve added the linear model as a comparison.

But as we talked about our aging pooches, two important misreadings came to the fore. My poor mates were wondering why I got on my soapbox about this. Sorry friends!

I got excitable because: if I could allay the misconsumption and over-reach of grossly simplifying statistics in my everyday work, half the battle would be won.

So here are two key errors one could make when reading the dog chart. And why they matter for schools.

Doggy doo-doo

The first error that crops up with this chart is the normative positioning of what is ‘unusual’.

That canine pups mature so rapidly is striking. But this is normal in the mammal world. Actually, its us humans who are weird. We’re born relatively helpless compared to most animals. Some postulate this is actually a premature birth, precipitated by our huge heads that wouldn’t get through our mothers’ birth canals if we hung on any longer. We’ve extra-large cranial cavities (to house big brains), and this is at odds with narrow pelvis bones. Others say its a perfectly normal gestation period, its just that building big brains takes a long time, so post-birth development is prolonged. Whatever the reason, human babies are pretty useless. To make the point: check out these photos of a baby springbok getting to its feet having just dropped.

So dogs mature rapidly, like most mammals. Thinking this rate of aging is crae is actually more indicative of our normative assumptions about what is the baseline and what is the comparator. We subtly delineate what is the ‘centre’ (the normal) and what is the ‘deviant’ which is compared and adjudged as ‘weird’, ‘crazy’ or ‘fascinating’.

(Just as a sub-text; this is the exact same argument that decolonisation advocates make about positioning Europe as the ‘normal’ or the ‘good’ and everybody else as ‘weird’ or ‘exotic’.)

The second error was brought about by trying to locate our own pooches on the chart. This error is the lossiness of internal variation created by categorizing into groups, namely ‘small dog’, ‘medium dog’ & ‘large dog’.

We have dogs weighing at 30kg, or about 66lbs. Well inside the 50lbs< category for a ‘large dog’.

Except these numbers for ‘large dog’ are averages. And the range of the category ‘large dog’ is from 50lbs… to 140lbs. I mean: they even used the picture of a Great Dane! That’s right. A 30kg dog is ‘large’. And so is a 60kg dog. Anyone who’s owned a Pointer (30kgs) and a Great Dane (more like 50-60kgs) will tell you that Pointers live a lot longer than Danes.

So there’s a whole range of variety all called ‘large dog’. To assume a 30kg/66lb dog ages at the same rate as a 60kg/132lb dog but not at the same rate as a 25kg/49lb dog (‘medium’) is to miss the continuity of the underlying data that is presented as discrete. For our dogs, a blend of medium and large was more appropriate, given their position at the lower end of the ‘large dog’ category.

So what? Why should I care? What’s this got to do with schools and education?

Lots.

We make decisions all the time about schools, education policy and what needs changing based on data. And a lot of the narratives are set by a few privileged individuals who get to set and publish the numbers. We consume them uncritically. And it’s a massive problem.

Here are two examples of the exact same two mistakes as made with the dog chart, that are made every day with public schools in South Africa.

1. Normative assumptions about what is the baseline and what is the comparator

In education circles, we use ex-White (euphemized to “ex-Model C”) schools as the norm to which everything else is compared. It’s JUST like using humans as the basis on which all other mammals are compared. Model C schools that produce ‘good’ results by the narrow measure of test scores1 make up less than 5% of all schools. They are not the norm. But you’d be forgiven for thinking they were if you read our education policy documents. Our policies talk about schools as if these were the norm, and the ideal. Big mistake.

You can see it in the newspapers and hear it on the radio as well. The chattering middle classes think that the schools they send their kids to are the norm and ideal. It becomes their point of comparison, to which everything else is compared. Just like assuming human aging rates are the point for comparing mammalian aging rates.

It’s a problem because we keep prescribing theories of change and interventions for the 95% based on the 5%. I’m sure the inherent problems with doing that need no further explanation.

2. Lossiness of internal variation created by categorizing into groups

Beyond the massive (moral) problem of calling schools dysfunctional and assuming they are ‘dysfunctional’ in all the same ways, here’s a different categorization that does the same erasing work as the group “large dogs” did for our dog chart. School quintiles.

We often talk about the quintiles as if they bear any relationship to reality. Firstly they don’t. They are allocated based on geographical location, not on the socio-economic situation of the learners who attend. We lack the apparatus and capacity to evaluate every school’s student population every year for their family’s economic situation. We just can’t do it. So we use the school’s physical location as a very very SWAK proxy. The result is a school can be funded as if it was ‘wealthy’ and serve very poor learners. This funding gap has disastrous consequences for the school concerned.

The second is that there is any difference between Quintiles 1,2 and 3. There isn’t. They are the same. Same public funding. No private funding. Effectively these are the school category “can’t charge fees”. And: like our category “Small dogs”, this lower range hides a lot less variation than the upper categories, even though it looks like three different categories. That is: we create the impression of variation where there is none, by giving groups different names. There’s no difference between Quintiles 1, 2 and 3 in reality.

But the categories for the upper ranges are a different story. Our third false assumption is to assume that schools in Quintile 5 are the same because they are publicly funded the same (same goes for Quintile 4). These two groups are, legally, the “can charge fees” group. Their public funding (intraquintile) is the same. But their private funding (because they can charge fees) is VERY different.

 Being legally allowed to charge fees is all very well and good. But you can charge fees until you whistle “She’ll be coming ’round the mountain”. That doesn’t mean you’ll get them. Fee charging and fee collecting are two entirely different problems.

 

Quintile 5 in particular, is a group that hides a huge amount of variation. Like the category “large dogs”, there are schools in Quintile 5 that command and collect in excess of R30 000 a year for each child they enrol. And there are schools in Quintile 5 that cannot get fees in from one month to the next and have to tell their school-employed teachers (if they have any) that they won’t get paid this month (“sorry”). To talk about “Quintile 5 schools” is to talk about “large dogs”. It’s actually not very useful.

Worse still, equal “quintiles” give the impression that the groups are equal in clout and distribution because they are equal in the number of schools that fall into each group. But just like our dog categories, where “small” is only from 1-8kg (and there’s no such thing as a 0 kg dog!), “medium” is from 8kg-25kg… and “large” is from 25kg… up to 80kg! (think Newfoundland/Landseer dogs). These categories are FAR from equal in weighting by any measure. But parity between groups is constructed in our minds by the way the categories are presented.

And the soapbox is for… ?

My work as a critical education policy sociologist is to point these issues out. These categories and norms aren’t just ‘mistakes’ in schooling: they serve interests. They prop up those with power and silence those without. If I can, through my work, get the general public to consume education statistics more sceptically, with more nuance, there would be more bottom-up accountability from the general demos towards those who make decisions and distribute resources.

Now that’s something to woof about.


1 Ironically judging a specific school based on test scores is like judging an individual animal (or human) based on their numerical age. It misses diet, exercise, genes, environment… as if all 36 year olds are equally healthy and their bodies look the same. Schools with the same test scores are vastly different. And these differences matter.

Published by

Sara

I'm a mathematics teacher currently working in the area of teacher development at the University of Cape Town. I've an interest in language in education, education policy and sociology and general ideas around equity and adequacy in public primary and secondary schooling in South Africa and other developing contexts. I'm currently doing my PhD at UCT. When not thinking, reading and writing about education issues, or working with teachers, I can normally be found either somewhere on the slopes of Table Mountain with my dog, or behind a piano.