PISA test, analysis of school performance.

PISA test data

For a recent, great Meetup of the NYC Data Wranglers we were exploring three very different data-sets. While I was working with a small team on predicting oil-prices from historical OPEC data during the Meetup, on the train to the Meetup I did some exploratory data analysis on a data-set measuring the influence of external factors on kid’s school performance. Here is a small summary of my findings.

The data was collected by the “Programme for International Student Assessment” (PISA test) in order to identify potential policies to help improving school performance. Here I analyze PISA test data reporting students performance in reading, science and math for 68 countries (34 OECD members, 34 non-OECD members). The performance for each country is reported for groups classified by i) class size, ii) age of father, iii) age of mother, iv) public or private school or v) teacher morale.

Using this data I want to test the following hypotheses:

1) Performance in reading, science and math are highly correlated.
2) Students in OECD member countries outperform non-member countries.
3) The smaller the class, the higher the performance.
4) Private schools outperform public schools.
5) Teacher morale is an important contributor to performance.
6) The older the parents, the higher the performance.

All data and code used to perform this analysis can be downloaded from my github repository. While my narrative sometimes suggests causality, I am well aware that the data presented here can only measure correlation not causation.

Correlation of performance metrics

I hypothesize that the systematic differences in the students performance outweigh the topic specific differences. To test this hypothesis, I combined all five tables, such that each group in each country contributes 3 performance measurements. I then scatter-plotted performance measurements for reading, math and science pairwise (Figure 1, lower triangular panels). The diagonal appearance of all pairwise scatterplots indicates a high correlation between performance measurements. In agreement, a Pearson correlation test results in correlation coefficients > 0.95 for all pairs (Figure 1, upper triangular panels). This confirms that all three measurements are highly correlated.

Figure 1. School performance measures are highly correlated. Scatter plots of performance measures (lower triangular panels) for OECD member (green) and non-member states (red). Densities of performance values (diagonal panels) and pairwise correlation coefficients between performance measures (upper triangular panels). Boxplots of performance values for countries grouped  by OECD membership (right column).

Figure 1. School performance measures are highly correlated. Scatter plots of performance measures (lower triangular panels) for OECD member (green) and non-member states (red). Densities of performance values (diagonal panels) and pairwise correlation coefficients between performance measures (upper triangular panels). Boxplots of performance values for countries grouped by OECD membership (right column). School performance is measured in arbitrary units and scaled such that the mean for each performance measure is 500 and the standard deviation 100.

Performance of OECD member and non-member countries

The Organisation for Economic Cooperation and Development (OECD) acts as a forum to allow comparison of policy approaches, identify good practices and coordinate their realization in member countries. Educational performance quantified here can be used as a measure of the OECD performance itself. I used distinct colors for OECD member (green) and non-members states (red) in Figure 1. Distributions for all three performance measures are significantly different between these groups of countries (Figure 1. Diagonal panels) as measured by Kolmogorov-Smirnoff test ( for reading, math and science). Interestingly, there are minor peaks within the distributions for both classes which overlap with the major peak of the other. This may be due to a confounding variable, and thus the correlation between OECD membership and school performance may not be direct. An obvious candidate for such a confounding variable may be wealth of the country.

School performance and class size

I hypothesize that a higher teacher to student ratio improves school performance. To test this, I visualized the distribution of performance (average of math, science and reading) using boxplots for each class size group for OECD member and non-member states (Figure 2 left panel). Interestingly, my hypothesis “the smaller the class, the better the performance” is wrong: For OECD non-members performance is only decreased by very large classes (> 45 students). Similarly, performance correlates negatively for classes with too many students in OECD member classes. Surprisingly, performance decreases significantly for classes with very few students, resulting in an optimal class size of 31-35 students.

Figure 2. External factors affect school performance as measured by the PISA test. Influence of class size (left), school form (middle) and teacher morale (right) on school performance. Each panel is split for OECD member (right) and non-member states (left). Performance distributions are represented by boxplots for each class. School performance is measured in arbitrary units and scaled such that the mean for each performance measure is 500 and the standard deviation 100.

Figure 2. External factors affect school performance as measured by the PISA test. Influence of class size (left), school form (middle) and teacher morale (right) on school performance. Each panel is split for OECD member (right) and non-member states (left). Performance distributions are represented by boxplots for each class. School performance is measured in arbitrary units and scaled such that the mean for each performance measure is 500 and the standard deviation 100.

School form, teacher morale and school performance

Using the same visualization technique I test wether private schools outperform public schools and whether teacher morale is highly correlated with performance. Both hypotheses hold true (Figure 2 middle and right panel).

School performance and parental age

Finally, I wanted to address if the age of parents is correlated with school performance of their kids. The performances were grouped by age of the mother and father respectively and represented with boxplots (Figure 3). Note that the age of the investigated students is ~15 years. Thus, parents in the first group (<36 years) became parents in their teens. As hypothesized, very young parents correlate with decreased school performance of their kids. While parental age correlates positively with performance for the entire age range for non-OECD countries, there is an optimal age (46-50 years) for OECD countries. The behavior for both OECD member and non-member countries is unspecific with respect to parental gender. Importantly, this correlation is likely to be no causation, but a secondary effect reflecting the tendency of late pregnancy in specific groups of society.

Figure 3. Parental age affects school performance as measured by the PISA test. Influence of maternal (left) and paternal age (right) on school performance. Each panel is split for OECD member (right) and non-member states (left). Performance distributions are represented by boxplots for each class.

Figure 3. Parental age affects school performance as measured by the PISA test. Influence of maternal (left) and paternal age (right) on school performance. Each panel is split for OECD member (right) and non-member states (left). Performance distributions are represented by boxplots for each class. School performance is measured in arbitrary units and scaled such that the mean for each performance measure is 500 and the standard deviation 100.

Summary and interpretation

This exploratory analysis of the PISA test data allowed me to make the following observations concerning school performance:

1) Performance in reading, math and science is highly correlated.
2) Performance is significantly higher in OECD member versus non-member countries. My gut feeling tells me that this is very likely to be a confounding effect, potentially measuring wealth of the country. This hypothesis could be easily tested using for example the GDP as a predictor.
3) There seems to be an optimal class size. Again, this has to be taken with a grain of salt and tests for indirect correlations have to be performed. If this correlation holds and can be extended to a causation, this would be the most interesting result of this analysis, because it would provide an inexpensive way to improve educational performance.
4) Private schools outperform public schools and
5) teacher morale correlates positively with performance. While these results are expected, the latter may provide a relatively inexpensive lever to increase school performance.
6) The older the parents, the better the kids perform. Again, this effect is very likely to be secondary and is not exploitable to improve education.