Wednesday, July 30, 2014

A Tale of Three Students

Most normal people (a.k.a. non-statisticians) tend to think in linear terms. The trouble is most normal behavior doesn't tend to be very linear. As a rule, you're better off thinking in terms of of U-curves (things go either up or down then come back) and S-curves (things are level, they move either up or down, then they level off again). These are still approximations, but they are usually more reasonable approximations. 

This is particularly true in education. Arguably the best way to model learning is with a series of S-curves. We work and study with little progress, then we have a period of improvement, then we hit another plateau.

If we think in terms of straight lines, ranking students is relatively easy.

But if we think in terms of S-curves, which is a great deal more realistic, things get more complicated.

The x-axis doesn't actually mean anything (this is made-up data), but let's say it represents months studying a language and the lines represent daily test scores. Now, who's the best student depends on when you ask, and that raises some troubling points. 

We tend to put too much faith both in the metrics we use to evaluate students and in the linearity of human behavior. We are not straight -line animals but we have a bad habit of making straight line decisions. In this case, think of what would happen if we made a decision on who to drop from a program after two months. 

No comments:

Post a Comment