Lies, Damned Lies and Statistics

A new study comparing public and private school 4th graders finds that public school kids are ahead after all. (From the New York Times) "Though private school students have long scored higher on the national assessment, commonly referred to as "the nation's report card," the new study used advanced statistical techniques to adjust for the effects of income, school and home circumstances." ...

"The study found that while the raw scores of fourth graders in Roman Catholic schools, for example, were 14.3 points higher than those in public schools, when adjustments were made for student backgrounds, those in Catholic schools scored 3.4 points lower than those in public schools. A spokeswoman for the National Catholic Education Association did not respond to requests for comment.

The exam is scored on a 0-to-500-point scale, with 235 being the average score at fourth grade, and 278 being the average score at eighth grade. A 10-to-11-point difference in test scores is roughly equivalent to one grade level.

The study also found that charter schools, privately operated and publicly financed, did significantly worse than public schools in the fourth grade, once student populations were taken into account. In the eighth grade, it found, students in charters did slightly better than those in public schools, though the sample size was small and the difference was not statistically significant.

"Over all," it said, "demographic differences between students in public and private schools more than account for the relatively high raw scores of private schools. Indeed, after controlling for these differences, the presumably advantageous private school effect disappears, and even reverses in most cases."

Bad news for private and charter schools, although I'm as suspicious as Benjamin Disraeli of "advanced statistical techniques". I found out when DH was in grad school that you can massage numbers or choose to ignore certain types of data so your results show anything you like. The article is currently being peer-reviewed. I hope the NYT will be as eager to tell us what other sociologists thought of the study.