Teachers and Income: What Did the Kindergarten Study Really Find?

“But what about the kindergarten study?”  I hear this question all the time.  Questioners are referring, of course, to Chetty et al.’s “How Does Your Kindergarten Classroom Affect Your Earnings?  Evidence from Project STAR.”  Fans of the paper often claim that it shows that spending more on better teachers passes a cost-benefit test with flying colors: “One good kindergarten teacher adds $782,000 in value per year!”*

If you actually read the paper, though, a different story emerges.  The authors are extremely careful and transparent.  Here’s what they really find about the effect of early schooling on adult income:

1. Project STAR was an experiment designed to test the effect of class size.  The experiment found that students assigned to small classes earned $4 more per year.  If you add demographic controls, students assigned to small classes earned $124 less per year. (more)

2. You can use Project STAR’s data to (non-experimentally**) test for other effects.  When you do, almost all measures of teacher quality fail to increase adult earnings:

The few other observable teacher characteristics in the STAR data (degrees, race, and progress on a career ladder) have no significant impact on scores or earnings.

3. There is one measure of teacher quality that does matter: Whether the teacher has more than 10 years of experience.  Chetty et al. find that students assigned to a kindergarten teacher over this experience cut-off eventually earn $1093 extra dollars per year.  But bear two reservations in mind.  (a) The t-stat is only 2.4 – extremely low for a non-experimental test with 6005 observations.  (b) If you measure experience in years, rather than using their binary “more than 10 years of experience” variable, the point estimate is a statistically insignificant $57 per year.

4. Teacher experience only matters in kindergarten:

The effect of teacher experience on test scores is no longer statistically significant in grades 1-3. Consistent with this result, teacher experience in grades 1-3 also does not have a statistically significant effect on wage earnings.

By my count, Chetty et al. have five measures of teacher quality (degrees, race, progress on a career ladder, >10 years experience, and years of experience), study four different grades (K, 1, 2, and 3), and discover precisely one statistically significant effect on income.  With 20 different measures, you’d expect one to be statistically significant at the 5% level by chance alone.  And these are just the non-experimental results.  Experimentally, Project STAR finds that class size does not raise adult income.

“How Does Your Kindergarten Classroom Affect Your Earnings?” is one of the most impressive empirical papers ever written.  The data collection is amazing.  The empirical analysis is clear and careful.  Above all else, the authors’ methods are transparent.  Lesser authors would have buried the conflicting findings.  Chetty et al. took the high road. 

Unfortunately, the world is so eager for stories about the power of early education that their paper is being badly misinterpreted.  Chetty et al. don’t confirm romantic hopes about teachers that change young minds forever.  Despite their pro-education tone, they expose these hopes as wishful thinking.

* This is the paper’s back-of-the-envelope calculation of the present discounted value of 1 SD improvement in “classroom quality,” not teacher quality.  But the figure has a life of its own.

** Raj Chetty, who kindly previewed this post, argues that all their results are “experimental,” but this is a stretch.  The STAR experiment was randomly assigning kids to big or small classes.  Using the same data to isolate other effects remains observational.  Any of the measured effects could ultimately reflect correlated confounding variables – as they could in any observational study.  E.g. maybe experienced teachers have nicer classrooms, or more teacher’s aides.

The post appeared first on Econlib.