3:30–4:30 pm Maria Goeppert-Mayer Lecture Hall
Physicists do Social Science: The Case of GRE's
Michael Weissman, University of Illinois, Urbana-Champaign
A well-funded influential paper concluded that GREs do not help predict whether physics grad students will get Ph.D.s. The paper makes numerous elementary statistics errors, including introduction of unnecessary collider stratification bias, variance inflation by collinearity and range restriction, a peculiar choice of null hypothesis on subgroups, blurring the distinction between failure to reject a null and accepting a null, an obviously biased method for imputing missing data, an extraordinary procedure for radically inflating confidence intervals in a figure, and omission of needed data. I estimate the actual approximate incremental predictive power of GREs using standard methods by scrounging data found in the paper and in equally flawed follow-ups. These papers exhibit exactly the sort of research techniques which we should be teaching students to avoid. Widespread acceptance of such methods raises questions about the culture of physics.