What’s in a Letter? Using Natural Language Processing to Investigate the Prevalence of Linguistic Biases in Teacher Letters of Recommendation for Postsecondary Applications

Brian Heseung Kim

While scholars have already uncovered many ways that low-income, first-generation-to-college, and racial/ethnic minority students are systematically disadvantaged across the postsecondary application portfolio – from standardized tests to advanced course-taking opportunities – we know almost nothing about whether teacher letters of recommendation advance or impede these students’ college aspirations. This blind spot is especially concerning given mounting evidence that recommendation letters in other contexts can contain biased language, that teachers can form biased perceptions of their students’ abilities, and that narrative application components more generally may contribute to racial discrimination in selective college admissions. Meanwhile, institutions continue to move away from standardized test scores — positioning recommendations to be even more prominent going forward.In my dissertation, I will conduct the first system-wide, large-scale text analysis of teacher recommendation letters in postsecondary applications. With application and recommendation data from 2 million students, 500,000 teachers, and 400 postsecondary institutions, I will examine the prevalence of linguistic biases within these letters: whether students are described by teachers in systematically different ways across racial/ethnic, gender, and socioeconomic groups. By combining rigorous econometric frameworks with sophisticated natural language processing (NLP) techniques, I can analyze variation in letter characteristics at unprecedented scale and fidelity while accounting for salient confounding factors like student academic and extracurricular qualifications. It is paramount that we better understand the role of these letters in ameliorating or exacerbating inequity, and these analyses will provide urgent insights for college admissions practices, affirmative action litigation, and NLP methodologies for education research.

About Brian Heseung Kim

Brian Heseung Kim is a doctoral student in the University of Virginia’s Education Policy program. His primary research interests revolve around supporting students of historically underserved backgrounds through major decision-making junctures like the postsecondary application process and the post-graduation job application process. Leveraging a combination of traditional econometric methods, machine learning approaches, and natural language processing techniques, his current projects examine the potential for biases in teacher recommendation letters for postsecondary applications, the variation in virtual advising practices in a college completion nudging intervention, and the use of job recommendation algorithms to support low-income students through the post-graduation job search. In future research, Brian will continue exploring this intersection of education, data science, economics, and ethics.Before beginning his graduate program, Brian studied education from a wide variety of perspectives: as a traditionally-certified high school teacher and teacher leader in Maine, a college counselor with Upward Bound, a tutor of elementary school students through America Reads and Counts, an instructor of economic game theory for gifted and talented students, and an admissions interviewer for Bowdoin College, where he received his B.A. in English and Economics. During his doctoral program, Brian has also served as a Research and Equity intern with the Virginia Department of Education and a Data Science Fellow with DonorsChoose. In his own time, Brian enjoys hiking, 3D printing, music production, and generally getting good-but-not-very-good at all things