Kaitlyn Zhou
Incoming Assistant Professor at Cornell Information Science
Researcher at Together AI
PhD in Computer Science from Stanford NLP
kaitlynz@cornell.edu
Bio
I will be joining Cornell Information Science as an Assistant Professor in August 2026! I will be recruiting students in natural language processing and human-computer interactions! If you're interested, please reach out!
I research the dynamics of human interaction with language models (human-LM interaction), focusing on how generated language shapes human decision-making, reliance, and trust. My research 1) identifies model overconfidence as a key risk of human-LM interactions [EMNLP'23, ACL'24], 2) builds context-aware evaluation frameworks for emergent human-LM interactions [NAACL'25], and 3) reimagines human-LM interactions for historically marginalized user groups via human-centered task ideation.
I just graduated with a Ph.D. in Computer Science from Stanford University, advised by Dan Jurafsky. Prior to Stanford, I served on the University of Washington Board of Regents as appointed by Governor Jay Inslee. I studied Computer Science and Human Centered Design and Engineering at the University of Washington (B.Sc., B.Se., M.S.) where I won the School of Engineering Dean's Medal of Excellence. I've been fortunate to spend summer doing research at Microsoft Research FATE (hosted by Alexandra Olteanu and Su Lin Blodgett) and Allen Institute for AI (hosted by Maarten Sap, Jena Hwang, and Xiang Ren).
Recent News
- October 2025: Our paper on overconfident multilingual language models (Humans overrely on overconfident language models) was presented at COLM 2025!
- August 2025: Started as a Research Scientist at Together AI working with James Zou
- June 2025: Our paper on reliance of various sociolects (Not Like Us, Hunty) was presented at FAccT 2025!
- April 2025: Our paper on human-LLM reliance (REL-A.I.) won Best Paper Runner-Up at NAACL 2025!
- February 2025: Two papers (REL-A.I. and Rethinking Word Similarity) accepted to NAACL as oral presentations!
Publications
Most recent publications on Google Scholar.
Humans overrely on overconfident language models, across languages
Neil Rathi, Dan Jurafsky, and Kaitlyn Zhou
arXiv preprint arXiv:2507.06306. 2025.
Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance
Kaitlyn Zhou, Jena D. Hwang, Xiang Ren, Nouha Dziri, Dan Jurafsky, and Maarten Sap
NAACL'25 (Best Paper Runner-Up): North American Chapter of the Association for Computational Linguistics. 2025.
Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty
Kaitlyn Zhou, Jena D. Hwang, Xiang Ren, and Maarten Sap
ACL'24: Annual Meeting of the Association for Computational Linguistics. 2024.
Navigating the Grey Area: Expressions of Overconfidence and Uncertainty in Language Models
Kaitlyn Zhou, Dan Jurafsky, and Tatsunori Hashimoto
EMNLP'23: Empirical Methods in Natural Language Processing. 2023.
Spotlight Tweets: A Lens for Exploring Attention Dynamics within Online Sensemaking during Crisis Events
Kaitlyn Zhou, Tom Wilson, Kate Starbird, and Emma S. Spiro
TSC'23: Transactions of Social Computing (Journal). 2023
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications
Kaitlyn Zhou, Su Lin Blodgett, Adam Trischler, Hal Daumé III, Kaheer Suleman, Alexandra Olteanu.
NAACL'22: North American Chapter of the Association for Computational Linguistics. 2022
The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality.
Mitchell L. Gordon, Kaitlyn Zhou, Kayur Patel, Tatsunori Hashimoto, and Michael S. Bernstein.
CHI'21. ACM Conference on Human Factors in Computing Systems. 2021
Humans overrely on overconfident language models, across languages
Neil Rathi, Dan Jurafsky, and Kaitlyn Zhou
arXiv preprint arXiv:2507.06306. 2025.
Not Like Us, Hunty: Measuring Perceptions and Behavioral Effects of Minoritized Anthropomorphic Cues in LLMs
Jeffrey Basoah, Daniel Chechelnitsky, Tao Long, Katharina Reinecke, Chrysoula Zerva, Kaitlyn Zhou, Mark Díaz, and Maarten Sap
FAccT'25: ACM Conference on Fairness, Accountability, and Transparency. 2025.
ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanations
Brihi Joshi, Keyu He, Sahana Ramnath, Sadra Sabouri, Kaitlyn Zhou, Souti Chattopadhyay, Swabha Swayamdipta, and Xiang Ren
ACL'25 (Findings): Association for Computational Linguistics. 2025.
Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance
Kaitlyn Zhou, Jena D. Hwang, Xiang Ren, Nouha Dziri, Dan Jurafsky, and Maarten Sap
NAACL'25 (Best Paper Runner-Up): North American Chapter of the Association for Computational Linguistics. 2025.
Rethinking Word Similarity: Semantic Similarity through Classification Confusion
Kaitlyn Zhou, Haishan Gao, Sarah Chen, Dan Edelstein, Dan Jurafsky, Chen Shani
NAACL'25 (Oral) North American Chapter of the Association for Computational Linguistics. 2025.
Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty
Kaitlyn Zhou, Jena D. Hwang, Xiang Ren, and Maarten Sap
ACL'24: Annual Meeting of the Association for Computational Linguistics. 2024.
Navigating the Grey Area: Expressions of Overconfidence and Uncertainty in Language Models
Kaitlyn Zhou, Dan Jurafsky, and Tatsunori Hashimoto
EMNLP'23: Empirical Methods in Natural Language Processing. 2023.
Spotlight Tweets: A Lens for Exploring Attention Dynamics within Online Sensemaking during Crisis Events
Kaitlyn Zhou, Tom Wilson, Kate Starbird, and Emma S. Spiro
TSC'23: Transactions of Social Computing (Journal). 2023
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications
Kaitlyn Zhou, Su Lin Blodgett, Adam Trischler, Hal Daumé III, Kaheer Suleman, Alexandra Olteanu.
NAACL'22: North American Chapter of the Association for Computational Linguistics. 2022
Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words
Kaitlyn Zhou, Kawin Ethayarajh, Dallas Card, Dan Jurafsky
ACL'22: Annual Meeting of the Association for Computational Linguistics. 2022.
Richer Countries and Richer Representations
Kaitlyn Zhou, Kawin Ethayarajh, Dan Jurafsky
ACL'22: Annual Meeting of the Association for Computational Linguistics (Findings). 2022.
On the Opportunities and Risks of Foundation Models
Rishi Bommasani, Drew A. Hudson ... Kaitlyn Zhou , Percy Liang et al.
Preprint
The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality.
Mitchell L. Gordon, Kaitlyn Zhou, Kayur Patel, Tatsunori Hashimoto, and Michael S. Bernstein.
CHI'21. ACM Conference on Human Factors in Computing Systems. 2021
Assembling strategic narratives: Information operations as collaborative work within an online community.
Tom Wilson, Kaitlyn Zhou, and Kate Starbird
CSCW'2018: ACM SIGCHI Conference on Computer-Supported Cooperative Work & Social Computing
Centralized, parallel, and distributed information processing during collective sensemaking.
Peter Kraft, Kaitlyn Zhou, Isabelle Edwards, Kate Starbird, and Emma S. Spiro
CHI'17. ACM Conference on Human Factors in Computing Systems. 2017