Research

Manuscripts

Inioluwa Deborah Raji, Lydia T. Liu, Angela Zhou, et al.
Bridging Prediction and Intervention Problems in Social Systems.
Working paper. Full author list and draft available upon request.

View abstract

Many applications of algorithmic decision support (ADS) are often framed as isolated prediction problems—aimed at capturing relevant information about one population sample and extrapolating patterns to others. However, in practice, ADS systems act more like holistic policy interventions once deployed. Their predictions are shaped by complex interactions among stakeholders, infrastructures, and deployment contexts, which also mediate the system's real-world impact. This white paper revisits the limitations of the prediction paradigm in describing machine learning’s role in social systems. We offer statistical frameworks and tools to evaluate ADS models beyond prediction accuracy—advocating instead for an intervention-based lens in their design, evaluation, and implementation.

Kara Schechtman, Benjamin Brandon, Jenise Stafford, Hannah Li^{^}, Lydia T. Liu^{^}.
Discretion in the Loop: Human Expertise in Algorithm-Assisted College Advising. Manuscript in submission.
Poster to appear at IC2S2, Norrköping, Sweden, July 2025.
A preliminary version was accepted for presentation at the Workshop on AI & Analytics for Social Good, College Park, MD, May 2025.

Peer-reviewed publications

Journal articles

Joshua Cohen and Lydia T. Liu.
The Reach of Fairness.
ACM Journal on Responsible Computing. Just Accepted (February 2025). [eprint] [preprint]
Lydia T. Liu^*, Serena Wang^*, Tolani Britton^{^}, Rediet Abebe^{^}.
Reimagining the Machine Learning Life Cycle to Improve Educational Outcomes of Students.
Proceedings of the National Academy of Sciences 120.9 (2023): e2204781120. [eprint] [slides]
A preliminary version (“Promises and Pitfalls of Machine Learning in Education”) was presented as a poster at the Research Conference on Communications, Information, and Internet Policy (TPRC 2021).
Lydia T. Liu, Feng Ruan, Horia Mania, Michael I. Jordan.
Bandit Learning in Decentralized Matching Markets.
Journal of Machine Learning Research, 22(211):1−34, 2021. [journal] [arxiv]
Presented as a poster at Workshop on Operations of People-Centric Systems (EC ‘21)
Zhizhen Zhao, Lydia T. Liu, Amit Singer.
Steerable ePCA: Rotationally Invariant Exponential Family PCA.
IEEE Transactions on Image Processing, vol. 29, pp. 6069-6081, 2020. [doi] [arxiv]
Lydia T. Liu^*, Edgar Dobriban^*, and Amit Singer.
ePCA: High Dimensional Exponential Family PCA.
Annals of Applied Statistics 2018, Vol. 12, No. 4, 2121-2150. [doi] [arxiv] [software]

Conference proceedings

Inioluwa Deborah Raji and Lydia T. Liu.
Evaluating Prediction-based Interventions with Human Decision Makers In Mind.
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics (AISTATS), 2025. [arxiv]
A preliminary version was presented as a poster at the ICML 2024 workshop on Humans, Algorithmic Decision-Making and Society.
Lydia T. Liu, Solon Barocas, Jon Kleinberg, Karen Levy.
On the Actionability of Outcome Prediction.
Proceedings of the AAAI conference on Artificial Intelligence, 2024. [doi] [arxiv]
Lydia T. Liu, Nikhil Garg, Christian Borgs.
Strategic ranking.
Proceedings of The 25th International Conference on Artificial Intelligence and Statistics (AISTATS), 2022. [arxiv]
Presented as a poster at the ACM conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO), 2021.
Esther Rolf, Max Simchowitz, Sarah Dean, Lydia T. Liu, Daniel Björkegren, Moritz Hardt, Joshua Blumenstock.
Balancing Competing Objectives with Noisy Data: Score-Based Classifiers for Welfare-Aware Machine Learning.
Proceedings of the 37th International Conference on Machine Learning (ICML), 2020. [arxiv]
Lydia T. Liu^*, Horia Mania^*, Michael I. Jordan.
Competing Bandits in Matching Markets.
Proceedings of The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), 2020. [arxiv]
Lydia T. Liu, Ashia Wilson, Nika Haghtalab, Adam Tauman Kalai, Christian Borgs, Jennifer Chayes.
The Disparate Equilibria of Algorithmic Decision Making when Individuals Invest Rationally.
Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (ACM FAT*), Barcelona, Spain, 2020. [doi] [arxiv]
Lydia T. Liu^*, Max Simchowitz^*, Moritz Hardt.
The Implicit Fairness Criterion of Unconstrained Learning.
Proceedings of the 36th International Conference on Machine Learning (ICML), Long Beach, California, USA, 2019. [arxiv] [code]
Chi Jin^*, Lydia T. Liu^*, Rong Ge, Michael I. Jordan.
On the Local Minima of the Empirical Risk.
Advances in Neural Information Processing Systems (NeurIPS) 32, Montréal, Canada, 2018. Spotlight. [arxiv]
Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, Moritz Hardt.
Delayed Impact of Fair Machine Learning.
Proceedings of the 35th International Conference on Machine Learning (ICML), Stockholm, Sweden, 2018. Best Paper Award. [arxiv] [code]

Workshop proceedings

Jane Castleman, Nimra Nadeem, Tanvi Namjoshi, Lydia T. Liu.
Rethinking Math Benchmarks: Implications for AI in Education.
Proceedings of the Innovation and Responsibility in AI-Supported Education Workshop. [paper]
Yuhan Liu, Yuhan Zheng, Siyuan Zhang, Lydia T. Liu.
Evaluating Fairness in Black-box Algorithmic Markets: A Case Study of Ride Sharing in Chicago.
International Conference on Machine Learning (ICML) Workshop on Humans, Algorithmic Decision-Making and Society: Modeling Interactions and Impact. [arxiv]
Benedikt Stroebl, Rajiv Krishna Swamy, Lydia T. Liu.
A Baseline that Matters: Categorical Prioritization as Benchmark for Social Policy Algorithms.
International Conference on Machine Learning (ICML) Workshop on Humans, Algorithmic Decision-Making and Society: Modeling Interactions and Impact. [poster]
Esther Rolf, Max Simchowitz, Sarah Dean, Lydia T. Liu, Daniel Björkegren, Moritz Hardt, Joshua Blumenstock.
Balancing Competing Objectives for Welfare-Aware Machine Learning with Imperfect Data.
NeurIPS Joint Workshop on AI for Social Good, Vancouver, Canada, 2019. Best Paper Award.
Lydia T. Liu, Urun Dogan, and Katja Hofmann.
Decoding multitask DQN in the world of Minecraft.
The 13th European Workshop on Reinforcement Learning, Barcelona, Spain, 2016. Also presented at the 11th Women in Machine Learning Workshop and the Deep Reinforcement Learning Workshop at NeurIPS 2016.

^* ^{^} equal contribution

Technical reports

Social Dynamics of Machine Learning for Decision Making.
PhD Dissertation, University of California Berkeley, Spring 2022.

Blog posts

On the Actionability of Outcome Prediction. Montreal AI Ethics Institute, AI Ethics Brief. Feb 2024.
When bias begets bias: A source of negative feedback loops in AI systems. Microsoft Research Blog. Jan 2020.
Delayed Impact of Fair Machine Learning. Co-authored with Sarah Dean, Esther Rolf, Max Simchowitz, Moritz Hardt. Berkeley AI Research Blog. May 2018.

Selected Talks

Minding the actionability gap of predictions in responsible AI: lessons from education
- TTIC Workshop on Data-Driven Decision Processes: From Theory to Practice, Chicago, IL. Aug 2024.
- Tucker Holland Seminar, Educational Testing Service, Princeton, NJ. May 2024.
- Columbia Business School Workshop on Challenges in Operationalizing Responsible AI, New York, NY. Oct, 2023.
What Really Matters for Fairness in Machine Learning: Delayed Impact and Other Desiderata
- Simons Institute workshop on Societal Considerations and Applications, Berkeley, CA. Nov 2022. [video]
- Joint Statiscal Meetings (JSM) Session on Ethics and Trust in Statistical Learning, Toronto, Canada. Aug 2023.
Social Dynamics of Machine Learning for Decision Making
- BIRS Workshop on New Interactions Between Statistics and Optimization, Banff, Canada. May 2022.
Reimagining the Machine Learning Life Cycle in Education
- BAIR/CPAR/BDD seminar, Berkeley, CA. Feb 2022. [slides]
- Everyday AI Make-a-thon (with Serena Wang), MIT, Cambridge, MA. Mar 2022.
Learning in Decentralized Matching Markets
- Foundations of Data Science Institute (FODSI) Seminar, May 2021. [slides]
- Beneficial AI Seminar, Center for Human-Compatible AI (CHAI), Berkeley, CA. Jun 2021.
The Disparate Equilibria of Algorithmic Decision Making when Individuals Invest Rationally
- University of Bristol Data Science seminar, Bristol, England. (Zoom) Jan 2021.
- Center for Human-Compatible AI (CHAI) seminar, Berkeley, CA. (Zoom) Jul 2020.
- ACM FAccT, Barcelona, Spain. Jan 2020. [slides] [video]
Competing Bandits in Matching Markets
- BAIR/CPAR/BDD seminar, Berkeley, CA, USA. Nov 2019.
- DS102 “Data, Inference, and Decisions” Lecture, Berkeley, CA, USA. Nov 2019. [video]
- Third Annual Berkeley-Stanford Econometrics Jamboree. Nov 2019.
- Simons Institute Program on Online and Matching-Based Market Design, Deep Dive Seminar. Oct 2019.
- EC 2019 Workshop on Learning in Presence of Strategic Behavior, Phoenix, AZ, USA. June 2019. [slides]
The Implicit Fairness Criterion of Unconstrained Learning
- ICML, Long Beach, CA, USA. June 2019. [slides] [video, starts at 34:50]
Delayed Impact of Fair Machine Learning
- Spring Meeting, Young Center for Interdisciplinary Research, Bielefeld, Germany. (Zoom) March 2021.
- MSR AI Seminar, Redmond, WA. November 2018. [video]
- MPI for Software Systems, Saarbrucken, Germany. October 2018.
- MPI for Intelligent Systems, Tuebingen, Germany. October 2018.
- ICML plenary talk, Stockholm, Sweden. July 2018. [slides] [video]
ePCA: High Dimensional Exponential Family PCA
- John Storey’s group. Princeton, NJ. April 2017. [slides]
- Program in Applied and Computational Mathematics Certificate Program Oral Presentations. Princeton, NJ. April 2017. Best Project

Workshops co-organized

BIRS workshop on Bridging Prediction and Intervention Problems in Social Systems, June 3-7, 2024. Banff, AB, Canada.
AAAI 2024 Workshop on AI for Education: Bridging Innovation and Responsibility, Feb 26-27, 2024. Vancouver, BC, Canada.
NeurIPS 2020 Workshop on Consequential Decision Making in Dynamic Environments. Virtual.