Publications

Designing Staged Evaluation Workflows for LLMs: Integrating Domain Experts, Lay Users, and Model-Generated Evaluation Criteria
Annalisa Szymanski, Simret Araya Gebreegziabher, Oghenemaro Anuyah, Ronald A Metoyer, and Toby Jia-Jun Li
Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI 2026)
[Paper PDF]

Balancing Goals, Health, and Cost: A Food Information System for Managing Complex Choices and Fostering Sustained Food Agency
Annalisa Szymanski*, Jeongwon Jo*, Michelle Sawwan, Heather A Eicher-Miller, Ann-Marie Conrado, Danielle Wood, Tawanna Dillahunt, and Ronald A Metoyer *Equal Contribution
Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI 2026)
[Paper PDF]

Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study
Annalisa Szymanski, Oghenemaro Anuyah, Toby Jia-Jun Li, and Ronald A Metoyer
31st International Conference on Intelligent User Interfaces (IUI 2026)
[Paper PDF]

Evaluation Workflows for Large Language Models (LLMs) that Integrate Domain Expertise for Complex Knowledge Tasks
Annalisa Szymanski
30th International Conference on Intelligent User Interfaces Companion (IUI Companion 2025)
[Paper PDF]

Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks
Annalisa Szymanski*, Noah Ziems*, Heather A Eicher-Miller, Toby Jia-Jun Li, Meng Jiang, and Ronald A Metoyer *Equal Contribution
30th International Conference on Intelligent User Interfaces (IUI 2025)
[Paper PDF]

Integrating Expertise in LLMs: Crafting a Customized Nutrition Assistant with Refined Template Instructions
Annalisa Szymanski, Brianna L Wimer, Oghenemaro Anuyah, Heather A Eicher-Miller, and Ronald A Metoyer
Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI 2024)
[Paper PDF]

Beyond Static Labels: Unpacking Nutrition Comprehension in the Digital Age
Brianna L Wimer, Annalisa Szymanski, and Ronald A. Metoyer
Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI 2024)
[Paper PDF]

A Community Focused Approach Towards Making Healthy and Affordable Daily Diet Recommendations
Joe Germino*, Annalisa Szymanski*, Heather A. Eicher-Miller, Ronald A. Metoyer, and Nitesh V. Chawla *Equal Contribution
Frontiers in Big Data Volume 6 (Frontiers 2023)
[Paper PDF]