Can LLM Agents Really Simulate Clients in Counseling?

  • Yu, Seonhui; 
  • Kim, Dongmyung; 
  • Roh, Youjin; 
  • Kwon, Kyungmin; 
  • Oh, Hayoung
Citations

SCOPUS

0

초록

Recent studies have increasingly examined the use of large language models (LLMs) to simulate clients in counseling sessions, yet many report that LLM-generated responses tend to be excessively positive, reducing the realism of therapeutic interactions. This study conducts a quantitative analysis to verify whether such positivity bias consistently appears across a range of closed-source and open-source models and to evaluate strategies intended to mitigate it. To accomplish this, we developed a dataset and an evaluation model that score client utterances along sentiment and stance dimensions grounded in clinical expert criteria. Using multiple LLMs combined with various bias mitigation methods, we generated artificial counseling dialogues and compared them with real client data. Experimental findings show that most LLM clients exhibit significant positivity bias, revealing clear statistical differences from real counseling conversations. Although the proposed mitigation strategies alleviated this bias to some degree, they did not fully close the gap. The findings demonstrate the presence of a quantifiable gap between LLM-based counseling simulations and real client responses, suggesting the development of standardized evaluation frameworks, session-context-adaptive bias mitigation, and persona design methods to bridge this gap. All code and generated data used in this study are publicly available on our GitHub repository. 1.

키워드

bias mitigation; counseling simulation; LLM agents; positivity bias
제목
Can LLM Agents Really Simulate Clients in Counseling?
저자
Yu, Seonhui; Kim, Dongmyung; Roh, Youjin; Kwon, Kyungmin; Oh, Hayoung
DOI
10.1145/3748522.3779828
발행일
2026
유형
Conference Paper
저널명
Proceedings of the ACM Symposium on Applied Computing
페이지
865 ~ 874