Paper page – ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind

Large language models (LLMs) have shown promising potential in persuasion,
but existing works on training LLM persuaders are still preliminary. Notably,
while humans are skilled in modeling their opponent’s thoughts and opinions
proactively and dynamically, current LLMs struggle with such Theory of Mind
(ToM) reasoning, resulting in limited diversity and opponent awareness. To
address this limitation, we introduce Theory of Mind Augmented Persuader
(ToMAP), a novel approach for building more flexible persuader agents by
incorporating two theory of mind modules that enhance the persuader’s awareness
and analysis of the opponent’s mental state. Specifically, we begin by
prompting the persuader to consider possible objections to the target central
claim, and then use a text encoder paired with a trained MLP classifier to
predict the opponent’s current stance on these counterclaims. Our carefully
designed reinforcement learning schema enables the persuader learns how to
analyze opponent-related information and utilize it to generate more effective
arguments. Experiments show that the ToMAP persuader, while containing only 3B
parameters, outperforms much larger baselines, like GPT-4o, with a relative
gain of 39.4% across multiple persuadee models and diverse corpora. Notably,
ToMAP exhibits complex reasoning chains and reduced repetition during training,
which leads to more diverse and effective arguments. The opponent-aware feature
of ToMAP also makes it suitable for long conversations and enables it to employ
more logical and opponent-aware strategies. These results underscore our
method’s effectiveness and highlight its potential for developing more
persuasive language agents. Code is available at:
https://github.com/ulab-uiuc/ToMAP.