Options
Fearful Falcons and Angry Llamas : Emotion Category Annotations of Arguments by Humans and LLMs
Greschner, Lynn; Klinger, Roman (2026): Fearful Falcons and Angry Llamas : Emotion Category Annotations of Arguments by Humans and LLMs, in: Mika Hämäläinen, Emily Öhman, Yuri Bizzoni, u. a. (Hrsg.), Proceedings of the 5th International Conference on Natural Language Processing for Digital Humanities, Bamberg: Otto-Friedrich-Universität, S. 628–646.
Faculty/Chair:
Author:
Title of the compilation:
Proceedings of the 5th International Conference on Natural Language Processing for Digital Humanities
Editors:
Hämäläinen, Mika
Öhman, Emily
Bizzoni, Yuri
Miyagawa, So
Alnajjar, Khalid
Conference:
International Conference on Natural Language Processing for Digital Humanities ; Albuquerque, USA
Publisher Information:
Year of publication:
2026
Pages:
ISBN:
979-8-89176-234-3
Source/Other editions:
Mika Hämäläinen, Emily Öhman, Yuri Bizzoni, u. a. (Hrsg.), Proceedings of the 5th International Conference on Natural Language Processing for Digital Humanities, Stroudsburg, PA: Association for Computational Linguistics (ACL), 2025, S. 628–646, ISBN: 979-8-89176-234-3
Year of first publication:
2025
Language:
English
Abstract:
Arguments evoke emotions, influencing the effect of the argument itself. Not only the emotional intensity but also the category influences the argument’s effects, for instance, the willingness to adapt stances. While binary emotionality has been studied in argumentative texts, there is no work on discrete emotion categories (e.g., ‘anger’) in such data. To fill this gap, we crowdsource subjective annotations of emotion categories in a German argument corpus and evaluate automatic LLM-based labeling methods. Specifically, we compare three prompting strategies (zero-shot, one-shot, chain-of-thought) on three large instruction-tuned language models (Falcon-7b-instruct, Llama-3.1-8B-instruct, GPT-4o-mini). We further vary the definition of the output space to be binary (is there emotionality in the argument?), closed-domain (which emotion from a given label set is in the argument?), or open-domain (which emotion is in the argument?). We find that emotion categories enhance the prediction of emotionality in arguments, emphasizing the need for discrete emotion annotations in arguments. Across all prompt settings and models, automatic predictions show a high recall but low precision for predicting anger and fear, indicating a strong bias toward negative emotions.
Keywords: ; ; ;
argument
emotion
corpus
large language model
Peer Reviewed:
Yes:
International Distribution:
Yes:
Open Access Journal:
Yes:
Type:
Conferenceobject
Activation date:
May 22, 2026
Permalink
https://fis.uni-bamberg.de/handle/uniba/115224