返回Google圖書搜尋

Chapter An experimental annotation task to investigate annotators' subjectivity in a Misogyny dataset

Alice TontodimammaElisa IgnazziStefano AnzaniMarco StranisciValerio BasileLara FONTANELLA

出版	Firenze University Press, Genova University Press, 2023
ISBN	9791221501063
URL	http://books.google.com.hk/books?id=uqFa0AEACAAJ&hl=&source=gbs_api

註釋In recent years, hatred directed against women has spread exponentially, especially in online social media. Although this alarming phenomenon has given rise to many studies both from the viewpoint of computational linguistics and from that of machine learning, less effort has been devoted to analysing whether models for the detection of misogyny are affected by bias. An emerging topic that challenges traditional approaches for the creation of corpora is the presence of social bias in natural language processing (NLP). Many NLP tasks are subjective, in the sense that a variety of valid beliefs exist about what the correct data labels should be; some tasks, for example misogyny detection, are highly subjective, as different people have very different views about what should or should not be labelled as misogynous. An increasing number of scholars have proposed strategies for assessing the subjectivity of annotators, in order to reduce bias both in computational resources and in NLP models. In this work, we present two corpora: a corpus of messages posted on Twitter after the liberation of Silvia Romano on the 9th of May, 2020 and corpus of comments constructed starting from posts on Facebook that contained misogyny, developed through an experimental annotation task, to explore annotators' subjectivity. For a given comment, the annotation procedure consists in selecting one or more chunk from each text that is regarded as misogynistic and establishing whether a gender stereotype is present. Each comment is annotated by at least three annotators in order to better analyse their subjectivity. The annotation process was carried by trainees who are engaged in an internship program. We propose a qualitative-quantitative analysis of the resulting corpus, which may include non-harmonised annotations.