Holy $#!t: Are popular toxicity models simply profanity detectors?

The text highlights the challenges faced by AI models in accurately identifying toxic language, particularly when it involves profanity used in a positive context. Despite advancements in natural language processing (NLP) technologies like contextual word embeddings and transformers, current models often mislabel enthusiastic or supportive messages containing profanity as toxic, as demonstrated by Google's Perspective API. The misclassification issue largely stems from poor training datasets and non-native labelers who fail to grasp the nuances of language. To address this, a benchmark study was conducted using examples of both toxic and non-toxic profanity, revealing that the Perspective API scored a majority of non-toxic examples as highly toxic. The article emphasizes the need for improved data labeling and the importance of human judgment in evaluating language, acknowledging the potential of AI tools while recognizing their limitations in real-world applications.