Generative Language Models (GLMs) are now ubiquitous in various sectors, including customer service and content creation. Consequently, handling potential harmful content while keeping linguistic diversity and inclusivity has become important. Toxicity scoring systems aim to filter offensive or hurtful language, but often misidentify harmless language as harmful, especially from marginalized communities. This restricts access to relevant information and stifles cultural and linguistic expression. Present moderation methods generally use fixed thresholds for toxicity scoring, leading to inflexible and sometimes biased content filtering.
A team of researchers from Google DeepMind and UC San Diego has proposed dynamic thresholding for toxicity scoring in GLMs. The innovative idea consists of an algorithmic recourse mechanism that allows users to modify toxicity threshold for specific phrases, protecting them against unnecessary exposure to offensive language. Users can define and interact with content within their personal toxicity thresholds and provide feedback to inform future user-specific norms or models for toxicity.
In this system, users can preview content flagged by the initial toxicity assessment of the model. They can decide if this content should bypass automatic filters in future interactions. This enhances user involvement and tailors the GLM’s responses to align more closely with individual and societal norms. The model was tested via a pilot study involving 30 participants, illustrating its effectiveness and usability in real-world conditions.
The dynamic thresholding mechanism proved successful by securing an average System Usability Scale score of 66.8. Participants’ positive feedback, demonstrated the dynamic system’s superiority over the traditional fixed-threshold model. Participants praised the increased control and involvement provided by dynamic thresholding, which allowed for a more customized interaction by enabling modifications to content filtering based on individual user’s personal preferences.
In conclusion, dynamic thresholding for toxicity scoring in GLMs offers exciting prospects for user experience and agency. It marks a significant advancement towards more inclusive and flexible technology that respects the evolving nature of language and the diverse needs of users. However, additional research is needed to fully comprehend the impacts of this method and how it can be optimized for multiple applications. For those interested in further details, the original research paper is available for consultation. The researchers also maintain active social networking sites for updates and discussions.
