Large Language Models (LLMs) exhibit significant “dialect prejudice,” particularly against African American English (AAE), according to recent research by the Allen Institute for AI and Stanford University. The comprehensive study analysed five popular LLMs including OpenAI’s GPT-2, GPT-3.5, GPT-4 models, RoBERTa and T5. Utilising a ‘Matched Guise Probing’ methodology, the researchers entered similar AAE and Standard American English (SAE) sentences, asking the LLMs to assess the character of the hypothetical speaker.
The team found that all five LLMs displayed a clear bias, resulting in prejudiced decisions against AAE speakers. Specifically, the LLMs associated AAE prompts with negative adjectives such as “dirty”, “stupid”, “rude”, “ignorant”, and “lazy”. Moreover, this prejudice extends to areas such as employment and criminal justice. For example, the LLMs were more likely to assign lower-tier jobs to AAE speakers, assigning roles like cooks, security guards, soldiers, and cleaners, regardless of additional input clarifying that the speakers were not African American.
Furthermore, these models demonstrated bias when asked to comment on a speaker’s potential criminality in a hypothetical murder trial, using AAE and SAE inputs as pieces of evidence. The LLMs were found to have a higher penchant for determining guilty verdicts when processing AAE text inputs (27.7 percent) versus SAE (22.8 percent).
Despite the inherent bias present within the LLMs, attempts to correct these prejudices have been largely ineffectual. Increasing the sample size did facilitate a better understanding of AAE for the models, but did not eradicate linguistic prejudice. Additionally, structured human feedback training was found to only mask the racial stereotypes rather than reduce their intrinsic impact, leaving latent forms of racism untouched.
This research provides the first empirical evidence of covert racism based on language and dialect within LLMs. Cognitive psychologist and AI philosopher, Gary Marcus, suggests the LLM developers should withdraw their systems until these prejudices can be effectively addressed. The implications of the mass use of biased LLMs in operations such as employment are of significant concern, especially for AAE speakers.
