School of Engineering

To counter unsafe responses from chatbots, companies often use a process called red-teaming, in which human testers write prompts designed to elicit such responses so the artificial intelligence (AI) can be trained to avoid them. However, since it is impossible for human testers to cover every potential toxic prompt, MIT researchers developed a technique utilizing…

Large language models powering AI chatbots possess the potential for generating harmful content due to their exposure to countless websites, putting users at risk if the AI generates illegal activities description, illicit instructions, or personal information leakage. To mitigate such threats, AI-developing companies use a procedure known as red-teaming, where human testers compose prompts aimed…

Artificial intelligence (AI) chatbots like ChatGPT, capable of generating computer code, summarizing articles, and potentially even providing instructions for dangerous or illegal activities, pose unique safety challenges. To mitigate this risk, companies use a safeguarding process known as red-teaming, where human testers attempt to prompt inappropriate or unsafe responses from AI models. This process is…

Companies that build large language models, like those used in AI chatbots, routinely safeguard their systems using a process known as red-teaming. This involves human testers generating prompts designed to trigger unsafe or toxic responses from the bot, thus enabling creators to understand potential weaknesses and vulnerabilities. Despite the merits of this procedure, it often…

Artificial Intelligence chatbots have the capacity to construct helpful code, summarize articles, and even create more hazardous content. To prevent safety violations like these, companies employed a procedure known as "red-teaming" in which human testers crafted prompts intended to elicit unsafe responses from chatbots, which were then taught to avoid these inputs. However, this required…