MIT Schwarzman College of Computing

Large language models powering AI chatbots possess the potential for generating harmful content due to their exposure to countless websites, putting users at risk if the AI generates illegal activities description, illicit instructions, or personal information leakage. To mitigate such threats, AI-developing companies use a procedure known as red-teaming, where human testers compose prompts aimed…

Artificial intelligence (AI) chatbots like ChatGPT, capable of generating computer code, summarizing articles, and potentially even providing instructions for dangerous or illegal activities, pose unique safety challenges. To mitigate this risk, companies use a safeguarding process known as red-teaming, where human testers attempt to prompt inappropriate or unsafe responses from AI models. This process is…

Companies that build large language models, like those used in AI chatbots, routinely safeguard their systems using a process known as red-teaming. This involves human testers generating prompts designed to trigger unsafe or toxic responses from the bot, thus enabling creators to understand potential weaknesses and vulnerabilities. Despite the merits of this procedure, it often…

Artificial Intelligence chatbots have the capacity to construct helpful code, summarize articles, and even create more hazardous content. To prevent safety violations like these, companies employed a procedure known as "red-teaming" in which human testers crafted prompts intended to elicit unsafe responses from chatbots, which were then taught to avoid these inputs. However, this required…

Researchers from the Improbable AI Lab at MIT and the MIT-IBM Watson AI Lab have developed a new technique to improve "red-teaming," a process of safeguarding large language models, such as AI chatbot, through the use of machine learning. The new approach focuses on the automatic generation of diverse prompts that result in undesirable responses…