Researchers from IBM Research have developed a new architecture, dubbed Alignment Studio, which enables developers to mould large language models (LLMs) to fit specific societal norms, laws, values and regulations. The system is designed to mitigate ongoing challenges in the artificial intelligence (AI) sector surrounding issues such as hate speech and inappropriate language.
While efforts have been made to address these problems within the industry, an overarching solution has proven elusive due to the intricacy and diversity of contextual variables. According to IBM Research, the Alignment Studio provides a remedy to this by allowing developers to tailor AI behavior based on unique needs and requirements of specific applications, industries, and organizations.
The system is made up of three components: Framers, Instructors, and Auditors. Framers are responsible for identifying vital information from documents related to a certain field, creating data for model alignment, and constructing domain-specific ontologies for comprehensive coverage and clarification. Instructors, on the other hand, instill preferred behaviors and values into the LLMs through both supervised and reinforcement learning fine-tuning. Lastly, auditors ensure that models perform optimally through regular evaluation, which takes place at various stages.
The researchers’ report provided a practical demonstration of the Alignment Studio’s capabilities by aligning an internal enterprise bot with IBM’s business conduct guidelines. Doing so indicated that the tool can be highly effective in modifying AI behavior to adhere to certain organizational requirements. The report further suggested that the Architecture Studio could be deployed to align with a wide variety of contextual regulations in an efficient manner.
One example of this was IBM’s own Granite model, which the researchers aligned with company conduct guidelines using seed instruction data. The results showed that the aligned models were more faithful and relevant to policy guidelines than unaligned ones. Looking forward, the researchers plan to continue refining the alignment studio by developing semi-automated methods to identify misaligned responses, thereby enhancing its application and efficacy. They anticipate that these advancements will also enable alignment with an even broader range of value specifications.
This achievement is seen as a breakthrough in AI capabilities, opening new avenues for mitigating common issues and creating models that align better with human values and behaviors. With societal norms and regulations continuing to evolve, the ability to align AI with cultural and legal contexts will become increasingly important.
