Large Language Models (LLMs), significant advancements in the field of artificial intelligence (AI), have been identified as potential carriers of harmful information due to their extensive and varied training data. This information can include instructions on creating biological pathogens, which pose a threat if not adequately managed. Despite efforts to eliminate such details, LLMs can infer harmful facts from scattered hints across the dataset.
To investigate this, researchers from various universities and institutions studied an interesting phenomenon called inductive out-of-context reasoning (OOCR). It is the capacity of an LLM to apply inferred knowledge to new tasks independently of direct in-context learning, extrapolating hidden information from fragmented data pieces.
The study demonstrated that advanced LLMs could perform OOCR in several tasks. One such experiment involved training the LLM on data representing distances between cities, following which the model correctly identified the unfamiliar city and leveraged this understanding in responding to further inquiries about the city.
Further tests revealed that LLMs could identify a biased coin after training on the results of specific coin flips and even construct functions and calculate their inverses with absent explicit examples or explanations.
However, the performance of OOCR was inconsistent when dealing with complex structures or smaller models. This variance emphasizes the difficulties in ensuring reliable conclusions from LLMs.
The contributions of the researchers include the introduction of OOCR as a non-transparent method for LLMs to learn and reason. They created a comprehensive test suite to gauge the inductive OOCR capabilities of LLMs. The LLMs tested, GPT-3.5, and GPT-4, successfully completed all five tasks, with GPT-4 exhibiting superior OOCR capabilities.
These findings raise concerns about AI safety and potential deception from misaligned models as they can learn and use knowledge in ways hard for humans to monitor. The inferred information isn’t explicitly expressed, making it troublesome to control.
Overall, the capabilities, limitations, and implications of OOCR in LLMs offer a glimpse into the challenges and opportunities surrounding the future of AI safety. It’s crucial to acknowledge the potential risks and develop strategies to ensure the ethical and safe development of AI technologies.
