Research conducted by institutions like FAIR, Meta AI, Datashape, and INRIA explores the emergence of Chain-of-Thought (CoT) reasoning in Language Learning Models (LLMs). CoT enhances the capabilities of LLMs, enabling them to perform complex reasoning tasks, even though they are not explicitly designed for it. Even as LLMs are primarily trained for next-token prediction, they respond well when prompted to articulate their thought process in detailed steps. Studies have revealed that LLMs struggle with single-token prediction problems but excel in situations where they can generate a sequence of tokens, which they utilize as a form of computational tape to solve complex reasoning problems.
The researchers have sought to comprehend how CoT reasoning develops in transformers. Iteration heads, a type of specialized attention mechanism crucial for recurrent reasoning, were introduced and their growth and functionality within the network have been monitored. These iteration heads have been shown to be effective in enabling transformers to solve intricate problems via multi-step reasoning by focusing on simple, controlled tasks such as copying and polynomial iterations. Furthermore, these capabilities have been found to be highly transferrable between different tasks, which suggests that transformers could establish internal reasoning circuits, largely influenced by their training data. This suggests an explanation for the impressive CoT capabilities observed in larger models.
The study has spotlighted how transformers, especially in the context of language models, can learn and implement iterative algorithms that involve sequential processing steps. The researchers aim to clarify how transformers use CoT reasoning to efficiently solve tasks like copying problems and polynomial iteration. Various algorithmic representations and synthetic data have been used to investigate the emergence of CoT reasoning mechanisms like iteration heads within transformer architectures. This has facilitated a detailed understanding of how transformers approach iterative tasks, shedding light on their reasoning faculties further than mere token prediction.
The findings have also highlighted how the strategic selection of training data can boost the efficiency of learning in language models and facilitate the transfer of skills. For instance, when a model is trained on an easier task before being tuned to tackle a more challenging one, this leads to improved performance and faster convergence. Researchers discovered that training a model on a polynomial iteration task before switching to the parity problem considerably reduced the time it took to learn the parity task. This reflects the significant role that data selection plays and the function of inductive biases in shaping learning dynamics and influencing model performance.
In conclusion, while this study mainly focuses on controlled scenarios, it suggests that transformers are very likely to develop internal multi-step reasoning circuits that can be applied across different tasks. The research also brings attention to a limitation of transformers concerning the maintenance of internal states, which could potentially affect their applicability to complex algorithms and language modeling.
