Historically, thinking around decision-making has dichotomized habitual and goal-oriented behavior, treating them as independent activities controlled by distinct neural systems. Habitual behaviors, being automatic, are fast and model-free while goal-oriented behaviors, requiring deliberate action, are slower, model-based but demanding computationally. Microsoft researchers, however, have proposed an innovative Bayesian behavior framework that attempts to synergize these two kinds of behavior. This groundbreaking approach employs variational Bayesian techniques to merge these behaviors using an element referred to as the Bayesian intention variable.
The Bayesian intention variable stands as a dynamic intention that can adapt based on sensory cues, typical of habitual actions, and specific goals, characteristic to goal-driven behavior. As such, the concept facilitates seamless transition and interaction between these two behaviors. The framework essentially works by minimizing the divergence between habitual and goal-oriented intentions. It combines the two kinds of intentions applying inverse variance-weighted averaging and creating a unified intention. This process allows agents the benefit of habitual behaviors’ efficiency and the flexibility of proactive goal-oriented planning.
The functionality of the framework was put to the test through vision-based sensorimotor tasks in a T-maze environment. Three significant outcomes were observed: First, there was a natural transition from slower, goal-oriented actions to swifter habitual behaviors through ongoing trials, which decidedly decreased the computational demands of the goal-oriented processes. Second, despite changes in reward values (Reward Devaluation), agents remained consistent in their habitual behaviors, mirroring real-world behavioral patterns observed in psychology. Lastly, the framework’s ability to generalize behaviors by leveraging pre-developed habitual skills was exhibited when agents efficiently achieved new goals devoid of additional training.
In a nutshell, Microsoft’s research offers a remarkable breakthrough in understanding and modeling behavior by merging habitual and goal-oriented actions in a Bayesian framework. By bridging the gap between these two behaviors, this novel approach magnifies the effectiveness and adaptability of decision-making processes in both natural and artificial agents. This pioneering research is a significant contribution to optimizing decision-making by unifying previously distinct behavioral practices. The research project and its findings, which hold the potential for wide-ranging applications, are attributed to the multi-disciplinary team of researchers at Microsoft.
