Designing computation workflows for AI applications faces complexities, requiring the management of various parameters such as prompts and machine learning hyperparameters. Improvements made post-deployment are often manual, making the technology harder to update. Traditional optimization methods like Bayesian Optimization and Reinforcement Learning often call for greater efficiency due to the intricate nature of these systems. In light of these challenges, researchers from Microsoft Research and Stanford University developed a framework titled Trace, intending to automate the design and updating of AI systems.
Trace looks at the computational workflow as a graph, similar to neural networks and optimizes various parameters using Optimization with Trace Oracle (OPTO). The system efficiently converts workflows into OPTO instances, allowing a general-purpose optimizer, OptoPrime, to iteratively update parameters according to execution traces and feedback.
Current frameworks like LangChain, Semantic Kernels, AutoGen, and DSPy can compose and optimize computational workflows, but they mainly use scalar feedback and black-box search techniques. In comparison, Trace employs execution tracing, a more generalized computational graph to accommodate different kinds of workflows. The OPTO framework in Trace supports the joint optimization of prompts, hyperparameters, and codes, adapting dynamically to changes in the workflow structure. It extends AutoDiff principles to non-differentiable workflows, enabling efficient self-adapting agents, and improves the general-purpose optimization for various applications.
The OptoPrime optimization algorithm, which is LLM-based, was designed specifically for OPTO problems. It uses the coding and debugging abilities of LLMs to manage execution trace subgraphs. OptoPrime includes a memory module to track parameters and feedback pairs, a feature that adds robustness to the system. Experiments have shown OptoPrime’s effectiveness in numerical optimization, traffic control, prompt optimization, and long-horizon robot control tasks, outperforming other optimizers, specifically when utilizing execution trace information and memory.
Trace transforms computational workflow optimization issues into OPTO problems, utilizing the OPTO optimizer, OptoPrime. This signifies a step toward the optimization paradigm’s revolutionary trajectory with numerous future directions. Enhancements in LLM reasoning, including the Chain of Thought, Few-shot Prompting, Tool Use, and Multi-agent Workflows, could optimize OPTO optimizers. A hybrid workflow encompassing LLMs and search algorithms could result in a general-purpose OPTO optimizer. Propagator specialization for specific computations could also enhance efficiency.
