Language models have undergone significant developments in recent years which has revolutionized artificial intelligence (AI). Large language models (LLMs) are responsible for the creation of language agents capable of autonomously solving complex tasks. However, the development of these agents involves challenges that limit their adaptability, robustness, and versatility. Manual task decomposition into LLM pipelines is a complicated and labor-intensive process that restricts the optimization of language agents. Hence, there is a need to transition from an engineering approach to a more data-centric learning paradigm.
Attempts to optimize language agents have been made through automated prompt engineering and agent optimization methods. However, they are divided into two categories: prompt-based and search-based. The former optimizes specific components within an agent pipeline while the latter finds optimal prompts or nodes in a combinatory space. Both methods have limitations, including difficulty with real-world tasks and a tendency to settle on local optima rather than optimal solutions.
A new training framework inspired by neural network learning, called the Agent Symbolic Learning Framework, has been introduced by researchers at AIWaves Inc. This innovative approach maps agent pipelines to computational graphs, nodes to layers, and prompts and tools to weights. The analogy is between language agents and neural networks using a method akin to backpropagation. An agent’s performance is assessed using “language loss,” and “language gradients” are calculated through back-propagation. These gradients facilitate the optimization of all symbolic components, sidestepping local optima, and making effective learning for complex tasks possible.
This unique approach involves key aspects such as Agent Pipeline, Nodes, Trajectory, Language Loss, Language Gradient, symbolic optimizers (PromptOptimizer, ToolOptimizer, PipelineOptimizer), and batched training. These features jointly optimize all symbolic components within an agent system, which enhances the efficiency of language agents in handling real-world complex tasks and allows them to evolve post-deployment.
Significant performance improvements were demonstrated by this framework across various benchmark tests, as well as on tasks related to software development and creative writing. It consistently outperforms other methods and is robust and effective in optimizing language agents for tasks where traditional methods struggle.
This new method is a significant step towards artificial general intelligence, as it aims to transition from model-centric to data-centric research. By making the code and prompts open-source, it can potentially accelerate progress in this field and revolutionize language agent development application.
The research for this project has been credited to the team at AIWaves Inc. and has been posted on the MarkTechPost website. The community is encouraged to further engage with this work via the 46k+ ML SubReddit, LinkedIn Group, Twitter, and upcoming AI Webinars. This research intends to boost advancements in the field of artificial intelligence and promote efforts towards developing AI frameworks for optimizing language agents.
