The Amazon EU Design and Construction (D&C) team has developed an artificial intelligence (AI) powered solution that improves the accuracy and efficiency of the construction process of Amazon warehouses. This solution involves a language learning model (LLM) which allows engineers to retrieve relevant information from a large volume of unorganized documents, improving the quality and efficiency of their construction projects. The D&C team tested this technology in a pilot project involving Amazon engineers, who provided feedback on its performance.
Analyzing this information highlighted certain limitations including inaccuracies and hallucinations, prompting the team to use reinforcement learning to train the model. A second AI model was employed to generate feedback scores for additional training data. The improvements led to an increase in the quality of the bot response. The model uses Amazon’s SageMaker JumpStart for model deployment, fine-tuning, and reinforcement learning.
The pilot project was important in gathering specific feedback from engineers. They were able to report on five different satisfaction levels and provide responses to questions. The majority of feedback was positive, with scope for further improvement identified.
The model was improved thanks to a three-step process: supervised fine-tuning using labelled data, collection of user feedback to further refine the model, and fine-tuning using reinforcement learning from human feedback. This allowed for human feedback to be incorporated into the rewards function, aligning the model more closely with human goals.
Implementing this AI-powered solution reduced workload for subject matter experts by 80%, by providing automatic AI scores which can be filtered, sorted, and grouped. Duplicating this process could yield benefits for other large organizations as it offers the capacity to improve bot response quality through ongoing reinforcement learning, while minimizing the effort required from subject matter experts.
The Amazon D&C team will scale up the solution by connecting it with Amazon Engineering’s data infrastructure, and design a framework for automating the learning process with a human in the loop. They will also continue to refine the AI feedback quality by tuning the prompt template.
The authors recommend experimenting with this model for organizations wanting to introduce a similar system. With user feedback and reinforcement learning processes, efficiency and quality of construction projects can be greatly improved.
