MIT and MIT-IBM Watson AI Lab researchers have developed an automated system that trains users to effectively collaborate with artificial intelligence (AI). The system, which is designed to be customized for different tasks, identifies the circumstances under which a user should pay attention to the AI’s recommendations and describes these conditions in natural language. Initially, the system derives the rules for interaction in the context of a specific task. After some training exercises with the AI, the user begins to understand when the system’s advice is most helpful.
Researchers found that in image prediction tasks, the accuracy rate increased by around 5% when AI was used in collaboration with humans following the onboarding procedure. Interestingly, user performance worsened when they were only told when to trust the AI without going through the onboarding process.
The training process was developed to help users understand when AI tools would be beneficial because, unlike other tools, AI tools often do not come with tutorials.
The method is scalable and can be utilized in numerous situations where AI models and humans work together. It can be beneficial in areas like social media content moderation, writing, and programming. The researchers believe that this type of onboarding may be an essential aspect of training for medical professionals in the future.
Nonetheless, the utility of the onboarding method depends on the amount of data available. If there is not sufficient data, the onboarding process may be less effective. Future plans for the researchers include conducting larger studies to evaluate the short- and long-term effects of onboarding, leveraging unlabeled data, and discovering methods to effectually reduce the number of regions without excluding critical examples.
The collaboration hopes to help humans understand when AI is trustworthy, improving human-AI team interactions. For instance, a radiologist might utilize this training method to determine when to trust an AI model’s suggestion regarding a diagnosis, thereby improving her diagnostic accuracy. This innovative method could potentially fill an essential gap in the use and integration of AI in various fields of work.
