Artificial intelligence (AI) is evolving beyond chatbots and proving beneficial in performing intricate and autonomous tasks. However, their limited supervision raises concerns about safety and their potential to cause harm. A team composed of researchers from the Quebec AI Institute, Harvard University, the Cooperative AI Foundation, University of Cambridge, and other prominent institutions have identified risks and proposed safety countermeasures.
AI, previously perceived as a response generator agent like ChatGPT, has now evolved to an autonomous entity that carries out tasks to achieve a predetermined goal. For instance, the Rabbit R1 device acts as an AI agent to surf the web and book flights. However, these autonomous AI agents operate with minimal supervision which could pose serious safety risks.
The researchers identified five potential risks of AI agents with inadequate monitoring. They include malicious use by a low-skilled actor, overreliance and disempowerment in high-stake situations, delayed and diversely spread impacts of bad decisions, hazards occurring from multiple agents interaction, and the problematic detection of harmful behavior in sub-agents created by an agent.
To address these risks, researchers proposed three regulatory measures to enhance safety and visibility. They recommended implementing agent identifiers allowing an agent to identify itself, thus making the interaction manageable and linking actions to a specific agent and its creator. Real-time monitoring is proposed to flag rule violations or detect sub-agent generation. Lastly, creating activity logs would help seize damages caused by the agent, thereby allowing researchers to understand how to rectify the issue.
However, the challenge lies in incorporating these safety measures without violating privacy laws. It has been noted that to manage AI risks, political will, sociotechnical infrastructure, and public influence are necessary. Therefore, improving AI agent operation visibility is key to a safer AI environment. Researchers emphasize establishing governance framework, and holding key stakeholders accountable, thus gradually shifting from app-based interactions.
