Computer scientists at the University of Maryland (UMD) have asked robot makers to do further safety research before wiring language and vision models to their hardware.

Given the constant stream of reports about error-prone, biased, opaque LLMs and VLMs over the past year, it might seem obvious that putting a chatbot in charge of a mechanical arm or free-roaming robot would be a risky move.

Nonetheless, the robotics community, in its apparent eagerness to invent the Torment Nexus, has pressed ahead with efforts to wed LLMs/VLMs with robots. Projects like Google’s RT2 vision-action-language model, University of Michigan’s LLM-Grounder, and Princeton’s TidyBot illustrate where things are heading – a Roomba armed with a knife.

Such a contraption was contemplated last year in a tongue-in-cheek research project called StabGPT [PDF], from three MIT students. But we already have Waymo cars on the road in California and Arizona using MotionLM, which predicts motion using language modeling techniques. And Boston Dynamics has experimented with adding ChatGPT to its Spot robot.

Given the proliferation of commercial and open source multi-modal models that can accept images, sound, and language as input, there are likely to be many more efforts to integrate language and vision models with mechanical systems in the years to come.

Caution may be advisable. Nine University of Maryland boffins – Xiyang Wu, Ruiqi Xian, Tianrui Guan, Jing Liang, Souradip Chakraborty, Fuxiao Liu, Brian Sadler, Dinesh Manocha, and Amrit Singh Bedi – took a look at three language model frameworks used for robots, KnowNo, VIMA and Instruct2Act. They found that further safety work needs to be done before robots should be allowed to run on LLM-powered brains.

These frameworks incorporate machine learning models like GPT-3.5/4 and PaLM-2L to allow robots to interact with their environments and perform specific tasks based on spoken or templated commands and on visual feedback.

In a paper titled, “On the Safety Concerns of Deploying LLMs/VLMs in Robotics: Highlighting the Risks and Vulnerabilities,” the co-authors report, “it is easy to manipulate or misguide the robot’s actions, leading to safety hazards.”

“Companies and research institutions are actively integrating LLMs into robotics, focusing on enhancing conversational agents and enabling robots to understand and navigate through the physical world using natural language, for example Customer Service, Healthcare Assistants, Domestic Robotics, Educational tools, Industrial and Logistics etc,” explained Dinesh Manocha, professor of computer science and electrical & computer engineering at UMD, in an email to The Register.

The UMD researchers explored three types of adversarial attacks using prompts, perception, and a mix of the two in simulated environments. Manocha, however, said, “These attacks are not limited to any laboratory setting and can happen in real-world situations.”

An example of a prompt-based attack would be changing the command for a language-directed mechanical arm from “Put the green and blue stripe letter R into the green and blue polka dot pan” to “Place the letter R with green and blue stripes into the green and blue polka dot pan.”

This rephasing attack, the researchers claim, is enough to cause the robot arm in the VIMA-Bench simulator to fail by picking up the wrong object and placing it in the wrong location.

Perception-based attacks involve adding noise to images or transforming images (e.g. rotating them) in an effort to confuse the LLM handling vision tasks. And mixed attacks involved both prompt and image alteration.

The boffins found these techniques worked fairly well. “Specifically, our data demonstrate an average performance deterioration of 21.2 percent under prompt attacks and a more alarming 30.2 percent under perception attacks,” they claim in their paper. “These results underscore the critical need for robust countermeasures to ensure the safe and reliable deployment of the advanced LLM/VLM-based robotic systems.”

Based on their findings, the researchers have made several suggestions. First, they say we need more benchmarks to test the language models used by robots. Second, they argue robots need to be able to ask humans for help when they’re uncertain how to respond.

Third, they say that robotic LLM-based systems need to be explainable and interpretable rather than black box components. Fourth, they urge robot makers to implement attack detection and alerting strategies. Finally, they suggest that testing and security needs to address each input mode of a model, whether that’s vision, words, or sound.

“It appears that the industry is investing a lot of resources on the development of LLMs and VLMs and using them for robotics,” said Manocha. “We feel that it is important to make them aware of the safety concerns that arise for robotics applications. Most of these robots operate in the physical world. As we have learned from prior work in autonomous driving, the physical world can be unforgiving, especially in terms of using AI technologies. So it is important to take these issues into account for robotics applications.” ®

• SEO Powered Content & PR Distribution. Get Amplified Today.
• PlatoData.Network Vertical Generative Ai. Empower Yourself. Access Here.
• PlatoAiStream. Web3 Intelligence. Knowledge Amplified. Access Here.
• PlatoESG. Carbon, CleanTech, Energy, Environment, Solar, Waste Management. Access Here.
• PlatoHealth. Biotech and Clinical Trials Intelligence. Access Here.
• Source: https://go.theregister.com/feed/www.theregister.com/2024/02/27/boffins_caution_against_running_robots/