Industry Briefing
A single destination for timely, editor-curated robotics news from around the world.
Voice AI Systems Are Vulnerable to Hidden Audio Attacks
Researchers are set to unveil alarming findings regarding AI-powered voice and audio tools at the IEEE Symposium on Security and Privacy in San Francisco next week. The study reveals that modified audio clips, imperceptible to human ears, can manipulate large audio-language models (LALMs) to execute unauthorized commands with a success rate between 79 and 96 percent. This vulnerability allows attackers to control devices, conduct sensitive web searches, and even send emails containing user data without the user's knowledge. The research, led by Meng Chen, a Ph.D. student at Zhejiang University in China, demonstrates that these attacks can be executed in real-time and do not require the attacker to have full control over the user's instructions. Instead, adversarial audio can be embedded in various media, such as online videos or voice notes, making it a pervasive threat. The technique, dubbed AudioHijack, exploits a critical flaw in LALM design, allowing malicious instructions to be hidden within manipulated audio clips. The researchers tested their method on 13 leading open models, including those from Microsoft and Mistral, and found that their attacks could be adapted to commercial models as well. While Microsoft acknowledged the importance of the research in enhancing model resilience, Mistral did not respond to inquiries. The study highlights the challenges in defending against such attacks, as common defenses have proven largely ineffective, underscoring the urgent need for improved security measures in AI audio technologies.
Hacking Digital-audio Adversarial-attacks Open-source-software Cybersecurity
DJI Releases Findings of the Most Comprehensive Independent Security Assessment of Its Drone Systems to Date
A recent five-month adversarial testing conducted by the U.S. cybersecurity firm OnDefend has revealed no critical, high, or medium-risk vulnerabilities in the DJI Air 3S and Matrice 4E drones. The testing, aimed at assessing the cybersecurity resilience of these popular drone models, took place in various controlled environments to simulate potential cyber threats. The results, released in October 2023, underscore the effectiveness of DJI's security measures in protecting against cyberattacks. This thorough evaluation is part of ongoing efforts to ensure the safety and reliability of drone technology amid increasing concerns over cybersecurity in the aviation sector.
Claude Mythos Preview Requires New Ways to Keep Code Secure
Malicious actors are increasingly leveraging generative AI to conduct cyberattacks, employing AI-generated deepfakes for scams, AI-assisted malware, and chatbots for phishing campaigns. In early April, Anthropic’s Frontier Red Team revealed that its Claude Mythos Preview model identified thousands of critical vulnerabilities across major operating systems and web browsers, despite not being specifically trained for this purpose. This prompted the launch of Project Glasswing, a collaborative initiative with tech giants like Amazon Web Services, Apple, Google, Microsoft, and Nvidia, aimed at using Mythos Preview to enhance software security. While generative AI demonstrates remarkable capabilities in identifying code vulnerabilities, experts warn that these same abilities can be exploited by cybercriminals. Jeremy Katz, vice president of code security at Sonar, noted that AI can effectively pinpoint security flaws within extensive codebases. However, the technology is not without its challenges, including the potential for false positives, which complicates the process for open-source maintainers. To mitigate these issues, cybersecurity professionals advocate for a balanced approach that incorporates human oversight in the verification of AI findings. Techniques such as adversarial self-review and dynamic threat modeling are suggested to enhance the reliability of AI tools. Experts emphasize the importance of integrating security measures earlier in the software development lifecycle and providing ongoing training for developers to preemptively address vulnerabilities. As AI continues to evolve in its ability to detect and classify security weaknesses, the focus will shift towards effectively remediating these vulnerabilities at scale.
Anthropic Coding Artificial-intelligenceRobotics needs a service framework.
RSF defines a common language for robot service capability, lifecycle operations, certification pathways, and service-provider networks.
