US Models Outperform China's Kimi K3 in Cybersecurity Tests with 76% vs 32% Scores
- Published
- Source
- InterestingEngineering.com
A recent evaluation by the UK Artificial Intelligence Security Institute and the US Center for AI Standards and Innovation revealed that Moonshot AI's Kimi K3 scored 32.2% in offensive cybersecurity capabilities, significantly lower than the average score of 76.2% achieved by leading US models. This assessment comes amid concerns in Washington about China's advancements in AI following the introduction of Kimi K3, touted as its most powerful large language model.
The findings are crucial as they highlight the performance gap between US and Chinese AI models in cybersecurity, particularly in developing exploits for software vulnerabilities. Kimi K3's performance was tested using ExploitBench, a benchmark from Carnegie Mellon University, where it failed to achieve arbitrary code execution on any of the 41 tasks, while top US models succeeded in 20 out of 41 tasks on average.
Looking ahead, the report indicates that while Kimi K3 demonstrated some capability in autonomous cyber operations, it still lags behind US counterparts. The evaluation was preliminary, and further assessments across a broader range of tasks may provide more insights into Kimi K3's capabilities and limitations in cybersecurity applications. No further timeline was disclosed at the time of publication.
Original report
US models beat China’s Kimi K3 with a 76% score over 32% in cyber benchmarks, tests show
InterestingEngineering.com · Maria MocerinoThis briefing is an independently written summary based on publicly available reporting and is provided for industry information and news discovery. The original report and source publication are credited and linked where applicable. RobotToday does not claim ownership of third-party source material.
Rights concerns? Contact [email protected] with the relevant URL and details. We will review the matter and take appropriate action where warranted.
More related news
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
Related Suppliers
Fireworks AI
United States
Redwood City AI infra startup (2022) serving open-source LLMs like Kimi K3 via API, used by Cursor, Vercel and Notion.
Ohio State University Center for Design and Manufacturing Excellence (CDME)
United States
Ohio State applied R&D center translating new technologies into market-ready products across additive manufacturing, robotics and AI systems.
Fraunhofer Institute for Manufacturing Engineering and Automation IPA
Germany
Stuttgart applied-research institute, est. 1959, that developed the Care-O-bot service robots and runs an independent industrial benchmark for humanoids.
Bifrost AI
United States
Bifrost AI builds simulation infrastructure for physical AI: the Manifold robot-policy evaluation platform and the Stardust synthetic 3D sensor data platform.
CosmicBrain AI
Physical AI firm scoping supervised humanoid pilots for data-center rack inspection and parts runs, and publishing the Cosmic 0.5 human-to-humanoid skill stack.
ALIAS ROBOTICS
Spain
Vitoria-Gasteiz, Basque Country robot cybersecurity firm; develops Robot Immune System, alias LLM family, and NATO-grade Alurity toolbox.
Custos Robotics ApS
Denmark
Danish defense-focused robotics subsidiary of Capra Robotics, developing autonomous mobile robots for mine clearance, UXO removal, and security operations.
Beijing Institute for General Artificial Intelligence 北京通用人工智能研究院
China
Beijing non-profit AGI institute led by Zhu Songchun whose embodied-AI stack includes the TongBrain robot brain and the open-source TongSIM simulator.
Shanghai Artificial Intelligence Laboratory 上海人工智能实验室
China
Shanghai-based national AI research institute founded 2020; develops the InternLM model family and the Intern-Robotics embodied-AI engine.
Regulus Cyber
Israel
Israeli GNSS cybersecurity firm (2016, Haifa); Pyramid software blocks GPS spoofing on autonomous vehicles; Ring R1 C-UAS system defeats rogue drones.
Skeye Aerial Solutions
Netherlands
Dutch drone survey and inspection company founded 2013, acquired by Terra Drone 2018, specialising in oil & gas aerial inspection and geo-data capture.
Callisto (Beijing) Technology Co., Ltd. 木卫四科技
China
Beijing automotive cybersecurity firm founded 2021; S3-VSOC platform reportedly monitors more than 2.8 million connected vehicles.
Robotics needs a service framework.
RSF defines a common language for robot service capability, lifecycle operations, certification pathways, and service-provider networks.
Join the RobotToday community on LinkedIn
Daily robotics news, in-depth analysis, conference highlights, and discussions with professionals worldwide.
