Amazon Science Study Finds Dynamics-Aware Planning Improves D-PAD Navigation by 8.8 Points
- Published
- Source
- amazon.science
A 2026 Amazon Science publication by Ishitwa Viranchi, Nagesh Kashyap Akshintala, and Mrunal Shah studies autonomous task completion across 68 streaming applications on TV devices using only directional-pad (D-PAD) inputs. The researchers compare a vision-language model (VLM) that maps screenshots directly to D-PAD actions with a system that models focus-transition dynamics using Bayesian belief updates and graph-based path planning. The evaluated tasks are login, reaching subscription plans, content search, and settings navigation.
The study frames D-PAD navigation as a partially observable Markov decision process and tests three open-weight models: Google Gemma 3 at 4B and 27B parameters, and Alibaba Qwen3-VL at 235B. The dynamics-aware system improves login success for every model, by a mean of 8.8 percentage points. Its relative advantage is larger for smaller models, ranging from 16.2 points for the 4B model to 2.9 points for the 235B model. The results suggest that explicitly tracking interface navigation can improve performance beyond relying on screenshot-to-action prediction alone.
The reported comparison also finds that the 4B model with dynamics awareness achieves 32.4% success, exceeding the 27B model without dynamics awareness at 26.5%. The publication does not disclose a further timeline or deployment milestone. A relevant unresolved point is whether the reported gains extend beyond the tested 68 streaming applications and four task categories.
Original report
Dynamics-aware navigation for autonomous UI agents on directional-input devices
amazon.science · Amazon ScienceThis briefing is an independently written summary based on publicly available reporting and is provided for industry information and news discovery. The original report and source publication are credited and linked where applicable. RobotToday does not claim ownership of third-party source material.
Rights concerns? Contact [email protected] with the relevant URL and details. We will review the matter and take appropriate action where warranted.
More related news
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
-
9
RobotToday Weekly November 9–13, 2026 RobotToday · 5 days ago
- 10
-
11
AIS 2027: AI Sensors and Transducers, Kunming RobotToday · 1 week ago
- 12
Related Suppliers
Google DeepMind
Alphabet's London-based AI lab, developer of the Gemini Robotics vision-language-action models and the open-source MuJoCo physics simulator.
Google Ventures (GV)
Alphabet's independent venture capital arm investing in life sciences, AI, robotics, and enterprise technology
Loft Orbital
US space infrastructure firm operating shared YAM satellite buses that hosted the first onboard demo of Google DeepMind's Gemma AI model in orbit.
Zyphra Technologies Inc.
Palo Alto AI company building compact language, vision-language, and text-to-speech models including the open-source Zamba and Zonos model families.
ModelBest 面壁智能
ModelBest is a Beijing AI company whose MiniCPM edge-side language models run on-device in Samsung Galaxy AI and automotive cockpit systems.
Cerebras Systems
Sunnyvale, California AI-compute company founded 2015; builds wafer-scale WSE-3/CS-3 chips and, since August 2026, the CS-4 accelerator.
Dexmate
Santa Clara maker, founded 2024, of Vega, a foldable wheeled bimanual humanoid on Jetson Thor listed from USD 83,000, plus grippers and a teleop exoskeleton.
Dongguan BanRin Robot Technology Co., Ltd. 东莞市本润机器人科技股份有限公司
NEEQ-quoted Dongguan maker of BHS, BCS/BCSG, BHD and BCD harmonic reducers plus 4-axis and 6-axis industrial robots and screw-driving automation.
Passion Mobility 沛昇動能
Taiwan developer of self-following robotic carts and AGVs — CaddyTrek golf caddy, Tkart and vision-navigated Ukart for warehouse material handling
Lotte Engineering & Construction Co., Ltd.
Lotte Group's construction arm, deploying an autonomous floor-cleaning robot co-developed with TXR Robotics on its Incheon apartment sites since 2026.
Telefactor Robotics
US specialist in teleoperated robot vision, dexterous grippers, and control interfaces for bomb disposal and hazardous environment robots.
Dronetag
Prague maker of drone Remote ID transmitters and RIDER and Scout detection receivers, reporting over 50,000 devices sold to 25,000+ customers.
Robotics needs a service framework.
RSF defines a common language for robot service capability, lifecycle operations, certification pathways, and service-provider networks.
Join the RobotToday community on LinkedIn
Daily robotics news, in-depth analysis, conference highlights, and discussions with professionals worldwide.