Boston Dynamics ATLAS Robot Debuts New 50 DOF AI with Toyotas LBM

➡ Boston Dynamics and Toyota Research Institute have made significant advancements in humanoid automation with their Atlas robot. The robot uses a large behavior model to predict and adapt its actions, even handling unexpected events. It can perform complex tasks like folding robot legs, retrieving items, and even tying ropes, all initiated by high-level language prompts. The goal is to train Atlas on a wide range of experiences, allowing it to adapt and generalize across various tasks, with the ultimate aim of deploying it for real-world use.

Boston Dynamics and Toyota Research Institute just unveiled several breakthroughs, but how close are they to realising real-world, general-purpose humanoid automation? To start, these newest advancements centre around developing large behaviour models and deploying them on Boston Dynamics Atlas Humanoid. At the heart of this new large behaviour model is a 450 million parameter diffusion transformer architecture, which allows Atlas’s AI to ingest not only image streams at 30 frames per second, but also proprioceptive signals to track body position in space, as well as natural language prompts to specify the robot’s goals. With all of this information, the model then predicts sequences of its own future actions with timestamps for each portion, allowing for flexible and adaptive behaviour.

But yet another breakthrough feature is the model’s ability to adjust the speed of execution at inference time, all without the need for retraining. For example, the same policy can run at regular speed, or even three times the normal speed, with minimal loss of performance in tasks. Importantly, this kind of flexible timing control is what will allow robots to work faster than human tele-operators, especially for repetitive or highly-optimised tasks. And to demonstrate the practical value of large behaviour models, Boston Dynamics demoed a complex, multi-step task known as the SPOT Workshop, where Atlas is asked to do a three-step task.

First, Atlas grasps and folds the legs of SPOT robots from a cart and places them on a shelf. Second, the robot retrieves faceplates from the same cart and then opens a bin on a lower shelf to deposit them inside. Third, once the cart is cleared, Atlas turns and transfers all remaining SPOT parts into a blue bin behind it. But the twist is that each sub-task is actually initiated by a high-level language prompt, and Atlas coordinates its locomotion, body posture and fine manipulation to complete the entire sequence. Incredibly, the policy is so robust that it even handles unexpected events, including parts falling on the floor or the lid closing unexpectedly.

This is achieved by incorporating examples of failures and recovery into the training data, allowing the model to learn to estimate the state of the world from its own senses and then react intelligently, all without requiring explicit programming for every possible contingency. And Atlas’ range of abilities is only expanding, with the robot no longer being limited to rigid object manipulation. Instead, the team has already benchmarked the robot on dozens of complex tasks, including rope tying, flipping bar stools, unfurling table cloth or flexible materials are particularly challenging due to their unpredictable shapes and dynamics.

But because the learning process is driven by demonstration, Atlas can simply leverage its large behavior model to bridge the gap. This is thanks in part to Atlas being equipped with 50 degrees of freedom to specifically focus on manipulation. Each gripper features 7 degrees of freedom for diverse grasping, including power grips and pinch grips. Plus, a pair of high dynamic range stereo cameras integrated into the robot’s head provide critical visual input for situational awareness and policy execution. In fact, until now, robots built for generalist capabilities have faced steep challenges because of the need to handle a wide range of objects, adapt to unpredictable environments and recover gracefully when things go wrong.

But Boston Dynamics believes the most promising way to achieve general competency is to simply train Atlas on vast and varied experiences, much like how large language models learn from massive text datasets. In fact, their goal is to build an automation platform where robots can not only learn but also adapt and even generalize across a multitude of manipulation and locomotion tasks, ultimately scaling up for real-world general deployment. So to achieve this, Boston Dynamics and Toyota Research Institute have officially joined forces to focus on the development, training and evaluation of large behavior models.

They believe these neural network policies will allow Atlas to perform long horizon, end-to-end tasks, all conditioned on images, the robot’s internal sense of joint positions and forces, and natural language commands. And to do it, Atlas learns step by step in four key stages, starting with data collection. High-quality data is gathered using both real-world robot teleoperation, where a human operator guides the robot through complex interactions and simulated environments. This dual approach ensures a diverse and robust training dataset, capturing everything from fine finger movements to full-body reconfiguration and whole body locomotion.

Once collected in the second stage, the data is carefully processed and annotated, making it suitable for feeding into large-scale machine learning pipelines. In stage three, neural network policies are trained on all of this data, learning to map sensor inputs, including vision, proprioception and language prompts, to a sequence of control actions for the robot. In the fourth stage, the trained policies are tested across a comprehensive suite of tasks. Insights from these evaluations inform what new data should be gathered next and which architectural tweaks could further boost performance. Incredibly, this rigorous loop of data collection, model training and iterative refinement has allowed the research team to rapidly improve the capabilities of Atlas, enabling it to handle tasks that would have been considered out of reach for humanoid robots just a few years ago.

On top of this, Boston Dynamics has developed a state-of-the-art teleoperation system to collect demonstration data for model training. Operators use a virtual reality headset and body trackers to immerse themselves in the movements. Earlier versions of the system allowed operators to perform a variety of tasks while standing still, but later upgrades introduced full body tracking, including the feet, allowing Atlas to step, crouch, reach and assume wide stances as needed. This teleoperation interface is not just a tool for demonstration, but also an important benchmark for evaluating the performance of the policies.

Furthermore, the neural networks that power Atlas are designed to use the same control interface as the teleoperation system, ensuring a seamless transition from human-guided demonstration to fully autonomous execution. And looking into the future, this data-driven approach to learning is general enough to be used for almost any downstream task that can be demoed via teleoperation. While Boston Dynamics has a strong start, there’s still a great deal of work to do. Boston Dynamics will use their established baseline of tasks and performance to focus on scaling their data flywheel to increase throughput, quality, the number of tasks, and their difficulty, all while also exploring new algorithmic ideas.

In fact, Boston Dynamics is pursuing multiple research paths, such as robotics performance areas like tactile feedback gripper control and rapid dynamic manipulation, integrating varied data sources like cross-embodiment and human-centric ego data, advancing reinforcement learning for vision-language action models, and implementing vision-language and vision-language action architectures to support more intricate long-horizon tasks and flexible reasoning. Altogether, Atlas has just unlocked a new level of AI automation in the real world to work in industrial settings, but how long do you think it will take for the company to make the transition into the home robot market? Tell us in the comments below what tasks you’d want your Atlas to be capable of out of the box, and how much would you be willing to pay for it? Anyways, like and subscribe for more of the latest AI news, and make sure to check out this video here for more on Atlas’s NVIDIA hardware.

Thanks for watching!
[tr:trw].