New Generalist AI Masters Humanoid Robot XHAND (4000 TRAJECTORIES 100+ TASKS)

Ann Hughes presents the latest advancements in AI robots, highlighting two new models: Figure Robotics’ Figure02 and Lymix Dynamics’ CL3. Both robots have advanced features for balance, movement, and task execution. Additionally, a new technology called Video Prediction Policy (VPP) has been introduced, which improves robot dexterity by predicting and adjusting actions in real time. Lastly, the X-hand from Robot Era, which provides detailed feedback on objects it handles, is also discussed.

I’m Ann Hughes, and I’m going to show you the top 5 latest AI robot advancements, plus at the end I’ll show you an open source AI that automates your entire computer. But first, Figure Robotics just released a video demonstration of their Figure02 robot dynamically stabilizing itself, even with disturbances, and this robot is 5 feet 6 inches tall and it lifts up to 44 pounds, and it features 16 degrees of freedom in each of its hands for precise two-handed tasks, and 195 degrees of joint motion with 150 newton meters of torque, and its torque-controlled walking system prioritizes balance and adaptive foot placement, which allows it to recover from these pushes seen in the demo, and it’s equipped with 6 RGB cameras, depth sensors, and a vision language model, and this allows the Figure02 to process 360 degree visuals at 7 and 9 Hertz, and it coordinates its 35 degrees of freedom action space at 200 Hertz with its Helix Vision Language Action model, with triple the computing power of the Figure01.

Meanwhile, Lymix Dynamics just unveiled another video of its newest CL3 humanoid robot, and this robot stands at 5 feet 5 inches tall with a total of 52 degrees of freedom, and we’re going to go through what we know in terms of specs. Currently, the robot features high torque density actuators with a hollow actuator design, and these actuators deliver increased power output while supporting a larger height variation for the dynamic stance adjustments of the robot, and the robot’s high-performance full-body balance control ensures stability during complex movements or rapid shifts in balance, and the robot’s waist rotation is omnidirectional, which allows it to pivot its torso in any direction while navigating, and the CL3 features advanced joint coordination, particularly in its hip joints, where it can rotate 360 degrees, and combined with full-body joint coordination, this enables for smoother transitions like lifting from a seated position or standing, or even executing multi-joint sequences with more accuracy, and demonstrations of the CL3 robot highlight its ability to bend and rotate, but currently there’s no word on how much this robot might cost, and in a breakthrough for robot dexterity, researchers just introduced video prediction policy, and it’s a pioneering generalist robot policy for multitask, dexterous, and manipulation, and VPP uses predictive visual representations to let a single generalist policy handle all kinds of manipulation tasks, but VPP’s model architecture actually employs a two-stage learning process, where in the first stage, a text-guided video prediction model is trained on extensive manipulation datasets, drawing on physical knowledge from internet-scale data, and this equips the model with a broad understanding of object interactions, and in the second stage, specialized networks aggregate predictive visual representations from the TVP model to generate precise robot actions, and in real-world experiments validating VPP’s abilities, they use the panda robot arm and the x-hand to achieve seriously cool generalist manipulation, showcasing all kinds of adaptability to physical systems requiring both fine motor skills and complex interactions, and a key highlight of VPP’s performance is its in-tool task rollouts, where researchers collected 4,000 trajectories across 100 tasks in 13 categories, and moving into the future, VPP could help humanoid robots generalize across tasks, as well as anticipate object dynamics to adjust its actions in real time.

Plus, remember these x-hands also provide detailed feedback on the size, shape, texture, and softness of the objects that it’s grasping, but make sure to tell us down below which robot hand you think is currently the most dexterous, and share how many active degrees of freedom it has. And when it comes to this x-hand from robot era, it features 12 fully active degrees of freedom, with 3 in each of its thumbs and index fingers, and 2 in each of its remaining fingers. Plus, it features 15 degrees of lateral index finger movement to enable further dexterity, and to top it all off, each fingertip has over 100 points of tactile array sensors, and this provides the robot with information on 3-dimensional force and temperature.

Next, researchers from the French Université de Lorraine have introduced Words to Contact, which is a brand new pipeline that uses large language models and vision language models to provide humanoid robots through tasks that require multiple points of contact, but this is based on natural language instructions. In fact, Words to Contact enables language assisted teleoperation and human-to-robot collaboration, and this allows humans to verbally direct robots to position support contacts before performing whole body reaching or manipulation tasks. And it works by translating verbal instructions into precise contact placement predictions within the robot’s field of view, and it also supports iterative corrections, which lets it refine its placements based on operator feedback until the desired location is achieved.

And when it comes to performance, Words to Contact was benchmarks using state-of-the-art LLMs and VLMs, and tests showed that even novice users quickly adapted to instructing the system, and to carry out real-world validation, they used the Talos humanoid robot, which successfully placed support contacts on various surfaces and maintained balance while reaching for distant objects, and this demonstrates that Word to Contact could push the envelope of humanoid dexterity, overall demonstrating Word to Contact’s potential for enhancing robot stability and human-to-robot interaction in the real world. And when it comes to smoother robotics, a toolkit just came out called Pyroki, and this streamlines inverse kinematics, trajectory optimization, online planning, and motion retargeting for humanoid robots and humanoid hands.

Plus, it features a flexible design that supports multiple robot types, and it’s enhanced by an interactive web viewer for real-time weight tuning and visualization. And Pyroki prioritizes both ease and speed, and it outperforms CuRobo with up to 1.7 times faster GPU-accelerated inverse kinematics, and this makes it a very powerful new tool for both robotics researchers and developers. And finally, ByteDance, the company behind TikTok, just launched an open-source AI automation agent called AgentTars, and it’s designed to streamline complex tasks by visually interpreting web content and interacting with command lines and file systems.

And while it’s only available in macOS, there is a Windows version that’s currently under development with AgentTars using an agent-based framework to autonomously plan and execute processes like web searching, and the system relies on Anthropic’s model context protocol, which integrates various tools to achieve a proto-AGI-like state. Plus, AgentTars features a multimodal interface with real-time feedback, and it provides a live view on action processes, including open documents and browser windows, with the event stream interface allowing users to monitor immediate statuses and intervene by adding instructions in real-time, and the project website shows off some practical examples, including a technical analysis of Tesla share price, as well as bug reports and different travel itineraries, but users can also export sessions as local HTML files or even upload them to external servers via the post requests, or they can even upload them to external servers with a post request, but the installation from GitHub does currently require API keys for the preferred models and search services with additional parameters for Azure OpenAI integration.

Tell us below what you think of this open source tool for controlling your computer, and if you want to know more about the latest in AI news, make sure to like and subscribe, and click this video here. [tr:trw].