Tesla Bot New GEN 3 AI Upgrade Home Robot Release (AI SMART GLASSES)

So it’s looking increasingly likely that users will be able to simply show their robot what to do, probably using a pair of extended reality smart glasses, which we’ll get into later in this video. And robots will just do it, with increasing reliability over time. Plus, NVIDIA’s newest breakthrough called DreamGen also just taught robots how to generalize to new tasks using a four-stage pipeline. And this generates synthetic robot trajectories using video world models, which are just AI systems that simulate and understand real-world environments, including physics and spatial relationships between objects. In fact, DreamGen taught humanoid robots to perform 22 novel behaviors across 10 new environments by using teleoperation data from a single pick-and-place task in one environment.

And it does it through a four-stage pipeline, where first, these video world models fine-tune on a target robot to capture its dynamics. Second, these models are prompted with initial frames and relevant language instructions to then generate videos showcasing both familiar and new behaviors in new environments. Third, pseudo robot actions are extracted using either one of two models. And fourth, the labeled videos are then used for humanoids for visual motor policy learning, and then they execute the tasks without any additional human input. But the real feat is in DreamGen’s zero-shot generalization, where it achieves zero-shot behavior and environment generalization.

And this is a first in robotics literature. In fact, for each task, just 50 neural trajectories were sufficient for training the visual motor policies. And unlike traditional simulation-based robot learning methods which struggle with the gap between going from simulation training to real-world tasks, DreamGen, on the other hand, generates real-to-real synthetic data. Furthermore, it excels at creating training data for more complex tasks like manipulating deformable objects, including folding clothes, as well as tool use like hammers. And the DreamGen pipeline is applicable to various robotic systems, including even single-arm, low-cost robots. But most importantly, NVIDIA’s DreamGen reduces reliance on human-tell operation by instead using GPU compute and world models for more scalable, generalizable robotic systems, particularly for vision-language action systems in real-world applications.

But there’s a new paradigm of hardware that’s coming to make robot training all the more Google’s new operating system that’s designed for extended reality glasses and headsets, as powered by Gemini AI. And it works with all kinds of devices, including VR headsets, as well as lightweight smart glasses, with Android XR promising seamless integration of multiple forms of AI assistance. In fact, it was developed in collaboration with Samsung and it’s optimized for Qualcomm’s Snapdragon chip, allowing Android XR to run on a spectrum of devices tailored for different needs. For instance, in terms of entertainment and work, immersive headsets like Samsung’s Project Muhan are set to launch later this year, offering an infinite screen for apps.

And because of Gemini AI, users can also use Google Maps, explore with contextual insights, or even check real-time stats via the internet, and hundreds of developers are already building for Android XR since its preview last year, with Google providing a demo showcasing the smart glasses in action, with users navigating backstage, managing notifications, and texting via Gemini’s intuitive interface. Plus, Android XR’s ability to understand context and intent makes it a also saw a breakthrough in generative AI as Google launched Flow, which is its AI filmmaking tool that integrates VO3, Imogen 4, and Gemini to supercharge the video creation process.

And Flow generates cinematic clips with text prompts, but it features a special leg up in realistic physics and lip-syncing, and this is all thanks to VO3, while Gemini handles prompting, and Imogen 4 makes the visuals. But unlike its predecessor video effects, Flow has even more advanced features for both novices and pros, with these including key tools like camera controls for more precise shot adjustments, scene builder for seamless scene extensions with consistent characters, plus it supports custom assets, and there’s even FlowTV which is a curated hub with VO-generated clips and prompts to inspire creators.

And when it comes to cost, in the US when using the Google AI Pro subscription, it’s gonna be $19.99 per month for 100 generations, while Ultra’s gonna be $2.49.99 per month, with VO3’s audio features like dialogue and ambient sounds. And finally, researchers just unveiled a unified vision-language action model for robots, and it seamlessly blends reasoning and action together. And it’s called 1-2-BLA, but it has a special twist because 1-2-BLA adaptively switches between reasoning and acting modes, and this allows it to excel in complex tasks with human-like flexibility, and it demonstrates remarkable generalization across unseen scenarios, such as long horizon task planning where 1-2-BLA tackles intricate, multi-step tasks like cooking a hotpot, or even mixing cocktails, and it interprets physical scenes, and it generates action plans while tracking progress and adjusts dynamically based on feedback, and co-training with synthetic embodied reasoning data enables it to handle novel tasks, such as preparing a tomato egg scramble, and this is all without prior exposure, and it achieves 92% success rates in lab tests across 50 diverse tasks.

And when it comes to error detection and recovery, the model’s real-time error detection sets it apart, with 1-2-BLA identifying mistakes during task execution, reasoning through corrective strategies and implementing precise recovery actions, and in trials it corrected 85% of errors in manipulation tasks, and because 1-2-BLA is also designed for natural human-to-robot interaction, it responds instantly to human intervention, and it seeks clarification when instructions are ambiguous. Like in a demo, it adjusted its actions when a user altered a cooking setup, and it proactively asked for input and unclear object references, achieving a 90% satisfaction rate in human-robot collaboration tests.

But 1-2-BLA’s superior visual understanding also allows it to identify and interact with unfamiliar objects, like a GoPro or a Sprite bottle based on language instructions, with its synthetic data pipeline generating over 100,000 vision language pairs, and this is all enhanced with its spatial relationships and semantics, which enables 88% accuracy and grounding tasks for unseen items. And on top of this, the model’s training leverages an automated pipeline which eliminates manual intervention, and this scalable approach supports visual grounding and long horizon planning, with 70% of training data derived synthetically, and this reduces reliance on costly real-world databases.

But make sure to like and subscribe, and check out this video here if you want to know more about the latest in AI news. [tr:trw].