Google DeepMind introduces on-device Gemini AI model for robots

Gemini Robotics On-Device is intended to make powerful robotics models more accessible and adaptable. | Source: Google DeepMind

Google DeepMind this week introduced an on-device Gemini Robotics model for general-purpose dexterity and fast task adaptation. DeepMind said this vision language action, or VLA, model will bring Gemini 2.0’s multimodal reasoning and real-world understanding into the physical world.

Gemini Robotics On-Device is a robotics foundation model for two-armed robots, engineered to require minimal computational resources. Since the model is optimized locally and operates independently of a data network, DeepMind said it’s helpful for latency-sensitive applications. It can also ensure robustness in environments with intermittent or zero connectivity.

In addition to Gemini Robotics On-Device, DeepMind introduced the Gemini Robotics software development kit (SDK). Developers can use it to evaluate the VLA model for their tasks and environments, test it in DeepMind’s MuJoCo physics simulator, and quickly adapt it to new domains, with as few as 50 to 100 demonstrations. Developers can access the SDK by signing up to DeepMind’s trusted tester program.

DeepMind builds on Gemini 2.0 momentum

It has been only a few months since DeepMind introduced Gemini Robotics, and it is already building on its task generalization and dexterity capabilities capabilities. The Google unit said the on-device model is:

• Designed for rapid experimentation with dexterous manipulation
• Adaptable to new tasks through fine-tuning to improve performance
• Optimized to run locally with low-latency inference

Gemini Robotics On-Device achieves strong visual, semantic, and behavioral generalization across a wide range of testing scenarios, the company claimed. The platform also enables robots to follow natural language instructions and complete highly dexterous tasks, such as unzipping bags or folding clothes. DeepMind will still offer the Gemini Robotics model for those seeking similar results without on-device limitations.

This system isn’t limited to tasks that will work out of the box. DeepMind said developers can adapt the model to achieve better performance for specific applications. The company tested the model on seven dexterous manipulation tasks of varying degrees of difficulty, including zipping a lunchbox, drawing a card, and pouring salad dressing.

DeepMind expands Gemini to more robot embodiments

While DeepMind trained its on-device model only for ALOHA robots, it was able to further adapt the model to a bi-arm Franka FR3 robot and the Apollo humanoid robot by Apptronik.

On the FR3 robot, DeepMind said the AI model followed general-purpose instructions. It could handle previously unseen objects and scenes, complete dexterous tasks like folding a dress, or execute industrial belt-assembly tasks that required precision and dexterity.

On the Apollo humanoid, DeepMind adapted the model to a significantly different embodiment. The same generalist model can follow natural language instructions and manipulate different objects, including previously unseen objects, in a general manner, said the company.

DeepMind asserted that it is developing all of its models in alignment with its AI principles and applying a holistic safety approach spanning semantic and physical safety. In practice, this means capturing semantic and content safety using the Live API and interfacing the models with low-level safety-critical controllers to execute the actions. 

The company recommends evaluating the end-to-end system on its recently developed semantic safety benchmark and performing red-teaming exercises at all levels to expose the model’s safety vulnerabilities.

DeepMind added that its Responsible Development & Innovation (ReDI) team continues to analyze and advise on the real-world impact of all Gemini Robotics models, finding ways to maximize their societal impact and minimize risk. Its Responsibility & Safety Council (RSC) then reviews the assessments, providing feedback to help further maximize benefits and minimize risk.

To gain a deeper understanding of Gemini Robotics On-Device’s usage and safety profile and to gather feedback, the company is initially releasing it to a select group of trusted testers.

Keep track of your essentials with the Apple AirTag 4 Pack, the ultimate tracking solution for your belongings. With over 5,972 ratings and a stellar 4.7-star average, this product has quickly become a customer favorite. Over 10,000 units were purchased in the past month, solidifying its status as a highly rated Amazon Choice product.

For just $79.98, you can enjoy peace of mind knowing your items are always within reach. Order now for only $79.98 at Amazon!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!