Robotic Alliance: Highlights from Day Two of GTC 2024

The GTC 2024 conference in San Jose is largely focused on artificial intelligence. However, on March 19, Nvidia also announced several significant products in the field of robotics.

I Am GR00T

The company introduced Project GR00T (Generalist Robot 00 Technology) — a multimodal artificial intelligence developed specifically for androids.

Jim Fan, Research Manager and Head of Embodied AI at Nvidia, wrote that GR00T enables robots to understand instructions through language, video, and human observation to perform various tasks.

Today is the beginning of our moonshot to solve embodied AGI in the physical world. I’m so excited to announce Project GR00T, our new initiative to create a general-purpose foundation model for humanoid robot learning.

The GR00T model will enable a robot to understand multimodal… pic.twitter.com/EqN19Z3cXH

— Jim Fan (@DrJimFan) March 18, 2024

According to Nvidia CEO Jensen Huang, GR00T will allow robots to learn coordination and complex movements, adapt to their surroundings, and interact with the real world. During his presentation, the company head showcased several androids equipped with GR00T performing various tasks.

For the duration of testing, the AI product is available only to select partner companies, including 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics, and XPENG Robotics.

“Creating foundational models for humanoid robots is one of the most exciting challenges in AI today. Leading researchers worldwide are coming together to make a giant leap towards general-purpose robotics,” Huang stated.

Currently, details about GR00T’s internal architecture remain unknown. The hardware operates thanks to the new Jetson Thor computing platform.

New Addition to the Jetson Family

Jetson Thor is capable of performing complex tasks involving human-machine interaction. The platform is based on the Thor SoC chip and includes the latest Nvidia Blackwell GPU.

The GPU performance is eight times higher than the previous Jetson Orin version.

Powered by Thor, the GR00T product was developed using Nvidia’s Isaac Robotic Platform tools, including the new Isaac Lab for reinforcement learning.

Isaac Manipulator offers libraries that help manufacturers of robotic manipulators add new movements to products and enhance their agility. They include models for object detection and tracking, assessment of their 6D position, and creating accurate predictions for interacting with the environment. 

Isaac Perceptor assists in managing androids in unstructured environments. For three-dimensional perception and volumetric vision, it uses a 360-degree multi-camera view and accelerated AI algorithms.

Exciting news from #GTC24! NVIDIA Isaac is revolutionizing robotics in manufacturing & logistics. With #GenAI & advanced simulation, NVIDIA Isaac is accelerating AI-enabled robots. Check out the new foundation models, robot learning framework, & more. https://t.co/nteaW1K5tl pic.twitter.com/4xzpUakpfe

— NVIDIA Robotics (@NVIDIARobotics) March 18, 2024

“Exciting news from GTC24! Nvidia Isaac is revolutionizing robotics! With GenAI and advanced simulation, Nvidia Isaac is accelerating AI-enabled robots. Check out the new foundation models, robot learning framework, and more,” the company’s robotics division reported.

March 19 also saw the announcement of a new alliance in the robotics sector.

Robots Join the Alliance

The Open Source Robotics Foundation launched the Open Source Robotics Alliance (OSRA). 

We are pleased to announce our partnership with @OpenRoboticsOrg in launching the Open Source Robotics Alliance. By joining forces with Open Robotics, #NVIDIA will continue to bring accelerated computing and #AI to open-source #robotics.

Learn more: https://t.co/9XfX6gvtuO pic.twitter.com/qgCDCRtY2B

— NVIDIA Robotics (@NVIDIARobotics) March 19, 2024

“We are pleased to announce our partnership with OpenRoboticsOrg in launching the Open Source Robotics Alliance. By joining forces with Open Robotics, we will continue to bring accelerated computing and AI to open-source robotics,” the statement reads.

The company became the first platinum member of OSRA. Intrinsic and Qualcomm joined the alliance with the same status.

During the first day of the GTC 2024 conference, Nvidia announced a new generation of AI chips and software for working with large language models.