Generative AI is Coming to Robots, Courtesy of NVIDIA

In response to a surge in generative artificial intelligence (AI), NVIDIA made significant updates to its Jetson platform for edge AI and robotics. The updates focus on the Isaac Robot Operating System (ROS) framework and Metropolis, two primary components of the platform.

Generative AI: The Game-Changer

For nearly a decade, the NVIDIA Jetson platform has seen significant adoption across various industries like transportation, manufacturing, and healthcare, with over 10,000 companies, 1,000 partners, and more than a million developers using it.

As generative AI continues to revolutionize text and natural language processing, it’s also impacting computer vision. This transformative technology is expected to bring advancements in autonomous machines and robotics, where cameras play a pivotal role.

While traditional convolutional neural networks (CNN) have been the mainstay for computer vision in the past, they are often tailored for specific tasks and require vast amounts of data, leading to longer development cycles. Generative AI, however, offers greater generalization, allowing for faster development and increased accuracy.

Furthermore, generative AI can recognize and interact with elements it hasn’t been trained on, which is the focal point of NVIDIA’s announcement. Generative AI has already shown its power in various applications, such as video search, real-time asset tracking, autonomous planning, robot navigation, imitation learning for robots, defect inspection, and natural human-robot interaction.

Also see: Top Generative AI Apps and Tools

NVIDIA has discovered that these advancements can also be leveraged at the edge. The company is pushing the boundaries of computer vision through generative AI.

NVIDIA has developed a variety of tutorials, guides, and samples to facilitate the use of popular generative AI models. While many NVIDIA customers – including tech giants like Amazon Web Services (AWS), Cisco, and Siemens – are integrating Jetson into their operations, NVIDIA introduced the Jetson Generative AI Lab to supercharge AI app development. The new software offers developers access to state-of-the-art open-source generative AI models.

“Almost all the popular generative AI models are only running in the cloud, but you can now run them at the edge using Jetson. We also have sample scripts if you want to create a distilled version. So Jetson developers can go ahead and start developing applications with generative AI,” said Deepu Talla, vice president of embedded and edge computing at NVIDIA, during a news briefing.

Also see: 100+ Top AI Companies 2023

Updates to Metropolis and Isaac Frameworks

Metropolis aims to help companies use computer vision AI to address critical operational and safety issues. It provides application programming interfaces (APIs) and microservices that facilitate the development of complex vision-based apps, primarily for video analytics.

Companies like BMW and PepsiCo are utilizing Metropolis tools to address issues related to the Internet of Things (IoT), sensors, and operations using vision artificial intelligence. By the end of the year, NVIDIA plans to release an enhanced set of Metropolis APIs and microservices for the Jetson platform.

The Isaac platform is popular among various sectors, such as warehouse automation, smart manufacturing, and agriculture, for creating robust robotics solutions. NVIDIA made significant upgrades to the Isaac platform, particularly its perception and simulation features. The Isaac ROS software offers a range of graphics processing unit (GPU)-accelerated functions like depth perception and 3D scene reconstruction. The latest version, Isaac ROS 2.0, is now available to be used with Jetson for developing robotic solutions.

Additionally, NVIDIA introduced AI reference workflows to reduce development time and costs, with plans to unveil NVIDIA JetPack 6 later this year. JetPack 6 will allow AI developers to access the latest computing features without fully upgrading Jetson Linux. The new version will work with various Linux distributions, including Ubuntu, Wind River Linux, and Redhawk Linux, among others.

Also see: Best Artificial Intelligence Software 2023

Pushing the Boundaries in Computer Vision

Generative AI’s transformative capabilities are set to redefine industries that rely on computer vision technologies. Real-world application is not a mere vision but an achievable future, according to Talla, who shared three use cases that showcase the potential of generative AI:

• The Transformer PeopleNet model has been trained to detect people in videos with high accuracy.
• The Detection Anything model employs vision transformers to perceive and classify objects in real-time.
• The Multi-Modal AI Visual Agent model facilitates text-to-image and image-to-image contextual searches.

In a retail setting, for example, this can provide real-time intelligence, analyze traffic flow, detect people, and generate alerts for potential security breaches. All three models are freely available for public use.

“People want to try these models out before they go into production, so we created a playground to allow folks to download the latest models and try them out,” said Talla.

Given the advancements made by NVIDIA, the Jetson platform will likely be instrumental in shaping how AI and robotics evolve. It promises to streamline development processes and offer a variety of real-world applications. 

Also see: Generative AI Companies: Top 12 Leaders

The post Generative AI is Coming to Robots, Courtesy of NVIDIA appeared first on eWEEK.