Nvidia Research announced today that it has developed a new AI agent, called Eureka, that is powered by OpenAI’s GPT-4 and can autonomously teach robots complex skills.
“Reinforcement learning has enabled impressive wins over the last decade, yet many challenges still exist, such as reward design, which remains a trial-and-error process,” Anima Anandkumar, senior director of AI research at Nvidia and an author of the Eureka paper, said in the blog post.
“Eureka is a first step toward developing new algorithms that integrate generative and reinforcement learning methods to solve hard tasks.”
Nvidia Research also published the Eureka library of AI algorithms for people to experiment with them using Nvidia Isaac Gym, a physics simulation reference application for reinforcement learning research.
The current Nvidia Research work builds on previous efforts including the recent Voyager, an AI agent built with GPT-4 that can autonomously play Minecraft.
In a new research paper titled “Eureka: Human-level reward design via coding large language models,” the authors said that Eureka “Exploits the remarkable zero-shot generation, code-writing, and in-context improvement capabilities of state-of-the-art LLMs, such as GPT-4, to perform evolutionary optimization over reward code.”
“Without any task-specific prompting or pre-defined reward templates, Eureka generates reward functions that outperform expert human-engineered rewards. In a diverse suite of 29 open-source RL environments that include 10 distinct robot morphologies, Eureka outperforms human experts on 83% of the tasks, leading to an average normalized improvement of 52%.”.