NVIDIA unveils ‘ACE’ AI model to bring game characters to life

NVIDIA introduced the Avatar Cloud Engine (ACE), an AI model that integrates artificial intelligence into NPC characters for natural interaction in games.

This tool allows software and game developers to create and integrate unique voice, conversation and motion models. The project is being implemented in collaboration with Convai, a company that develops conversational AI for online gaming environments.

Here’s how NVIDIA describes the basics of the model:

• NVIDIA NeMo – For building, customizing, and deploying language models with your own data. Large language models can be customized based on characters’ backgrounds and stories, and protected from counterproductive or insecure conversations with NeMo Guardrails

• NVIDIA Riva – for automatic speech recognition and text-to-speech for real-life conversation

• NVIDIA Omniverse Audio2Face – for instantly creating expressive game character facial animation to match any speech track. Audio2Face includes Omniverse connectors for Unreal Engine 5 so developers can add facial animation directly to MetaHuman characters

NVIDIA also showed off the model and showed an example of an AI-generated conversation. As you can see in the video below, the character answers questions based on a backstory using generative AI.

The new NVIDIA tool is a major advancement for the gaming industry. It’s hard to predict how AI will be incorporated into upcoming games, but there’s no doubt that the technology will significantly change the experience, at least when interacting with NPCs.

Game developers and startups are already using generative AI technologies from NVIDIA. For example, GSC Game World uses Audio2Face in the highly anticipated STALKER 2: Heart of Chornobyl. And indie developer Fallen Leaf uses Audio2Face to animate characters in Fort Solis, a third-person sci-fi thriller set on Mars. Moreover, Charisma.ai, an AI-backed virtual character creation company, uses Audio2Face for animation in their conversation engine.

NVIDIA is actively using AI across the board, and gaming is a key segment where the company is also rolling out its up to 16x compression technology, training DLSS 3 models on NVIDIA servers to improve detail and quality, and rolling out new caching technology that can help improve performance path tracing in games like Cyberpunk 2077 using this technology.