Sony has used generative AI to turn Aloy into an unsettling digital animatronic

Sony is reportedly exploring how generative AI tools like GPT could be integrated with characters from major PlayStation franchises—and the results are weird to say the least.

Internal footage sent to The Verge by an anonymous tipster shows an genAI-imbued version of Aloy from Horizon Forbidden West holding conversations with a player.

The video, which was published on YouTube but has now been removed due to a copyright claim, was narrated by Sony interactive Entertainment director of software engineering Sharwin Raghoebardajal who said the demo provides a “glimpse of what is possible.”

You can, however, see a clip from the demo in the original article and it’s all rather uncanny thanks to a combination of over-exaggerated facial expressions and limbs that remain strangely motionless.

The footage (which has also been viewed by Game Developer) shows Aloy speaking with a synthesized voice and facial movements while responding to queries with the help of OpenAI speech recognition and problem solving technologies Whisper and GPT-4.

Meta’s open source large language mode, Llama 3, was also used to power the demo—alongside Sony’s own internal Emotional Voice Synthesis (EVS) speech generation system.

Raghoebardajal explained audio to face animation was handled by Sony’s internal Mockingbird technology.

He didn’t answer some rather pressing questions during the video, such as why players would need to chat with Aloy or other PlayStation protagonists (who they usually embody) or how the tech would impact voice actors and developers.

Gen-AI concerns in games

Generative AI continues to encroach on the video game industry and has so far garnered a mixed response.

Executives and leadership figures at major studios seem intent on heralding the technology as a game changer capable of rewriting the rules of production and gameplay.

Others, such as voice actors and developers in the creative trenches, have regularly raised concerns about the impact the technology could have on job security and the potential for legal issues in relation to copyright and authorship.

Take your gaming to the next level! The Redragon S101 RGB Backlit Gaming Keyboard is an Amazon’s Choice product that delivers incredible value. This all-in-one PC Gamer Value Kit includes a Programmable Backlit Gaming Mouse, perfect for competitive gaming or casual use.

With 46,015 ratings, an average of 4.6 out of 5 stars, and over 4K+ bought in the past month, this kit is trusted by gamers everywhere! Available now for just $39.99 on Amazon. Plus, act fast and snag an exclusive 15% off coupon – but hurry, this offer won’t last long!

Score this incredible deal now!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!