How Respeecher Resurrected A Deceased Cyberpunk 2077 Actor With AI

0
894


In the current climate surrounding AI content in films, TV shows, and games, I was surprised at how upfront Respeecher is about its work in Cyberpunk 2077. The Ukrainian-based studio is best known for bringing back young Luke Skywalker in The Mandalorian and The Book of Boba Fett, where the debate involved not only ethics but quality too.


THEGAMER VIDEO OF THE DAY

SCROLL TO CONTINUE WITH CONTENT

But now, a Respeecher rep was in my Twitter DMs, sharing news that had largely flown under the radar – it revived the voice of a deceased actor for Cyberpunk 2077: Phantom Liberty.

Miłogost Reczek played Viktor in the Polish language dub of Cyberpunk 2077. It wasn’t his first gig with CD Projekt Red – he also played Vesemir in The Witcher games. He passed away in 2021, aged 60.

Viktor plays a small but integral part in V’s story, and was planned to feature in Cyberpunk 2077’s updates (eventually reduced to a singular update) when Reczek passed. With the family’s approval and participation, CD Projekt Red began to look into ways it could bring Reczek’s performance back for Phantom Liberty. Thankfully, it was already in touch with a studio who knew just what to do.

“We had been talking to CD Projekt Red for several years,” says Respeecher CEO, Alex Serdiuk. “They were exposed to the capabilities of our technology. We kept them updated.”

By the time CDPR had contacted Respeecher, the game studio had already decided against reusing old lines or hiring a sound-alike. Most crucially, it had Reczek’s family on board, something Serdiuk is quick to establish as a red line for Respeecher.

Cyberpunk 2077 Viktor vector

“When we started the company, we started with this ethics statement,” says Serdiuk. “We decline a lot of projects.”

Respeecher says it often doesn’t work directly with the actors, or the family of a deceased actor, as this is done by whichever studio hires them. With that in mind, I ask if that means they’ve even taken on a job without knowing how the original actor or their family felt about the matter.

“No, never. That has never happened over the six years of our existence. And that’s one of the reasons why Respeecher is the only voice cloning service that has their own IMDb page and has been used in massive productions.

“This trust, this respect, is important not just in terms of general ethics, and how we want to see our society move forward and adopt new technologies, but also a big part of the content itself,” he explains. “Whatever PR issues that can appear, if they use another technology and company, they will inevitably get worse results because no one also provides this unique service.”

Respeecher was founded in February 2018, and its headquarters is in Kyiv, Ukraine.

In this specific case, it’s hard to make the argument that this was in any way disrespectful towards any of the actors involved. Respeecher seems confident in this, hence its boldness in telling the world that it recreated a deceased actor’s voice, all while there were two strikes in the entertainment industry largely focussed on unethical AI usage.

Having spoken with a few AI enthusiasts in the past, including an incredibly concerning conversation with Georgi Dimitrov, the CEO of an “AI girlfriend” service, the differences here were striking. Dimitrov was quick to shoot down concerns surrounding ethics before I had the chance to bring them up. Serdiuk didn’t. From the way he speaks, it almost seems that he considers himself and Respeecher to be part of a different industry entirely, not considering the more insidious adopters of AI to be his peers at all.

“We built our business based on speech-to-speech voice conversion technology, [which] is a big differentiator from most of the synthetic speech in the market, which is text-to-speech. Speech-to-speech means that we need to have a performer – we need to have a good performer… we enhance the industry with that.”

There is, of course, the argument that every time a company uses Respeecher’s services, it’s taking away a job from another actor who could have replicated what the directors were looking for. Serdiuk, however, is adamant that its service will not do this, and will instead enable actors to get a wider range of acting gigs.

“Many of them […] use our technology so they can do character voices which are hard to get,” says Serdiuk. “For instance, if you need to do a character voice for animation which screams a lot, you can do that for like 15 minutes a day. And then you need to rest all day long, just because it hurts. Now they can train the model in their character voice and drive it in their normal voice.”

“If I reached out to voice actors in 2018, nine out of ten would say [they] are scared of this technology. Now nine out of ten would say, ‘Let’s explore that’. They understand that technology is not going anywhere. So it’s something that needs to be adopted.”

Speech-to-speech means that we need to have a performer – we need to have a good performer… we enhance the industry with that.

In my experience speaking with actors, this has varied. The likes of Yuri Lowenthal, Roger Clark, and Erica Lindbeck have been against both text-to-speech – which doesn’t require their involvement or consent – and the practice of game companies collecting their voice and mocap performances for use in other games. This would mean that they could later appear in games they hadn’t signed on for, without having to give any additional performances.

As Serdiuk explains, this isn’t what Respeecher wants to do. It wants to stick with speech-to-speech, which would require the actor to record new lines anyway. But some studios are clearly interested in replacing creatives altogether, and this darker side of AI usage cannot be ignored. In fact, actors tell me the abuses are already happening.

“Many gaming actors have already had to deal with and address having their facial data and mocap footage used in perpetuity by gaming studios, even for another game at a much later date,” Roger Clark, best known for playing Arthur Morgan in Red Dead Redemption 2, told me in July. “Consumers deserve better than some Frankensteined performance.”

Related: David Hayter Says It’s “Far Past Time” Unions Protected Voice Actors And Animators

As far as Serdiuk is concerned, this won’t replace the work of actors because it’s simply not very good. “We see many big players in the industry releasing technology similar to ours, like voice cloning technology, without managing control of the rights of the likeness or moderating content,” says Serdiuk. “That’s worrisome for actors because someone can use one of those technologies for their content. Fortunately, high-quality content would not [cause] that trouble, in my opinion. […] Writers are threatened more by that technology, but I am still a big believer that humans cannot be eliminated.”

Yet it’s perhaps Respeecher’s biggest gig yet that divides opinion on all of the factors to consider with AI, from its quality, or the impact it has on actors.

Star Wars has a murky history with bringing back its older characters. Han Solo was recast, but after his spin-off movie flopped, Disney seems to have been terrified to let another actor step into a character’s shoes again, reanimating younger Luke and Leia through a mix of deepfake and AI technology.

Yet the most immediate debate was its quality. Unlike Luke’s Mandalorian appearance, we all knew he was brought back by Respeecher ahead of his Book of Boba Fett inclusion. Serdiuk doesn’t agree with me when I say that the reception was mixed.

Luke Skywalker Book of Boba standing in front of Grogu

“In general, the reaction to The Mandalorian was extremely positive, we can say that 99 percent of fans were excited to see young Luke just because they were brought to their youth,” he says. “We [didn’t] see many complaints about Luke in Book of Boba Fett […] we managed to improve our quality even more.

“Star Wars fans, I believe, have adopted the idea and support recreating the character in this particular way. […] We’ve seen grown men crying just because they saw the Luke from their childhood.”

Again, in isolation, there was no foul play here. Like with Cyberpunk 2077, Serdiuk says that Mark Hamill was on board with bringing young Luke back before Respeecher was even hired. The idea that it could have stolen a role from another actor who could play a younger Luke isn’t one that Respeecher is concerned with, saying it is up to its clients to decide what direction to take.

When it does make its own decision about what to do with its tech, it looks promising. Serdiuk is eager to share Respeecher’s work with those who have had their voice boxes removed in surgery, using voice cloning technology to give them back the voice they have lost. The goals are ambitious – they aim to achieve voice conversion with these patients in less than 50 milliseconds, so that the software is hardly noticeable.

We’ve seen grown men crying just because they saw the Luke from their childhood.

Even just focusing on Respeecher’s entertainment industry ambitions, it’s far from over. Serdiuk admits that video games are much harder to work on, but after Cyberpunk 2077, he tells me we can expect to see his services pop up in more titles.

He also isn’t fearful that incoming legislation could hamper its work. He feels that his company has stayed ahead of regulation by sticking to its ethical code, and turning down gigs that would jeopardize that. The fact that it was willing to be among the first studios to pioneer bringing a late actor back for a role suggests it’s confident that its morals will keep it safe from breaches of ethics – a less cautious company could have easily handled this poorly, and drawn the ire of both the entertainment industry and lawmakers.

It certainly has more legitimacy than other AI providers, but going forward, it will have to ensure that actors aren’t just agreeing because they have to, but because it will truly assist them with their performances. Already, there are many understandably concerned creatives to win over, and to pull that off, it will take more low-key and considerate jobs like Cyberpunk 2077, not the big headlines and debates that come with immortalising a young Mark Hamill.

Next: Gaming Companies See AI As A Toy, Not A Tool



Source link