Why Talking Animals Are One of Hollywood’s Biggest Challenges
- In this episode of “Movies Insider,” we learn why CG talking animals are such a difficult area of visual effects to master.
- Mike and Eric break down the creation of challenging Weta characters, from the fox in A24’s “The Green Knight” to King Louie in Disney’s live-action “Jungle Book”.
- Visit the Insider home page for more stories.
Here is a transcript of the video:
Horse: Uh, do you want to calm down? Now read my lips. We need to talk.
Narrator: One of the most difficult tasks when it comes to visual effects is to make animals look lifelike when they talk. This is a challenge that the Weta Digital effects studio has taken on. And if you watch Weta’s talking fox in the 2021 movie “The Green Knight,” you can see how far we’ve come. But starting from here …
Put on: [neighs] Oh my. I can’t ask him that.
Narrator: So far …
Fox: To end his song as he sees fit.
Narrator: It was not easy.
When the entire CG Talking Animals genre was born in the 90s, filmmakers faced a major challenge: How do you make something that’s fundamentally incredible believable? Previously, films were content to double the mouth movements of live animal actors or animatronic puppets, and then compose live animal sequences with 3D digital models of the animal’s mouths. Over the years, studios like Weta have mastered the art of creating fully digital creatures that speak and express themselves in naturalistic ways. Mike was one of the main hosts of “Rise of the Planet of the Apes”, where the chimpanzee Caesar slowly learns to converse with humans.
Narrator: The human actors not only voiced but played the apes, filming in motion capture suits that followed their movements. The Weta animators then had to map these performances to the digital monkey models, forcing the monkeys to deliver their lines and move in a way that they stayed true to the actors’ choices without feeling too human. The vastly different anatomies of humans and apes often made it impossible to accurately translate these one-for-one actor choices in the dialogue scenes. For example, monkeys have thicker eyebrows and larger neck muscles, which had to move differently when monkeys spoke.
Mike: Making sure the wrinkles are working fine, how the muzzle swells, how still it is when it says its line, sometimes it’s not a dead translation.
Narrator: Nowhere are the anatomical differences more extreme than in this oral area, which is obviously crucial for dialogue. The muzzle of a chimpanzee, for example, is much more malleable and dexterous than a human mouth.
Eric: Apes have the ability to move their lips in such a big way as humans do. Narrator: This meant that in some cases animators had to mix up some of the actors’ human lip shapes on the monkey digital facial puppets, like when they pronounced the phoneme ‘sh’. A chimpanzee’s lips extend in a long fold to form this shape, which would distract from the actual dialogue. So Weta mastered that, tightening the shape of the mouth and keeping a fuller, more human lip. These lip and muzzle differences between humans and apes actually made Andy Serkis a perfect casting choice for Caesar, as he often articulates with his lips while delivering lines.
Eric: Andy can be an overachiever. It started in the early days of motion capture, when fidelity wasn’t as good. He hasn’t captured as much movement, so when you see him playing for Caesar, he moves his lips a bit more than you usually expect when you’re talking to him in person.
Narrator: Beyond the most visible parts of the mouth, Weta spends a lot of time looking at dental scans of the actors’ teeth and videos showing how the tongue moves inside the mouth.
Mike: All this internal structure inevitably determines the behavior of the surface of the mouth, the behavior of the body. Monkeys and any other character or animal that we do, we can actually take the geometry of their mouth, their teeth, their gums, their tongue, so that the same way we know how the lips move on a human skeleton, the same thing happens with a dog, with a monkey.
Narrator: This in-depth understanding of the anatomy of the mouth came into play in “The Jungle Book”, where Weta had to make the giant orangutan King Louie not only speak, but sing. The hosts had to figure out how far to keep Louie’s tongue in his mouth when he sang his lines.
♪ You’ll see it’s true ♪ ♪ Someone like me ♪
Ultimately, however, animation supervisor Paul Story said his team had to sell the singing performance with body movement first, such as carefully timing Louie’s chest movements for show her breathing between the lines of the song and coordinating her body movements with each little head. motion, much of which was based on videos of Christopher Walken reading his lines.
When Weta got the dogs talking and singing for another live-action remake, “Lady and the Tramp,” the hosts kept an even lighter touch on the mouth.
Mike: The difference between a monkey’s mouth and a dog’s mouth is, you know, a dog has a long muzzle, and as soon as you start to overdo it with the jaw, it suddenly becomes that supercharged performance, and if you have really quick dialogue or someone singing a song, he quickly falls apart. The thing that we found with the dogs and the fox in particular is that if we pushed the performance to the front of the mouth and then supported that performance in the rest of the face and all the way down the body, I think. how nice it is to found it.
THEady: Well, it’s goodbye, I think.
Narrator: This approach to the fox in “The Green Knight” helped keep the mystery around him. Since he speaks in what feels like a dream state, director David Lowery wanted to leave viewers guessing.
Fox: Are you that man?
Mike: The dialogue was almost like, wait, did I just see this? Does it speak?
Eric: David wanted it to stay so subtle and such a character that you never really understood.
Narrator: So the hosts focused on getting the fox’s breath moving around its body as it speaks, and the subtle movements of the nose, neck, and jowls. They even zoomed in on parts of the body that you wouldn’t consider speech related, like timing the fox’s vocal delivery with its footsteps and the twitching of its ears.
Mike: They all fold into this little dialogue, so the dialogue can be subtle on the front of the muzzle, but the whole character is talking.
Narrator: Including, perhaps most importantly, through the eyes.
Eric: We’ve always known that the eyes are where you get a lot of the character performance.
Narrator: That’s why Weta is always improving the way he models his eyes, including parts that we don’t even see on camera.
Eric: If you go back to Gollum, the eyes we had, it was three pieces of geometry with textures painted on them. Our eyes are so deep now that you can actually fly in them. We’re actually putting all the right strings and muscles, connecting the pupil and generating the iris, everything. We spoke to ophthalmologists. We have eye scans.
Narrator: The animators at Weta have found that for dialogue scenes, it is essential to focus on the eyes.
Mike: And so we’re really precise in making sure that there’s something, even in the digital space, off-camera, that won’t even be in the shot the characters are focusing on.
Narrator: The studio has also learned that our eyes are constantly making small movements to capture new pieces of a scene, even when we seem to be staring. There’s that little extra movement that sits on top of all of that detailed performance.
Narrator: For moments of emotional dialogue like this, the team consulted an ophthalmologist to find out how moisture beads up on the eyeballs and enters the crease between the lower eyelid and the eyeball.
Eric: Blood is flowing while someone is crying, you get that real redness in the wattling. You really feel that emotion with all of these extra details.
Narrator: This obsessive level of detail made it possible to branch out into new types of talking animals, even those that swim, in the second season of “The Umbrella Academy.” After creating the chimpanzee Pogo for the
series, Weta was recruited to build a Talking Goldfish.
A J: Your recovery has been an inspiration to all of us.
Narrator: This character, AJ Carmichael, was a new type of challenge, according to animation supervisor Aidan Martin, because normal fish don’t even have a tongue and there aren’t many parts of a fish’s face. that would articulate the way we see with human speech. . But the animators at Weta found a solution: adding controls to AJ’s lips and mouth so that they can overlap, allowing for a wider range of mouth shapes as he delivers his lines.
It’s about finding the right balance between what is anatomically possible and what looks believable on screen. Weta has mastered this balance to create talking animal characters who are just as expressive and believable as their human counterparts.
Caesar: Do not come back.