VFX Artist Breaks Down This Year's Best Visual Effects Nominees

My name's Kevin Baillie,

and I'm a visual effects artist.

I got my start when I was 18 years old in the industry,

on Star Wars: The Phantom Menace,

and have since worked on movies

like Harry Potter and the Goblet of Fire,

Transformers: Age of Extinction,

and a whole bunch of movies with Robert Zemeckis.

We're here today to talk about

the five visual effects Academy Award nominees,

and what makes them so groundbreaking.

What's really exciting to me as a visual effects artist

about this year's field of nominees,

is how diverse they are.

Some years you have three or four really similar films,

and one kind of stands above the rest,

and it's an easy guess.

This year, it's everything from digital characters,

to de-aging, to invisible effects, virtual production.

Every film stands for something different.

[upbeat music]

First, let's look at Avengers: Endgame.

Avengers: Endgame, which is a whole tie-up

to the Marvel comic universe,

really built upon successes of previous Marvel films

with their digital characters.

Thanos was an amazing performing digital character

in Infinity War that made use of machine learning,

and all kinds of cool tech,

to bring him to life in that film,

and they just elevated that technology

to a whole new level.

And not only was there Thanos,

in this film we also had smart Hulk, right?

So we had multiple digital characters

that were performing, emoting,

that the audience could really connect with.

In previous Marvel movies,

the Hulk has always been this sort of raging,

stupid, embodiment of anger,

and he didn't really have to connect with the audience

on any other level than that.

With smart Hulk in Avengers: Endgame,

he actually has got his wits about him.

Maybe smash a few things along the way.

I think it's gratuitous, but whatever.

[growls]

[Kevin] And he's sitting there performing

alongside his live action actors,

and needs to be up to the very same level as them

with every nuance of performance that he gives.

And now look at me.

Best of both worlds.

There was some pretty amazing technology

that went into bringing Thanos

and smart Hulk to life in Avengers: Endgame.

In the past there's been, you know,

techniques used where there's thousands

of dots on an actor's face,

and every single one of them is trying

to drive like a little piece

of a digital character's performance,

but there's always stuff that happens

in between the dots that we miss, right,

and we end up with this kind of uncanny valley effect

with digital characters.

Well, by using machine learning,

they spent a lot of time actually teaching the computer

how these faces should move,

and then when the actor goes to perform

with these head mounted cameras on,

they're actually not capturing that much data.

They're just sort of getting the gist

of what the actor's doing.

When I had the gauntlet, the stones,

I really tried to bring her back.

And they feed that to the machine learning algorithm,

and it analyzes what the face is doing,

and it effectively fills in the blanks,

and translates that to smart Hulk's performance.

So they actually use a lot of different sets

of input data, what we call it training data,

to learn how an actor's face should move.

So there is a very, very high resolution scan

of the actor's face that's hundreds of cameras,

multiple lights, that tell us not only

how their face is shaped,

but how their skin reacts to a different kind

of lighting hitting it.

And then we also put them through

what we call facial range of motions, right,

which is them going through every expression

you can possibly imagine.

It's actually very tiring going through this,

but what it allows us to do is to see

down to a pore level of detail

what their skin does in three dimensions,

as they go from a smile to a frown, for example.

Terabytes of data go into the input,

to train the machine learning algorithms,

and that allows us to actually not capture

a lot of data on set.

Then we can correlate that small amount of data

with this intense dataset that we have for training,

to create an incredibly rich performance

of a fully digital character.

[dramatic music]

In addition to these stunning digital characters

in Avengers: Endgame, there are entire worlds

that are designed from scratch,

just sort of out of peoples' imagination,

built down to every blade of grass,

and doing that in a level that is consistent

across an entire film,

and this film had 2500 individual shots

that had visual effects in them,

to make that consistent across the whole film,

is a huge, huge task on this scale,

and they really nailed it.

[Rey] People keep telling me they know me.

No one does.

[Ben] But I do.

Star Wars: The Rise of Skywalker

was a film that really played heavily on nostalgia

from a visual effects perspective,

and a storytelling perspective.

Not only did the visual effects team

have to kind of build on this Star Wars universe

that we've become so used to seeing,

and to have it feel really authentic,

and physical, and gritty, and grounded,

but they had to bring multitudes

of digital and actually live action puppets to life.

Star Wars has always been groundbreaking

from a technology perspective,

but it's also always been grounded in reality, right?

The world just feels tactile,

and in this film they went to huge lengths

to pay respect to how the original films were made.

They used tons of practical models,

and actual physical set builds,

in addition to a lot of digital techniques

that are cutting edge even for today.

That real blend of physical with state of the art,

is part of the Star Wars DNA.

One of the biggest challenges we always have

in deciding how we're gonna film a movie

is deciding what's gonna be real,

versus what's gonna be digital.

In the scene that takes place on the sunken Death Star,

Roger Guyette, the visual effects supervisor

and his team had to decide

what water should be real versus what should be digital,

and they ended up doing the vast majority

of the water digitally.

They actually created an entirely new ocean pipeline,

in order to be able to execute those shots.

But not all of the water was digital.

One of the really important things for us

as a visual effects team,

is to make sure that the actors always have context

to help with their performances,

and especially when it's something

that they need to touch.

Having something there for real,

like having these waves that are crashing over them,

be there for real,

is critical to achieving a believable illusion.

A really touching part of Star Wars: The Rise of Skywalker

was seeing Carrie Fisher onscreen again,

playing Princess Leia posthumously.

Instead of using the digital facial techniques

that are heavily used by some

of the other contenders this year,

they opted to use Carrie Fisher's real face,

filmed from footage from past films,

and augment it with digital hair,

and a digital body around her,

so that she fit into her environment.

I think that that really helped Carrie's soul

come out onscreen, and it was really appropriate

for that moment, right?

So, Thanos absolutely deserves to be a digital character.

Carrie in this film, the only way to do her

was to actually use her.

Since my days as an 18 year old on Star Wars episode one,

digital effects have come so, so far.

Back then we really had to be restrictive

about how we used the effects,

so digital characters were incredibly difficult to achieve.

We had to make sure that the camera wasn't moving too much

in any one shot, otherwise it would make

the shots take a long time, and be way too expensive.

Now, we're sort of freed up to be able

to do almost anything that we can imagine,

and so what filmmakers are challenged with these days

is not to ask whether something can be done,

which is what we faced back in the days

of Star Wars episode one, but should it be done,

and how does it service the story?

And I think Star Wars has evolved as a franchise

to be a great example of a film

that uses the right tool for the right job.

In your own time, gentlemen.

Must be something big if the General's here.

[General] They're walking into a trap.

Your orders are to deliver a message,

calling off tomorrow morning's attack.

If you don't get there in time,

[explosions]

we will lose 1600 men.

1917 is a movie that you might not look at

and say, Oh, that's a visual effects movie.

You would be sorely mistaken.

It is full of visual effects, beginning to end.

The whole movie plays as one continuous shot,

where there's no cuts that you can actually see

in the film,

and that required an immense amount of work

from the visual effects team.

The film was actually shot in several different locations,

some of them outdoors, some of them inside on sound stages,

over the course of several months.

And so how do you make that all look

like one seamless piece of storytelling?

Between each piece of footage,

shot on different days in different places,

digital effects artists had to seamlessly blend

from one to the next,

in a way that the audience can't actually perceive it.

One of the more dramatic examples

of one of these digital blends is a scene

where our hero character is running out of a village

that's on fire, away from gunmen,

jumps off a bridge and into a river,

and then floats down the river.

As the actor came around and jumped off the bridge,

he was transitioned to a fully digital actor,

until he lands in the water,

and then he became that same actor,

but on a different day, in a different place.

So, just that one example is thousands

of man hours of visual effects time,

to bring that to life.

In addition to all of the digital blends,

what a lot of people don't realize

is how much digital work is done

to actually plus out this World War I period world,

where tanks that are stuck in giant craters,

they're actually fully digital.

They were never built for real.

Big fields that have tons

of spent ammunition strewn all over,

very little of that was actually real.

And it all felt incredibly grounded.

It felt like it was really there,

and to me those kinds of effects,

what we call invisible visual effects,

that are there to support the story,

rather than be the story,

are some of my favorite kinds of effects.

It was like the army.

You followed orders,

you did the right thing,

you got rewarded.

The Irishman featured one specific technique

that is what got it nominated for an Oscar.

That is the de-aging of some of the most well known

and well loved actors of our time,

and to do that for the entire film,

without falling into the uncanny valley,

is a massive challenge.

What the visual effects supervisor, Pablo Helman,

and his team did on that film,

was to actually really use the actors facial expressions

on a movie set as ground truth,

and they made a digital mask effectively

that went over that actor's face.

To make these digital masks move perfectly,

they filmed every actor with not one camera, but three.

We have the main camera that was the normal camera

that you would use to shoot a movie,

and then we had two, what we call witness cameras

on either side.

And between those three cameras,

and a special piece of software

that Industrial Light and Magic wrote called Flux,

they were able to actually analyze every movement

of an actor's face.

Flux was actually able to figure out

what each actor was doing exactly,

create the younger version of them,

and then that would be superimposed

on the older version of the actor,

and that became the final result.

It was really important to Martin Scorsese

to be able to shoot this like a normal movie,

without motion capture, right?

He didn't want a bunch of technology getting

in the way of the process.

So the two cameras, the witness cameras

that are on either side of the main camera,

they actually shoot infrared footage,

because that allows the visual effects team

to light the scene in a way that allows those cameras

to actually see the actor.

We can't see infrared light,

so we can just pump in infrared light into the set,

and see really clearly what the actors are doing,

even though Scorsese wants to light it

as like a really dark moody scene.

So, what The Irishman visual effects team did

was actually brilliant,

is that they figured out how to get out of the way

of the filmmaking process of this, you know,

genius in Scorsese, to get all the data

that they needed.

A really important part of bringing these younger versions

of these actors to life was actually using thousands

of images and video clips

from each actor in their younger days,

so that they can not only build a face

that looks like kind of what we remember them

to have looked like in the past,

but also to help train algorithms

within the Flux software,

to make the faces move authentically.

Where are we gonna go but up?

When watching The Irishman

for all 3 1/2 hours of the movie,

I actually felt like they were really successful

in bringing these younger versions

of the actors to life.

There was some criticism about,

oh, you know, the posture of the actors

was a little more like an old man

than it was the younger version of yourself,

and to be honest, I think the sign of success

with any effect, is whether it helps

to engage you in the movie,

or if it bumps you out of the movie,

and for the duration of The Irishman,

I was just fully engaged in those characters.

So the little technical flaws,

of, in this instance posture,

they just didn't stand out to me.

I've heard a lot of actors talk about The Irishman

as giving them a new lease on life, right?

The fact that these actors could perform

as younger versions of themselves,

and do so so convincingly,

I think has pretty big ramifications

for storytellers moving forward.

We can now look at casting actors

based on their personality and their fit for a role,

and less for their age.

Some people talk about the ethical concerns

of being able to create a digital version

of any actor out there,

and I was like, Oh, are we gonna replace actors one day?

And I think The Irishman is proof positive

that that's just not gonna happen, it's nonsensical.

There is no amount of digital wizardry,

I promise you this, that is gonna bring a performance

to the level that a DeNiro is gonna bring

to the screen, right?

And that foundation, that soul of the actor,

that's what we're responding to

from an emotional perspective.

And as The Irishman proved to us, the digital wizardry,

more than replacing an actor's performance,

it's complementing it.

It's actually helping to highlight it,

and build upon it,

and that as a storyteller is

an incredibly exciting prospect to leverage

more and more into the future.

[Circle of Life]

In The Lion King, Rob Legato,

the visual effects supervisor, and his team,

created an entirely digital world

for the story to take place in.

In fact, there's only one shot in the movie,

the opening shot, that was filmed in Africa.

Everything else is completely virtual,

but it feels like real life.

It feels like you're watching

a National Geographic documentary.

Now the danger of having a completely virtual world

that you can do anything you want in,

is that it could end up looking like a video game.

No matter how realistic the grass looks,

if the camera's not moving correctly,

and if the lighting isn't cinematic,

it's just not gonna work.

So, what Rob Legato and his visual effects team did

on the film, is they designed virtual production tools

that allowed real camera teams,

with real camera cranes, and dollies,

and steady cams, that were puppeteering

these digital cameras in the virtual world.

And so what you end up with is a feel of the movie

that is every bit as naturalistic

as if it had been shot in the African Plains.

The thing that's always a dead giveaway

with digital effects is when things are too perfect, right?

The natural world, it has a certain amount of chaos

that is completely unavoidable,

and by using virtual production,

and real camera equipment

to design these digital camera moves,

that imperfection that we're so used

to seeing in cinema, is translated

onto these digital African Plains,

so that it just looks like something

that we're used to seeing.

Predicting who's gonna take home the Oscar this year

is, and I'm not copping out here,

it's just totally impossible.

All of the films that are nominated are so different,

and I really think that the winner is gonna be

probably the story that ended up connecting

with audiences the most.

[upbeat music]