Researcher Explains Why Humans Can't Spot Real-Life Deepfake Masks

[Matt] You may not be easily fooled

but a recent study showed that a third of people

couldn't tell at a glance which of these are masks.

Sure, they go by really fast but when many people

were given unlimited time, they still couldn't tell.

[upbeat music]

When they're given as long as they like,

they're having a really hard time

and giving us the wrong answer

to this seemingly simple question

on about one in five trials.

[Matt] We've all seen masks like these

in movies like Mission: Impossible

but they have made it out of Hollywood

and are being used to fool people in real life

and even commit crimes,

like this masked bank robber in California.

The police were looking for someone

of a particular description,

and it later transpired that they'd been looking for someone

who is totally different.

To learn more, I spoke to Rob Jenkins.

He's a psychologist who has published studies

looking at facial perception using hyper-realistic masks.

Can you talk about these experiments

you've been running in recent years

looking at how these masks are indeed fooling people?

These hyper-realistic masks break the connection

that we've all become used to

between facial appearance and identity

and we were interested in whether people are getting away

with that kind of concealment because

people just aren't paying attention.

They don't see any reason to be looking at

a particular person's face

or maybe they're just not expecting

to see something like a highly realistic mask

and so it doesn't occur to them

that that might be what they're seeing

or if the masks really are so realistic

that they just pass for regular humans,

and so we wanted to test that more formally.

We had this computer based task

where we're showing people pairs of images on screen

and they just have to say which one is the mask.

Now, we've done that in a couple of ways.

One is just kind of at a glance.

So it's to model the sort of situation where,

you know, someone walks past.

We're talking like a, you know, half a second or something,

they just catch your eye.

And we did that because we thought

this task might be too easy,

and we'll just see everyone's performing perfectly, right?

That wasn't the case at all.

So people were terrible at making these discriminations

when it was a brief presentation.

So we decided to make it easier on them

and just say, all right, you can take as long as you like

and when they're given as long as they like,

that improves performance a little bit

but they're having a really hard time

and giving us the wrong answer

to this seemingly simple question

on about one in five trials.

That's actually one of the reasons

I find this so interesting

is that we're evolutionarily programmed

to pick up on facial cues.

It's so much of our social interaction,

yet you have these masks that aren't necessarily

a hundred percent realistic still fooling

a good amount of people.

Humans are social animals and so we rely

very much on getting whatever insight we can

into each other's mental states from our appearance.

So we can tell if someone's following along in conversation.

We can tell if they're cross with us or pleased to see us,

or skeptical of what we're telling them,

and we need to do that.

We also need to be able to connect it

to particular individuals so that when we see them again,

we can remember what happened last time

and behave accordingly.

And this isn't necessarily for fun and games.

There's actually been some reports of these being used

in crimes to conceal identity, right?

We started to notice a few years ago

that we were seeing reports in the news

of things like bank robberies where,

say, the police, were looking for someone

of a particular description

and it later transpired that they'd been looking for someone

who is totally different from the appearance

of the actual culprit.

So we're talking things like totally different age,

maybe different race, different gender,

that sort of thing and so you can see

how that kind of misleading of the investigation

could really get things off on the wrong path.

There was a case in the United States where

there was a string of bank robberies

that CCTV footage showed appeared to be carried out

by someone in their 70s or 80s,

and this figure gained a kind of anti-hero status online,

known as the Geezer Bandit

and so people were following him and, you know,

to some extent there was some cheering on

of this pensioner who was sticking it to the man

and he almost got caught on his final job.

So he was escaping on foot from the scene of the crime

and apparently ran off at such great speed

that people watching started to wonder if this

really was someone in their 70s or 80s and at that stage,

opinion started to shift towards, okay,

well this looks as if it could be

someone much younger wearing a mask

that's putting us off the scent.

I think what's so interesting with that bank robbery case

is the disconnect between what people thought of

for his face and what people thought of

for his ability to run,

which old people don't do very well.

How might that disconnect be playing out

when people are making these judgements

about how a person might be able to move?

If we want to improve detection of these masks

when they're being misused for criminal activities,

part of the solution might be to look for other cues

that conflict with the signal you're getting from the face.

There could be mismatches in apparent age.

So if the way the person is moving looks very youthful,

but the face looks very old and wrinkled and haggard,

that sort of useful mismatching cue.

So there are a number of things that you could look for

that are not directly to do with facial appearance

but exploits the facial appearance

in conjunction with other messages that you're getting

and that sort of conflict could be

a part of the detection story, I think.

These hyper-realistic masks aren't anything new.

We've seen them in the movies but what seems to be new

is that the price has been down quite significantly.

Can you talk about maybe the evolution of the industry here,

how this is becoming more available

for regular folks like us?

I think this started off in, like,

Hollywood special effects industry

and there are a few craftsmen who are masters of this

particular form of artistry and involved in sculpting

and monster-making and all the rest of it,

and they developed these techniques using

various materials over the years.

They started around 600 up to a thousand, maybe $1,500.

So I have one here that is

kind of an old favorite, this one.

So as you can see, it's made of this silicone material.

Silicone's very durable but also very flexible

and it kind of looks like human skin, in fact,

which is the idea, of course.

And it's not what most people think of

when they think of a mask

because it's a single piece that goes right over the head,

down the neck and over the shoulders

so there's no visible join at all.

So each one is hand-painted.

Every one is unique but you see all the details,

like little freckles and all the wrinkles are there.

Each one is hand-punched with human hair so, in this case,

eyebrows, there's a bit of ear hair there for added realism.

This is another example I've got

that has a full head of hair.

All of these developments over a number of years

have led to the position we're in now

where there are some manufacturers around the world

producing similar products along those lines

and as a consequence the price has come down a bit,

and we're starting to see these are more widespread

then they were initially.

I mean, you have clearly shown that humans are

not very good at picking up these masks

but might that actually pose a problem

for facial recognition?

Are computers actually going to be able to

recognize these as fake faces?

Although these hyper-realistic face masks

look a lot like real faces

in the visible spectrum of light,

there are other parts of the light spectrum

you could potentially play with.

So let's think about infrared.

The thermal signature of a real face

is gonna be quite different

from the thermal signature of a masked face,

especially when that mask is made of silicone

because silicone is a very good heat insulator, right?

So you can imagine if you had infrared vision,

which is a problem for humans but not at all

a problem for automatic face recognition systems,

that might just be a really quick and simple way

to tell, well, that's not a real head that I'm looking at

because it's not glowing.

Are there gaps in the way that

we as humans can perceive faces?

This is a question that's kind of new to science.

And we know that there's a range of ability

in identification tasks, just telling who's who.

Some people are really, really good at that

and in security and forensic contexts,

there's increasing interest

in being able to screen for those people

so that you can assign people to tasks

that involve face recognition who are naturally good at that

and tend not to make mistakes.

Whether you could do the same with this new task

of detecting synthetic faces is an open question

but it's one that we're looking in to.

Thank you for showing us your real face today.

Ha, that's what you think!

[upbeat music]