Though we often can’t see it, facial recognition software is everywhere. We use it to open our phones, solve crimes, and in advertising. But many factors play into facial recognition, including adaptable characteristics like facial hair. Does facial hair affect facial recognition?
Although facial hair impacts how humans read and identify faces, studies and demonstrations show that facial hair has minimal to no impact on physiological biometric systems like facial recognition software. Facial hair can be used in combination with other techniques to alter a computer’s ability to recognize a face.
But how does facial recognition software work, and why facial hair doesn’t make an impact? Researchers and tech gurus have shown that specific techniques can limit your detection by facial recognition systems; to use these techniques, you must understand how computers read and decode faces.
How Does Facial Hair Affect Facial Recognition?
Facial hair seems like a foolproof way to trick facial recognition software. It changes daily and limits the view of some parts of your face, like your chin’s angle and curvature.
But facial hair by itself isn’t enough to fool facial recognition technology. This is because confusion, rather than occlusion, is a better strategy to fool facial recognition AI.
Occlusion of your face—covering it or obscuring your features so you can’t easily be recognized—might seem like a logical tactic to avoid facial recognition. But occlusion is challenging. Most of the key facial points that determine a face aren’t easily hidden, like your eyes and the bridge of your nose.
Even in a face-covering like a balaclava, your face can be recognized by facial recognition systems most of the time. Facial hair, glasses, hats, and scarves simply do not cover enough critical facial points to confuse the AI.
How Does Facial Recognition Work?
Facial recognition works by using 3D mapping and an algorithm that defines a human face’s essential characteristics. The software can then identify the critical features of a face and the distance between those features. The software then creates a map of the eyes, nose, mouth, and brows.
Facial recognition software often operates using line edge mapping (LEM). LEM essentially has two categories—a line and an edge—that help the computer determine whether it is looking at a face and what that face looks like.
The basic idea is relatively simple: an edge marks the edge of a face (hair, chin, etc.) while a line defines the features of that face. Looking at a line edge map, you can see the basic characteristics of a face—it is reminiscent of a simple sketch or line drawing.
LEM works by making sense of the lines and marking the edge of the face to avoid confusion with other objects nearby. Because each person has a unique face, having the measurements of the facial features helps the computer to know who a face belongs to.
As facial recognition systems are fed more training data, the error rates go down. The recognition algorithm improves itself as more facial data with corresponding identities are manually loaded. Machine learning provides additional refinement to the algorithm as operators correct inaccurately identified faces.
Facial recognition can also capture other data, however. Facial recognition systems are now accurate enough that it can estimate:
- Hair and skin texture and color,
- Emotion (i.e., is the face smiling, frowning, or neutral),
- Presence of glasses and facial hair,
- Approximate age of the face,
- The biological sex of the face,
- Pitch, roll, and yaw (this refers to the spatial orientation of the face: for example, whether a face is looking up toward the sky or turned 90 degrees to the left)
Where and Why Do We Use Facial Recognition?
We use facial recognition technology in all kinds of places—mostly in the field of cyber security and law enforcement.
China and the U.S. use facial recognition on government-issued IDs, which, in combination with security footage from public spaces, can help reduce crime rates and solve other security-related problems.
We also have facial recognition software in the palm of our hands. Apple iPhones use face recognition to unlock your phone. While not as advanced as other face recognition systems, Apple and Google have poured considerable resources into improving their capabilities.
Like all technological advancements, facial recognition has its pros and cons. Some uses for facial recognition include:
- Opening phones to improve cyber security.
- Finding missing persons.
- Preventing and prosecuting shoplifters.
- Tracking school attendance.
- Identifying important or dangerous people in large, busy spaces like casinos or sporting events.
- Targeting advertising at gas pumps and other places where security cameras are in use.
- Helping blind people determine body language and emotions.
Obviously, some of these uses are more beneficial than others.
Using facial recognition to help the blind or find a missing child is an incredible asset, while targeted advertising is more contentious.
In the right hands, facial recognition software is an excellent tool. Still, many critics believe there are downsides to the widespread implementation of facial recognition technology.
Human Facial Recognition vs. Facial Recognition Software
Humans and computers recognize faces differently. In a study done on facial recognition and facial hair, scientists found that many humans can be confused by changes in facial hair and won’t recognize a person if their facial hair changes dramatically.
Humans recognize faces holistically; this means that when we look at a person, we take in all their features at once, and use that holistic image to identify them in the future. When something changes that alter our image of a person considerably, like aging, weight gain, or facial hair, we sometimes struggle to identify a familiar face.
On the other hand, computers look at the distance between the eyes, shape, and size of the nose, or curve of the chin to determine who a person might be. Because they lack a holistic view, computers can struggle when angles change or see a person in profile.
But small shifts, like facial hair or glasses, don’t usually confuse computers as much as they confuse humans. The ratio of eyes, nose, mouth, and brows are all the same, so the computer can identify the person even if they age or grow a mustache.
New Scientist reported in 2017 that a University of Cambridge team was able to provide training data to a machine-learning algorithm to look beyond small shifts in appearance due to a cap, scarf, and glasses. Their “system accurately identified people a wearing scarf 77 per cent of the time – a cap and scarf 69 per cent of the time and a cap, scarf and glasses 55 per cent of the time.”
These are higher error rates become less significant as the field of computer vision evolves. Advancements in depth map analysis and deep learning will continuously improve real-world recognition performance.
Is it Possible to Trick Facial-Recognition Software?
Tricking facial recognition software is all about confusion, rather than occlusion. Consider some of these tricks to confuse facial recognition software:
- Wear retro-reflective clothing or NIR LEDs. Retro-reflective clothing and near infrared light-emitting diodes serve the same purpose. They reflect invisible light back into the camera lens, causing it to create an unstable image. By wearing LED goggles or a similar retro-reflective mask, you can confuse cameras by creating bright light spots.
- Use a camera finder. This isn’t about what you wear—it’s about what you know. Camera finders are tools that can detect the presence of hidden cameras by detecting the light reflecting off a camera lens. If you know where a camera is, you can more easily avoid it.
- Get experimental with your hair and makeup. Patterns on hair and makeup can confuse AI, especially if they stop hitting on important facial points like the cheeks, eyes, or brow line. Check out this video (1m33s@2x) below on how to use makeup to hide from facial recognition.
- Wear a prosthetic mask. This is by far the weirdest option, but security advocate Leo Selvaggio created a high-quality prosthetic of his own face you can wear as a mask to confuse cameras. It doesn’t hide your face—it changes it.
- If you can’t hide, cover. If you can’t use the techniques above, utilizing a balaclava, sunglasses, and a hat is usually enough to confuse facial recognition software.
While facial hair can fool the human eye, it can rarely fool a computer. That being said, facial hair can confuse facial recognition software when used in combination with other techniques.
While the use of facial recognition technology is expanding worldwide, there are ways to avoid it. Facial hair might not be the best strategy for fooling facial recognition AI. But, there are still many ways to confuse computers and preserve your privacy.