What is a “deepfake”? In the modern era of artificial intelligence (AI), a deepfake is either a computer-generated image or video or an original one that has been altered using advanced AI technology. Using deep learning—hence the “deep” part of deepfake—they aim to make the subject look or appear to be doing something they’ve never done or said, sometimes for innocuous reasons, and sometimes not.
It might be surprising to learn that deepfakes, in the modern sense, have been around for nearly a decade. However, even before that, it wasn’t uncommon for images and videos to be edited the manual way: with a talented artist and a copy of Photoshop. However, those days are by and large behind us, and AI is very much the future.
Regardless of how you feel about deepfakes, or AI-generated content in general, their existence has opened a proverbial can of worms. We’re now living in an age where it can be difficult to tell whether the images and videos we are seeing are real or simply deepfakes.
A Quick History of Deepfakes
Did you know that neural networks (the technology powering deep learning and thus deepfakes) were first conceptualized way back in the 1940s? Knowing precisely what neural networks are isn’t too important, but put simply, they try to imitate the human brain. So, in essence, they try to get computers to think like people think. Thus, deep learning is a way to get computers to try to learn as people do.
It wasn’t until the 1990s that researchers first started using deep learning as a way to alter the content of images and videos. Then, in 2017, the term “deepfake” was first coined by Reddit. The first iterations of deepfakes were used to simply “swap faces.” Though it was easy to tell that these early attempts at deepfakes were, well, fake, the technology has only grown and evolved since then. Today, however, you have to look pretty closely at deepfakes to be able to tell that they aren’t real.
On the positive side, deepfakes can be used for entertainment, historical recreations, scientific research, and demonstration of technology. They can, as an example, help people hear their own voice again if they can't speak or bring historical figures to life. Imagine being able to see Benjamin Franklin talk about his experiences flying kites.
Conversely, deepfakes can be used to impersonate people, spread misinformation, and influence people to behave in a way that benefits the faker. They can also be used for malicious purposes like identity theft, financial fraud, blackmail, cyberbullying, and scams. And naturally, the technology quickly found its way to pornography—both consensual and non-consensual.
How Deepfakes Are Made
There are two aspects to creating a deepfake: training the model and using the model. Most people won’t have to worry too much about the lengths one must go to in order to train a deep learning model that’s used for deepfakes. However, to create increasingly realistic deepfakes, the model must be continuously trained on new data, primarily images and video.
Currently, generative adversarial networks (GANs) provide the best results, though they must be trained on an even larger set of data. GANs consist of two neural networks, the generator and the discriminator, which are trained together. The generator creates fake images, while the discriminator tries to distinguish between the real ones and the fake ones. In a way, GANs get better by trying to outdo themselves, and the results are scarily good. It’s virtually impossible to tell that faces created by GANs are deepfaked—they look that real.
As for creating a deepfaked video itself, you would need to provide the model with as much material as possible—the more you use, the better the results. So, if you wanted to, say, create a deepfake of yourself, you would need to dump as many videos and selfies into the model as possible to train the AI as to what you look and act like. Granted, the models and AI are far faster and more efficient than they were just two years ago, so training a model takes much less time, and the results are much higher quality.
After that, most models typically fine-tune the results to improve the deepfake’s realism, including smoothing out any artifacts, adjusting the lighting and shadows, and syncing lip movements with audio. That audio itself can either be synthesized with AI or dubbed to match the lip movements of the generated video. This might involve using text-to-speech systems or other voice synthesis techniques to mimic the target person's voice.
The final step involves editing the generated content to ensure it blends seamlessly with any existing video or audio footage, which might include color correction, adjusting frame rates, and other video editing techniques.
After all that, you receive a deepfaked video that, hopefully, is good enough to pass for the real thing.
Some Famous Examples of Deepfakes
Deepfakes are becoming more and more widespread, and a lot of them are exceedingly high quality, so there are some really good examples of convincing—or just plain fun—deepfakes out there, usually with the AI-generated nature at the forefront.
One of the best examples is the TikTok account dedicated solely to Tom Cruise deepfakes. This project, a collaboration between visual effects artist Chris Ume and actor/musician Miles Fisher, sees Fisher acting out the role of Tom Cruise and then being deepfaked into looking like the actor.
It also helps that Fisher does a pretty decent Tom Cruise impersonation, so this TikTok account doesn’t need to go to great lengths to replicate the legendary actor’s iconic voice. Fisher’s impersonation also helps to establish this account as purely parody since they make no attempts to convince viewers that it’s really Tom Cruise in the videos and instead are just having some fun. They even have the word “parody” in the description.
Another really early yet surprisingly convincing deepfake example that you might remember is the one of Mark Zuckerberg sharing Facebook’s privacy concerns. Though technically not a super difficult deepfake—it uses an old video of the CEO talking about Russian interference as its basis—the result is still impressive and a bit scary, considering how realistic it is at first glance. Upon closer investigation, of course, it has the obvious uncanny valley effect that a lot of deepfakes have. Similarly, deepfakes have also been used to impersonate plenty of political figures—a fact that is of some concern as the 2024 US election approaches.
It’s not all bad, though, as there are plenty of other fun examples of deepfakes. Not to dive too deep into any movie or TV spoilers, but Disney has been using deepfakes to de-age various Star Wars characters for a few years now, including having a young Luke Skywalker appear in a couple of episodes of The Mandalorian. People have also had fun on YouTube, inserting actors into roles they were never in. Anyone familiar with the source material would, of course, know that these instances are all cleverly done deepfakes.
The Benefits of Deepfakes
While deepfakes can definitely be abused, that doesn’t mean that they’re all bad or all just for fun. The technology has some tangible benefits when used responsibly. For starters, when you use something like an AI headshot generator to create an image of yourself, it can be a great and effective way to create a “deepfaked” headshot portrait that you can use for a wide range of things, like social media or in a professional capacity, like LinkedIn.
The 2020 documentary, Welcome to Chechnya, focuses on the anti-gay purges within Chechnya in the 2010s that affected over 100 men. In the film, deepfake technology was used to conceal the identities of survivors with that of volunteers. The benefit of this alone is great, as victims can share their stories without having to worry about their identity being compromised.
It also beats the old methods of concealing someone’s identity: blurred faces and voice changers, which—though effective—can detract from the immersive nature of a film; after all, it’s harder to connect with a subject if you can’t see their eyes. Welcome to Chechnya went to great effort in overlaying the faces of the “activists,” as the film calls them, onto the survivors, treating them as 100% digital prosthetics.
Deepfakes also have some less serious benefits as well, including education and accessibility. For instance, rather than suffering through boring lectures or impenetrable textbooks, deepfakes allow for actual (relatively speaking) historical figures to teach kids subjects in a much more engaging way. AI tutors can also better anticipate students’ educational needs.
How to Tell Deepfakes Aren’t Real
As deepfakes continue to become more and more realistic, it’s even more imperative to be able to recognize when you see them, even if the deepfake in question isn’t necessarily “bad,” per se. There are still risks associated with deepfakes, specifically falling victim to misinformation, identity theft, and scams.
The good news is that there are a few telltale signs that something is a deepfake; the bad news, however, is that with as fast as this technology is advancing, these signs may quickly become obsolete.
The first sign to keep an eye out for is inconsistent blinking. As you might imagine, human blinking patterns, much like hands, are difficult for AI models to accurately replicate. So, check to see whether subjects are blinking excessively, or not enough. Similarly, faces can be difficult for AI models to get quite right, so be on the lookout for unexpected levels of asymmetry and mismatched skin tones.
Further, sometimes hair can appear overly smooth or unnaturally rigid in deepfakes. Teeth might end up just a little too perfect, or have unnatural gaps. Be sure to also take a good look at the subject’s eyes. Human eyes are reflective in wildly complex ways, so if the light in their eyes just doesn’t look quite right and instead looks a bit too flat or unnatural, the video might be a deepfake.
Body language can also be difficult for AI models to replicate, such as awkward gestures or unnatural posture. Overly stiff or unnaturally fluid movements can also appear in deepfake videos, both of which are uncommon in real human motion.
Tools for Detection
Unsurprisingly, perhaps the best way to detect AI-generated content like deepfakes is with AI-powered tools designed just for that purpose. As the famous saying goes: fight fire with fire. Already, there are programs designed specifically to analyze media for the subtle inconsistencies we mentioned above.
So, one thing you can do if you’re worried whether a video you’re watching is real is to plug it into a tool like Deepware or Sensity AI. Though once you’ve seen enough deepfake content, it can become significantly easier to spot when something isn’t real.
For images, Google Image Search can be really helpful. If you suspect an image might be altered, you can run a reverse image search to see if it has been used elsewhere or manipulated in any way.
Final Thoughts
The era of deepfakes isn’t just upon us; it’s been here now for the better part of a decade. While originally they were easy to recognize, those days have passed us by. Now, it can be exceedingly difficult to tell when a video is real.
Luckily, there are plenty of good, if not great, applications of deepfake technology, from education to filmmaking. And while the negative implications will most likely never truly vanish, fortunately, there are plenty of ways to tell when a video or image has either been generated from scratch or manipulated in some way, shape, or form.
And if your own personal analyses aren’t enough, there are lots of AI-powered tools to help you distinguish fact from fiction.