Deepfakes can be used for benign purposes; however, the technology poses significant threats. Threat actors use deepfakes to commit fraud and spread malicious content against victims.
Deep fakes with harmful intent include sending nonconsensual pornographic images of Taylor Swift without her knowledge, as well as a video that misled a finance worker into believing they were in a conference call with their company’s CFO. Such abuse of technology results in increased division between society members, as well as tangible costs to individuals duped into engaging in financial crimes as a result of being duped into them.
Definition
Deepfake refers to digital content produced using artificial intelligence (AI) with the purpose of appearing authentic. This often requires two algorithms – one for replicating source material and another that detects telltale signs that reveal it as fake (such as lip-syncing or off-center lighting).
Deep learning technology offers both positive and harmful applications; however, its use also comes with risks of manipulation or harm if tools that utilize deep-learning become cheaper, easier to use and widely available. Potential threats related to deepfake include misinformation, cybercrime and identity theft among others.
Nonconsensual pornography represents one of the primary risks, accounting for up to 96% of deepfakes on the Internet, often targeting celebrities and politicians. But its use may extend far beyond this type of content – it could even be used for revenge or extortion purposes, creating bullying situations at schools or workplaces and placing anyone into embarrassing, dangerous, or potentially compromised scenarios.
Purpose
Deep fakes can serve many purposes, from helping people hear their own voices again after illness or treatment to replacing dead actors for continuity reasons – Carrie Fisher and Peter Cushing both appear in new Star Wars films even though they passed before those movies were made.
Malicious use of this technology represents the greatest threat. Cyber threat actors can spoof voices and produce videos appearing to show people saying things they never actually said; for instance, hackers could pose as someone else to ask them for money transfer.
Nonconsensual deepfake pornography has long been used as a form of blackmail against women, damaging their reputation or seeking revenge against people they don’t like. Such software applications may also include spyware and Trojan programs designed specifically to send ads containing nonconsensual sexual material that can be used against them and harm their dignity and rights.
Techniques
Deepfake software utilizes algorithms and computer graphics technology to produce an image or recording that appears to show someone doing or saying something they did or said, when in reality this did not take place. The underlying technologies can be divided into three categories: GANs (generative adversarial networks), autoencoders and natural language processing (NLP).
These algorithms are programmed to recognize patterns in facial expressions, movements and speech of their targets and recreate them on another scene – by cloning images, matching body movements and overlaying scenes onto each other.
Deepfakes can create realistic-appearing deepfakes that are difficult to distinguish from their original versions, enabling threat actors to spread malicious content that causes harm such as financial loss, damage to professional or social standing, fear, humiliation or shame for victims and harm to digital platforms and services; in addition, this practice contributes to widespread suspicion and mistrust across society – creating serious concerns with regards to growing tech advances.
Detection
Detection technologies employ deep learning algorithms to detect signs of AI manipulation, such as discrepancies between images or videos, an unnatural lip sync, strange colors or reflections and out-of-place details (jewelry, hair strands or buttons) at odd angles or off center. They also include image recognition to look out for pixelation, blurring or other anomalies.
While most examples of Deep fake involve celebrities and public figures, threat actors have easy access to Deep fake technology for use against individuals for malicious purposes such as cyber influence campaigns, sex crimes like revenge pornography, stock manipulation and account takeover.
Companies such as Sensity are developing strategies for quickly and reliably detecting deepfakes that prioritize swift and cogent identification. Their software helps improve security in Know Your Customer (KYC) processes by flagging impersonation attempts and guarding against identity theft using facial manipulation detection technology. A project called Reality Defender also seeks to keep deepfakes out of people’s lives by acting like an antivirus/spam filter hybrid; prescreening media content before sending obvious manipulations into quarantine zones for immediate removal.