Attackers vs. Defenders: The Future of AI Security

AI technology is advancing faster than ever before. Generative AI is becoming increasingly sophisticated and is now driving innovation across various industries beyond everyday conversational services. Starting with ChatGPT, technologies like Microsoft's VASA-1, HeyGen, Sora AI, and Suno AI have emerged, enabling the creation of videos where people appear to speak and move using just a single photo and audio file. These technologies can also convert given videos into 40 languages and over 300 different voices, and they can even be used to produce movies or compose music.

 

However, with the advancement of these technologies, there is also an increase in cases of AI misuse, presenting new challenges for the security industry. This report will introduce the types of threats generated by AI, cases of AI misuse, and AhnLab's strategies for enhancing security using AI.

1. The Potential Risks of Generative AI

Cyber threats stemming from AI manifest in various forms, many of which have already materialized and are causing significant damage. There are four major potential risks arising from generative AI.

 

Figure 1. The potential risks of generative AI

 

First, hallucination refers to errors that occur during the process of generating information based on the data the AI has been trained on. AI may sometimes produce information that is either nonexistent or inaccurate, potentially leading users to make poor decisions and raising concerns about reliability.

 

Bias emerges when the inherent biases in training data cause problems. AI learns patterns and makes predictions based on training data, but if this data contains stereotypes or discriminatory elements related to gender, race, or religion, it increases the likelihood of generating biased outputs.

 

Copyright issues arise when AI incorporates copyrighted materials without authorization during large-scale data training, potentially infringing on the rights of content creators. The legal status of AI-generated content in relation to copyright remains unresolved and is a critical issue that must be addressed in the era of widespread AI adoption.

 

Finally, data privacy breaches are another significant concern. If the data used for AI training contains personal or sensitive information, its exposure or misuse can lead to serious consequences. This could result in the unauthorized use of personal information and severe violations of privacy rights.

 

2. Attacks on Generative AI

Threat actors are abusing the widespread commercialization of generative AI and the various ways it's being adopted by businesses and individuals to attack generative AI in multiple ways. In a broad sense, attacks on generative AI include ▲adversarial prompting, ▲data poisoning, and ▲model reversing.

 

First, adversarial prompting can be divided into jailbreaking and prompt leaking. Jailbreaking involves disabling or bypassing the ethical constraints of AI by exploiting vulnerabilities through carefully crafted prompts designed to circumvent its built-in rules. As a result, AI can operate in dangerous or illegal ways.

 

Prompt leaking, on the other hand, refers to the theft of prompts used in generative AI. Prompts are instructions or commands that guide the AI to perform tasks accurately. If system prompts are leaked through code-sharing platforms like GitHub, threat actors could exploit them to distort AI or undermine its reliability. This is particularly concerning for large language models like GPT models, as leaked prompts could significantly undermine data security.

 

The second attack type, data poisoning, involves deliberately injecting malicious or distorted data into the AI training process to manipulate its outputs. A well-known example is Microsoft's AI chatbot "Tay". Released in 2016, Tay learned and interacted based on user-provided data. However, some users fed Tay inappropriate content, including profanity, racism, and sexism, causing the chatbot to adopt and spew highly offensive and hateful remarks. As a result, Microsoft was forced to shut Tay down just 16 hours after its launch. This incident highlights the critical importance of high-quality training data and how attackers can abuse AI systems.

 

The third type, model reversing, is a technique where attackers perform repeated queries on an AI model to analyze its functioning and extract its training data. By studying the model's responses, threat actors can infer its internal structure and training data, potentially exposing confidential or sensitive information, including personal data. This attack method exploits security vulnerabilities in AI models, posing significant threats to data protection and model security.

 

3. Cyber Threats Advanced by the Application of AI

Threat actors are not only targeting AI systems but also rapidly incorporating AI into their attack campaigns. AI-powered attacks include ▲malware creation, ▲phishing attacks, ▲deepfakes, and ▲vulnerability analysis and hacking. These attacks have reached unprecedented levels of sophistication and scale, making them significantly more dangerous than traditional methods.

 

Figure 2. Methods of attack using AI

 

First, AI is being actively utilized to write malware and scripts. Attackers leverage AI to generate large volumes of obfuscated code designed to evade detection, enabling them to bypass existing security systems. Notably, AI significantly enhances the efficiency and speed of creating ransomware or malware aimed at stealing personal information. Recently, there has been an increase in cases where AI is used to search for images containing sensitive information, such as passwords or cryptocurrency wallet addresses, which are then abused for malicious purposes.

 

In addition, AI allows attackers to execute phishing attacks with greater precision. AI-generated phishing email content appears highly natural, as if written by a person. While traditional phishing emails could often be identified through specific words or awkward sentence structures, AI-powered phishing creates messages that are virtually indistinguishable from legitimate business emails or notifications since they are based on large language models. The advancement of AI has caused a more than 40-fold increase in phishing emails compared to the past, significantly enhancing their realism and impact, which underscores AI's profound influence on cyberattacks.

 

Deepfake technology allows for the creation of realistic fake videos using just a few images and voice samples. The quality of deepfakes continues to improve, and they are increasingly being abused for crimes such as spreading fake news, causing social unrest, producing fraudulent video content, and conducting voice phishing. For example, attackers can now fabricate videos of celebrities making false statements or use a person's voice to commit financial fraud, creating significant risks of social and financial disruption.

 

AI also plays a pivotal role in helping attackers analyze vulnerabilities and develop new hacking techniques. The evolution of AI technology has democratized attack methods, lowering the barriers to entry for cybercriminals. Additionally, by combining various attack methods, it has become possible to develop new hacking techniques and maximize the efficiency of cyberattacks. Through vulnerability detection and pattern analysis, it is possible to identify unknown security vulnerabilities and use them for zero-day attacks.

 

4. AhnLab's Security Enhancement Strategy Using AI

Given the sophistication and complexity of AI-driven attacks, traditional security approaches are insufficient for effective defense. Consequently, AI holds immense importance not only in countering AI-enabled attacks but also in addressing emerging threats.

 

Figure 3. AhnLab's AI technology application plan

 

AhnLab is actively applying AI technology to enhance its solutions and platforms. The company's AI capabilities are rooted in filing technology, which classifies malicious and legitimate files. Additionally, AhnLab trains its learning models using a wide range of data, including behavior-based analysis, URL information, and event logs. This technology is integrated into all major products, such as V3, V3 Mobile, EDR, MDS, TIP, and XDR, significantly improving overall threat detection, analysis, and response capabilities. Furthermore, AhnLab is advancing its security technology in multiple areas, including malware and phishing email detection and response, prediction of potential threats, and comprehensive risk management.

 

AhnLab's AI-based security technology is utilized in four major areas as follows.

 

1. AI Security Assistant: This service acts as an AI-powered assistant, helping security personnel monitor and respond to security events in real-time. The AI Security Assistant analyzes the breach details and suggests future response measures based on this analysis.

2. AI Detection & Response: This technology detects and responds to various threats such as deepfakes, malware, and phishing emails in real-time. AI identifies abnormal behavior much faster and more accurately than humans, enabling a response in the shortest possible time after a security incident occurs.

3. AI Proactive Detection: AI predicts and warns of potential risks before attacks occur. By learning attackers' behaviors, it identifies attack intentions in advance, assisting security teams in implementing effective threat defenses.

4. AI-native SOC: This approach integrates AI into the Security Operations Center (SOC) to monitor the security environment and automatically respond to identified threats.

 

Next, let's look at how AhnLab is specifically applying AI technology in various fields.

 

1) Malware Analysis and Response

AI plays a pivotal role in analyzing malware and deducing threat actors' intentions. AhnLab loads AI into its cloud-based malware threat analysis and response infrastructure, AhnLab Smart Defense (ASD), to analyze and respond to malware threats. The AI embedded in ASD examines the executable code and external characteristics of malware while performing dynamic behavior analysis to comprehensively understand attack techniques and inflow paths. It also conducts comparative analyses of similar and related samples to classify malware and automatically generate detection rules. Through this, the AI identifies and infers threat actors' intentions, automates the prediction of follow-up attacks, and enhances proactive defense readiness. AhnLab's V3, V3 Mobile, and MDS utilize AI to detect and block smishing emails and malicious files, while AhnLab TIP employs AI to generate malware analysis reports and trend reports.

 

2) Data Augmentation

AI augments data by predicting and learning patterns and variations of new malware types and attack techniques. This strengthens organizations' proactive defense systems. Previously, organizations had to wait for malware samples to be collected, but now, AI automatically analyzes new samples as they are introduced and predicts variations. For example, AI predicts variations in phishing emails or attack scripts and incorporates them into its learning process. Additionally, AI identifies malicious commands within attack scripts and generates regular expressions to detect similar commands. This data augmentation technology is used for detecting phishing emails, smishing texts, and executable files.

 

3) AI-based Phishing Email Detection

Phishing emails are a primary entry point for cyberattacks, making their accurate detection and blocking crucial. AhnLab leverages AI to classify and block phishing emails accurately. AI analyzes various elements such as the email subject, body, sender, attachments, and URLs to comprehensively determine whether an email is a phishing attempt. Additionally, it provides a detailed explanation of why the email was identified as phishing, enabling security personnel to quickly understand and respond to the attack. In particular, AI can continuously improve phishing email detection accuracy by incorporating user feedback, enabling the effective blocking of various types of phishing emails received by companies and users.

 

4) Threat Intelligence Prediction

AI analyzes vast volumes of warning logs to detect abnormal patterns, enabling the proactive identification of attack indicators at an early stage. Processing the numerous warning logs generated by security solutions and monitoring devices is an extremely challenging task. AI automates the analysis of these logs, extracting critical patterns to predict the likelihood of a threat. This reduces false positives and highlights significant events that security personnel might otherwise overlook. The AI goes beyond simply collecting warnings; it comprehensively analyzes various events to identify potential risks, offering analysts well-organized information to support early attack prevention.

 

5) Threat Intelligence Summary

AI compiles and summarizes complex logs and behavior detection results to aid security personnel in understanding threats and proposing necessary actions. Threat detection information from security solutions like AhnLab EDR and AhnLab XDR can be vast and intricate, posing challenges for analysts to interpret. AhnLab's AI Security Assistant, AhnLab Annie AI, interprets detected behaviors and logs, then summarizes the background, root, processes, and defense systems of threats to provide analysts with this information, along with appropriate response recommendations.

 

6) Automated Incident Response

AI leverages prewritten playbooks and models trained on past incident data to propose optimal recovery and response methods. Additionally, it automates incident response reporting to enhance cyber resilience. For example, in the event of a security incident, AI isolates malware and the network, alerts the personnel in charge, and drafts an incident report. If the report is satisfactory, it finalizes the process; otherwise, it conducts a comprehensive investigation of the suspicious system. The security personnel only needs to review and approve each step. This automated response system helps security personnel focus on more critical decision-making.

 

Conclusion

The battle between attackers and defenders, armed with AI, has now begun. The ways in which attackers are abusing AI are evolving at an increasing pace. In response, security personnel are working to build more precise and powerful AI-based defense systems and enhance their ability to detect and respond to threats in real-time.

 

AI-based attacks are rapidly becoming more sophisticated and intelligent, which means that security systems must continuously update to keep pace. AhnLab is applying AI technology across its products to dramatically improve detection and analysis speed, enabling more accurate analysis of threat data, and helping security personnel respond proactively and efficiently. Through this, AhnLab is contributing to strengthening cyber resilience and creating a safer, more robust corporate environment.

 

We must not use AI merely as a tool for blocking attacks but develop it as a core technology to revolutionize the paradigm of cybersecurity. In the future security landscape, AI will play a crucial role in narrowing the gap between attackers and defenders, enabling the development of intelligent and evolving security systems.