Impact of Generative AI on Cybersecurity Risks, Phishing, and Digital Defense in the US

Authors

  • Muhammad Imran Department of Information Technology, Washington University of Science and Technology, Alexandria, Virginia, USA Author

DOI:

https://doi.org/10.63544/0mn3qq49

Keywords:

Generative AI, Large Language Models, Phishing, Business Email Compromise, Cybersecurity Risk, Email Security, Digital Defense, United States

Abstract

The rapid advancement of generative artificial intelligence (GenAI) has reduced the expenses of creating natural, personalized and context-relevant content, thus challenging defenses based on the linguistic weaknesses of traditional phishing. This paper explores the effects of GenAI on cybersecurity risk and phishing and digital defense in the United States with a structured synthesis of 23 studies and a controlled detection experiment. The synthesis is structured around offensive and defensive applications of GenAI. The experiment evaluates six different detectors (keyword rules, word and character n-gram classifiers, header/URL metadata model, metadata-plus-intent model, and fused model) against traditional phishing, GenAI-style phishing, and GenAI phishing from compromised accounts. Pre-GenAI mail was trained on the detectors, which were then evaluated at a 0.5% false-positive rate, using five different seeds. GenAI-style messages did not include any misspellings or classic lure words and were lexically nearer to legitimate messages (median cosine similarity 0.22 compared to 0.13 for traditional phishing). None of them were detected by keyword rules and the word-level classifiers dropped from 1.0 on traditional phishing to 0.508. Metadata was effective in detecting 97% of GenAI phishing messages originating from attacker infrastructure, but just 3.6% of messages originating from compromised accounts. Adding 1,500 defender-generated examples of phishing using GenAI improved the results of fused-model recall from 0.149 to 0.790 on the compromised-account case with no corresponding increase in false positives. The results suggest that GenAI has transformed the phishing threat from content to identity and context; thus, U.S. organizations need to implement email authentication, defender-side generative augmentation and continuous retraining, and account-takeover controls.

REFERENCES

[1] S. Neupane, I. A. Fernandez, S. Mittal, and S. Rahimi, "Impacts and risk of generative AI technology on cyber defense," 2023, arXiv:2306.13033.

[2] F. Iqbal, F. Samsom, F. Kamoun, and Á. MacDermott, "When ChatGPT goes rogue: Exploring the potential cybersecurity threats of AI-powered conversational chatbots," Front. Commun. Netw., vol. 4, Art. no. 1220243, 2023.

[3] C. Barrett et al., "Identifying and mitigating the security risks of generative AI," Found. Trends Privacy Secur., vol. 6, no. 1, pp. 1–52, 2023.

[4] L. A. Bauer and V. Bindschaedler, "Generative models for security: Attacks, defenses, and opportunities," 2021, arXiv:2107.10139.

[5] O. O. Aramide, "AI-driven cybersecurity: The double-edged sword of automation and adversarial threats," Int. J. Humanities Inf. Technol., vol. 4, no. 4, pp. 19–38, 2022.

[6] A. Kucharavy et al., "Fundamentals of generative large language models and perspectives in cyber-defense," 2023, arXiv:2303.12132.

[7] E. Areghan, "From data breaches to deepfakes: A comprehensive review of evolving cyber threats and online risk management," Commun. Phys. Sci., vol. 9, no. 4, pp. 538–553, 2023.

[8] L. Shrestha, H. Balogun, and S. Khan, "AI-driven phishing: Techniques, threats, and defence strategies," in Proc. Int. Conf. Global Secur., Safety, Sustainability. Cham, Switzerland: Springer, Oct. 2023, pp. 121–143.

[9] L. Xu, E. Qereshniku, H. Hazari, and M. Edwards, "Understanding national security threats enabled by artificial intelligence: Implications for CSIS," Artif. Intell., vol. 101, no. 9, 2023.

[10] P. S. Dhoni and R. Kumar, "Synergizing generative artificial intelligence and cybersecurity: Roles of generative artificial intelligence entities, companies, agencies and government in enhancing cybersecurity," J. Global Res. Comput. Sci., vol. 14, no. 3, pp. 1–16, 2023.

[11] S. Oh and T. Shon, "Cybersecurity issues in generative AI," in Proc. Int. Conf. Platform Technol. Service (PlatCon), Aug. 2023, pp. 97–100.

[12] F. Teichmann, "Ransomware attacks in the context of generative artificial intelligence—An experimental study," Int. Cybersecur. Law Rev., vol. 4, no. 4, pp. 399–414, 2023.

[13] W. H. Elashmawi, H. Osman, M. Osama, and N. Nader, "Development generative AI for cybersecurity: Evaluating script generation and attack classification in penetration testing," in Proc. Int. Workshop Adv. Inf. Secur. Manage. Appl. Cham, Switzerland: Springer, Sep. 2023, pp. 48–62.

[14] A. Badmus and M. E. Adebayo, "Compliance-aware DevOps for generative AI: Integrating legal risk management, data controls, and model governance to mitigate deepfake and data privacy risks in synthetic media deployment," Scholars Int. J. Law, Crime Justice, vol. 3, no. 12, pp. 468–478, 2020.

[15] O. Krishnamurthy, "Enhancing cyber security enhancement through generative AI," Int. J. Universal Sci. Eng., vol. 9, no. 1, pp. 35–50, 2023.

[16] T. I. Adeyinka and K. I. Adeyinka, "Leveraging generative AI for automated cyber threat simulation and response frameworks," SSRN, 2023, Paper 5334064.

[17] N. Batel, Q. Zhou, and P. Wagner, "Generative AI-assisted cyber threat intelligence analysis and automated incident response," J. Adv. Artif. Intell. Res., vol. 2, no. 1, 2023.

[18] K. Kumar and B. Bhushan, "Augmenting cybersecurity and fraud detection using artificial intelligence advancements," in Proc. Int. Conf. Comput., Commun., Intell. Syst. (ICCCIS), Nov. 2023, pp. 1207–1212.

[19] M. Muthusamy, "Generative AI–powered cloud and machine learning architectures for digital privacy and risk management in banking and trade systems over 5G networks," Int. J. Adv. Res. Comput. Sci. Technol., vol. 6, no. 4, pp. 8681–8690, 2023.

[20] A. Svendsen and B. Garvey, "Prompt-engineering testing ChatGPT4 and Bard for assessing generative-AI efficacy to support decision-making," SSRN, 2023, Paper 4495320.

[21] I. H. Sarker, "Multi-aspects AI-based modeling and adversarial learning for cybersecurity intelligence and robustness: A comprehensive overview," Secur. Privacy, vol. 6, no. 5, Art. no. e295, 2023.

[22] T. C. Truong, Q. B. Diep, and I. Zelinka, "Artificial intelligence in the cyber domain: Offense and defense," Symmetry, vol. 12, no. 3, Art. no. 410, 2020.

[23] M. E. Bonfanti, "Artificial intelligence and the offence–defence balance in cyber security," in Cyber Security: Socio-Technological Uncertainty and Political Fragmentation. London, U.K.: Routledge, 2022, pp. 64–79.

Downloads

Published

30-06-2024

How to Cite

Impact of Generative AI on Cybersecurity Risks, Phishing, and Digital Defense in the US. (2024). Journal of Engineering and Computational Intelligence Review, 2(1), 63-73. https://doi.org/10.63544/0mn3qq49

Share

Similar Articles

11-20 of 80

You may also start an advanced similarity search for this article.