Misuse of large language models: Exploiting weaknesses for target-specific outputs
DOI:
https://doi.org/10.14512/tatup.33.2.29Keywords:
prompt engineering, jailbreak prompts, transformers, deep learning, large language modelsAbstract
Prompt engineering in large language models (LLMs) in combination with external context can be misused for jailbreaks in order to generate malicious outputs. In the process, jailbreak prompts are apparently amplified in such a way that LLMs can generate malicious outputs on a large scale despite their initial training. As social bots, these can contribute to the dissemination of misinformation, hate speech, and discriminatory content. Using GPT4-x-Vicuna-13b-4bit from NousResearch, we demonstrate in this article the effectiveness of jailbreak prompts and external contexts via Jupyter Notebook based on the Python programming language. In addition, we highlight the methodological foundations of prompt engineering and its potential to create malicious content in order to sensitize researchers, practitioners, and policymakers to the importance of responsible development and deployment of LLMs.
References
Agrawal, Sweta; Zhou, Chunting; Lewis, Mike; Zettlemoyer, Luke; Ghazvininejad, Marjan (2023): In-context examples selection for machine translation. In: arxiv.org, 05. 12. 2022. https://doi.org/10.48550/arXiv.2212.02437 DOI: https://doi.org/10.18653/v1/2023.findings-acl.564
Arora, Simran et al. (2023): Ask me anything. A simple strategy for prompting language models. In: arxiv.org, 05. 10. 2022. https://doi.org/10.48550/arXiv.2210.02441
Ba, Jimmy; Kiros, Jamie; Hinton, Geoffrey (2016): Layer normalization. In: arxiv.org, 21. 06. 2016. https://doi.org/10.48550/arXiv.1607.06450
Birhane, Adeba; Kasirzadeh, Atoosa; Leslie, David; Wachter, Sandra (2023): Science in the age of large language models. In: Nature Reviews Physics 5 (5), pp. 277–280. https://doi.org/10.1038/s42254-023-00581-4 DOI: https://doi.org/10.1038/s42254-023-00581-4
Chen, Canyu; Shu, Kai (2023): Can LLM-generated misinformation be detected? In: arxiv.org, 25. 09. 2023. https://doi.org/10.48550/arXiv.2309.13788
Chiang, David; Cholak, Peter; Pillay, Anand (2023): Tighter bounds on the expressivity of transformer encoders. In: arxiv.org, 01. 06. 2023. https://doi.org/10.48550/arXiv.2301.10743
Douglas, Michael (2023): Large language models. In: arxiv.org, 25. 01. 2023. https://doi.org/10.48550/arXiv.2307.05782
Edelman, Benjamin; Goel, Surbhi; Kakade, Sham; Zhang, Cyril (2021): Inductive biases and variable creation in self-attention mechanisms. In: arxiv.org, 19. 10. 2021. https://doi.org/10.48550/arXiv.2110.10090
EU – European Union (2024): AI act. Available online at https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai, last accessed on 26. 04. 2024.
Friedman, Dan; Wettig, Alexander; Chen, Danqi (2023): Learning transformer programs. In: arxiv.org, 01. 06. 2023. https://doi.org/10.48550/arXiv.2306.01128
Garg, Shivam; Tsipras, Dimitris; Liang, Percy; Valiant, Gregory (2022): What can transformers learn in-context. A case study of simple function classes. In: arxiv.org, 01. 08. 2022. https://doi.org/10.48550/arXiv.2208.01066
Graves, Alex (2014): Generating sequences with recurrent neural networks. In: arxiv.org, 04. 08. 2013. https://doi.org/10.48550/arXiv.1308.0850
Gudibande, Arnav et al. (2023): The false promise of imitating proprietary LLMs. In: arxiv.org, 25. 05. 2023. https://doi.org/10.48550/arXiv.2305.15717
Hsieh, Cheng-Yu; Lee, Chen-Yu (2023): Distilling step-by-step. Outperforming larger language models with less training data and smaller model sizes, 21. 09. 2023. Available online at https://blog.research.google/2023/09/distilling-step-by-step-outperforming.html, last accessed on 26. 04. 2024. DOI: https://doi.org/10.18653/v1/2023.findings-acl.507
Karanjay, Rabimba (2023): Targeted phishing campaigns using large scale language models. In: arxiv.org, 30. 12. 2022. https://doi.org/10.48550/arXiv.2301.00665
Kossen, Jannik; Rainforth, Tom; Gal, Yarin (2023): In-context learning in LLMs learns label relationships but is not conventional learning. In: arxiv.org, 23. 07. 2023. https://doi.org/10.48550/arXiv.2307.12375
Learn Prompting (2023): Prompt hacking. Jailbreaking. Available online at https://learnprompting.org/de/docs/prompt_hacking/jailbreaking, last accessed on 26. 04. 2024.
Mahowald, Kyle; Ivanova, Anna; Blank, Idan; Kanwisher, Nancy; Tenenbaum, Joshua; Fedorenko, Evelina (2023): Dissociating language and thought in LLMs. A cognitive perspective. In: arxiv.org, 16. 01. 2023. https://doi.org/10.48550/arXiv.2301.06627
Minhyeok, Lee (2023): A mathematical investigation of hallucination and creativity in GPT models. In: MDPI 11 (10), pp. 1–17. https://doi.org/10.3390/math11102320 DOI: https://doi.org/10.3390/math11102320
NOMIC.AI (2024): GPT4All. Available online at https://gpt4all.io, last accessed on 26. 04. 2024.
OpenAI (2022): Lessons learned on language model safety and misuse. Available online at https://openai.com/research/language-model-safety-and-misuse, last accessed on 26. 04. 2024.
Peng, Baolin et al. (2023): Check your facts and try again. Improving LLMs with external knowledge and automated feedback. In: arxiv.org, 24. 02. 2023. https://doi.org/10.48550/arXiv.2302.12813
Shen, Xinyue; Chen, Zeyuan; Backes, Michael; Shen, Yun; Zhang, Yang (2023): Do anything now. Characterizing and evaluating in-the-wild jailbreak prompts on LLMs. In: arxiv.org, 07. 08. 2023. https://doi.org/10.48550/arXiv.2308.03825
Strauß, Stefan (2021): Don’t let me be misunderstood. Critical AI literacy for the constructive use of AI technology. In: TATuP – Journal for Technology Assessment in Theory and Praxis 30 (3), pp. 44–49. https://doi.org/10.14512/tatup.30.3.44 DOI: https://doi.org/10.14512/tatup.30.3.44
Sutskever, Ilya; Vinyals, Oriol; Le, Quoc (2014): Sequence to sequence learning with neural networks. In: arxiv.org, 10. 09. 2014. https://doi.org/10.48550/arXiv.1409.3215
Vaswani, Ashish et al. (2017): Attention is all you need. In: arxiv.org, 12. 06. 2017. https://doi.org/10.48550/arXiv.1706.03762
Warstadt, Alex; Bowman, Samuel (2022): What artificial neural networks can tell us about human language acquisition. In: arxiv.org, 17. 08. 2022. https://doi.org/10.48550/arXiv.2208.07998 DOI: https://doi.org/10.1201/9781003205388-2
White, Jules et al. (2023): A prompt pattern catalog to enhance prompt engineering with ChatGPT. In: arxiv.org, 21. 02. 2023. https://doi.org/10.48550/arXiv.2302.11382
Xu, Jiashu; Ma, Mingyu; Wang, Fei; Xiao, Chaowei; Chen, Muhao (2023): Instructions as backdoors. Backdoor vulnerabilities of instruction tuning for LLMs. In: arxiv.org, 24. 05. 2023. https://doi.org/10.48550/arXiv.2305.14710
Yang, Kai-Cheng; Menczer, Filippo (2023): Anatomy of an AI‑powered malicious social botnet. In: arxiv.org. https://doi.org/10.48550/arXiv.2307.16336 DOI: https://doi.org/10.51685/jqd.2024.icwsm.7
Zhong, Zexuan; Lei, Tao; Chen, Danqi (2022): Training language models with memory augmentation. In: arxiv.org, 25. 05. 2022. https://doi.org/10.48550/arXiv.2205.12674 DOI: https://doi.org/10.18653/v1/2022.emnlp-main.382
Published
Issue
Section
License
Copyright (c) 2024 Dennis Klinkhammer
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
