MOSCOW, August 12 Users have learned to manipulate artificial intelligence, which is used in chatbots for searching, analyzing sites with answers to requests — they post special phrases on their sites so that neural networks perform certain actions, the «Laboratory» reported. Kaspersogo».
«Kaspersky Lab specialists studied open data and internal sources to find out how and why people use indirect prompt injection (indirect injections of seeding) — a cyber risk that many systems based on large language models (LLM) are exposed to. We are talking about text descriptions of tasks that chatbots must perform. … People can post special phrases — injections — on their websites and in documents published online so that neural networks give other users a response that takes into account the goals of the interested parties,» Kaspersky Lab said. 
Solutions based on large language models are used not only in chatbots, but also in search engines — AI helps to summarize the results for a user's query.
As Kaspersky Lab experts have found out, there are several areas in which users use such tricks. For example, «injections» are used to promote resumes among other profiles when searching for a job — the applicant writes instructions to the AI with a request to respond as positively as possible to the candidate, skip the resume to the next stage or give it a higher priority. The instructions are invisible to the recruiter, because they usually merge with the background of the page. However, neural networks that analyze resumes read these phrases.
Similar injections are used for advertising purposes: they are placed on websites of various products and services. The instructions are aimed at search chatbots — they are asked to give a more positive assessment of a specific product in responses to queries. Some users post instructions for neural networks to protest the widespread use of AI. For example, one Brazilian artist asked neural networks not to read, use, store, process, adapt or repeat certain content on his website.
«Today, the most important thing is to assess the potential risks of such cyberattacks. The creators of basic models (for example, GPT-4) use a variety of techniques to significantly increase the complexity of injections — from special training (as in the case of the latest model from OpenAI) to the creation of special models that can detect such attacks in advance (for example, from Google),» commented Vladislav Tushkanov, head of the machine learning technology research and development group.
He also noted that the cases of using «injections» detected by Kaspersky did not have malicious intent. At the moment, cyber threats such as phishing or data theft using «injections» are theoretical. «However, cybercriminals are also showing an active interest in neural networks. To protect existing and future solutions based on large language models, it is necessary to assess risks and study all possible methods of bypassing restrictions,» Tushkanov added.