УЦСБ has published the second part of its overview of AI system security, devoted to techniques for attacking large language models. The practical takeaway for teams embedding LLMs in products: checking only the chat field is not enough — the data the model reads and the tools it uses can also be attacked.
The article focuses on LLM applications because they are increasingly being connected to business processes and given agentic capabilities. The author distinguishes this class of systems from traditional machine learning and computer vision models: they have their own attack specifics, so the article considers them separately. The overview’s main thread runs from manual attempts to persuade a model to automated schemes in which one LLM selects prompts for another.
You can attack more than just the model
An LLM is a prominent part of an application, but it is not the whole attack surface. If a system uses RAG, an attacker can try to influence the documents it draws context from. An instruction can be hidden in a webpage, email, PDF, image, or audio file that the model processes at a user’s request.
There is also the separate risk of poisoning the RAG corpus: a malicious document is added to the data source, and the system may then return it to different users. Such a document may sometimes contain an instruction for the model, and sometimes simply spread false information. The first case also involves indirect prompt injection; in the second, the target is the data, not how instructions are interpreted.
When an agent has access to external tools, successfully influencing the model can potentially turn a response into an action. That is why privilege boundaries and isolation of integrations matter. Meanwhile, ordinary infrastructure vulnerabilities are still with us: the author mentions the compromise of the Ultralytics YOLO library through fake pull requests, credential theft through LiteLLM, and an incident involving the postmark-mcp MCP server, whose excessive privileges made it possible to act under a developer’s account. So LLM application security is not just about having a good system prompt.
Three axes of classification
Attacks can be broken down by delivery method, target, and creation method. Direct prompt injection comes from a user who enters a malicious instruction themselves. Indirect injection uses external material that the system reads, even though the user may not know it contains content hidden from them.
By purpose, the author identifies the generation of harmful content, execution of unwanted actions, and data theft. An attempt to bypass a model’s restrictions to obtain a prohibited answer is called a jailbreak. The target of a leak may be the system prompt, confidential data from RAG, or information that made its way into the model’s weights during training.
Finally, attacks can be manual or automated. The manual approach relies on the operator coming up with the wording, checking the response, and changing the prompt themselves. Automation makes it possible to try variations faster and cover more scenarios. Some optimization methods require access to the target model’s weights, while others use a separate attacking LLM and interact with the target as a black box.
From encodings to role-play scenarios
Manual techniques include so-called token smuggling: the meaning of a prompt is disguised using encodings, ciphers, transliteration, or lookalike characters. The aim is to get past a filter that reacts to specific words or patterns. But the model has to understand that representation: if its training did not cover the required formats, the result may be an incoherent response or a hallucination. Even with modern models, success is not guaranteed, so unusual encoding by itself does not mean an automatic bypass.
Another well-known group is DAN and role-play attacks. The prompt is framed as a scenario in which the model is supposedly meant to play a character that ignores the usual restrictions. The examples in the overview use fictional game rules or pressure on the model within such a scenario. These methods require manual work, and developers may eventually introduce countermeasures against familiar templates.
Persuasive Adversarial Prompts, or PAP, rely not on encoding but on natural language. An attacker may frame a prompt as research, appeal to authority or emotions, create a sense of urgency, or gradually move from a safe topic to a risky one. УЦСБ recounts the classification by researchers at EPFL and Yale, which includes 13 tactics and 40 persuasion techniques. It is precisely the ordinary appearance of such wording that makes it harder to distinguish from legitimate conversation.
The article also cites research according to which GPT-4 was more vulnerable to persuasive attacks than GPT-3.5 at the time of that study. The author suggests that a greater ability to handle linguistic nuances may also expand the possibilities for manipulating those nuances. This is the result of a specific study, not a universal ranking of all models.
Optimization and “LLM versus LLM” attacks
GCG, or Greedy Coordinate Gradient, selects an adversarial suffix using gradient optimization. The method uses the model’s internal data, so the classic version requires access to its weights. The sequence found can be tried on other models, including closed ones, but the effectiveness of such transfer should not be taken for granted. For systems accessible only through an API, direct use of classic GCG is limited.
The name AutoDAN refers to different works. One approach automates the creation of readable attacks and is related to earlier optimization methods; another work uses a hierarchical genetic algorithm without access to the weights. Readability may help against filters that reject incoherent suffixes, but it creates a trade-off: the most effective sequence for a bypass will not necessarily be understandable text.
The PAIR and TAP methods treat the target model as a black box. In PAIR, an attacking LLM creates a prompt, the target responds, and a separate judge model evaluates the result. Based on the evaluation, the attacking model changes its next attempt. The quality of the conclusion depends on the judge: if it misidentifies a refusal or a success, the attack evaluation will also be inaccurate. The basic scheme described in the article does not retain previous prompts as context for the target model.
TAP develops the idea of iterative selection: the attacking system creates several branches of prompts, and an evaluator prunes the weaker ones before sending the rest to the target. According to the study’s description, this reduces the number of unpromising requests to the target model. The comparison with GCG and PAIR in the paper was conducted on specific models: GCG showed better results when weights were accessible and there was no external protection, while TAP had a higher rate of successful attacks on closed models in that testing.
AutoDAN-Turbo adds memory of previous results to the loop. The system has an attacking and a target LLM, an evaluator, a summarizer, and a strategy library. The summarizer analyzes more successful prompts and adds the patterns it finds to the library; during subsequent attempts, the system looks for strategies similar to those that worked for a particular type of refusal. This changes the scale of the operation: automation can accumulate knowledge about approaches, rather than just generate individual variations.
For teams developing such systems, the practical minimum from the article is to assess risks, follow the principle of least privilege, and test resilience to manual attacks. The author suggests taking a list of prohibited topics and trying different ways of phrasing them; automated red teaming can be added later. It is also important to limit agents’ permissions, isolate tools, and control input and output data. Otherwise, the model may refuse in chat, while integrations still remain a place where a mistake has practical consequences. Setting up only a prompt filter and considering the job done is a convenient shortcut, but it does not remove the attack surface.









