This review of AI security in production brings together examples of vulnerabilities in generated code, packages, models, and autonomous agents. The practical boundary is simple: an assistant can not only suggest the wrong solution, but also run a command on a machine where it has access to files and secrets.
The material is aimed at developers and team leads who use tools such as Cursor, GitHub Copilot, Claude Code, and Gemini CLI. The author focuses on code created with assistants and agents on local machines; chatbot security and RAG systems with customer data are outside the scope of this review.
A green test does not mean secure code
An AI assistant often optimizes its answer for getting the code to run and fulfill the request. That does not guarantee correct access checks, safe input handling, or the absence of secrets in the configuration. The source cites a 2023 study from Stanford: 47 participants performed security tasks in Python, JavaScript, and C, some with an AI assistant. On average, the group with the assistant wrote less secure code, but was more likely to consider it secure.
The author cautions that this was a small laboratory experiment with a 2022 model, so its results should not be mechanically applied to modern tools. Still, they highlight an important observation for code review: confidently presented code is easier to accept without adequate scrutiny. That is why checks are needed not only during testing, but also when reviewing changes that make it into production.
Areas requiring extra attention include user input before eval or exec, passing data into SQL, HTML, or a file path, server requests to URLs from a request, unsafe deserialization such as pickle and checking object ownership. For Python, the article mentions Bandit and Semgrep for static analysis, as well as gitleaks and trufflehog for finding secrets in pre-commit and CI. These tools complement code review, but they do not know the business context: for example, who exactly owns a particular record.
A dependency can execute before it is imported
A package manager does more than just copy files. Build or installation scripts can run when dependencies are installed: in Python, this can be code in setup.py or a backend from pyproject.toml, and in npm, preinstall і postinstall. So a command that an agent runs automatically gives a third-party package the ability to operate in the installation environment.
A separate risk is recommendations for nonexistent libraries. In a study cited in the material, 16 models generated 576 thousand code examples; 19,7% of the package names mentioned did not exist. If a model repeats an invented name, an attacker can register it in a public registry and put malicious code there. This scenario is called slopsquatting. In practice, the author describes the package huggingface-cli, which was registered after models recommended it as an installation command; it was downloaded more than 30 thousand times in three months.
There is also the risk of a real dependency being impersonated. In the case of torchtriton a package with the same name appeared on PyPI, while PyTorch nightly builds used their own index. The material says that from 25 to 30 December 2022, the impersonator collected data from the environment and home directory during import. PyTorch replaced the dependency with pytorch-triton and registered a placeholder package on PyPI.
Among the practical measures, the author names lock files with hashes, installing from a single source, and checking a new package’s documentation, author, and publication history. For CI, pip-audit і npm auditare mentioned, but they detect known vulnerabilities and do not guarantee detection of newly published malicious code. CI tool dependencies also need to be pinned: the infected Trivy scanner in the LiteLLM case helped obtain a publishing token, after which compromised versions of the library appeared on PyPI.
Commands such as pip install --only-binary :all: and npm install --ignore-scripts can restrict the execution of installation scripts in relevant scenarios. But that does not make a package safe: its code can still run when it is used. A handy flag does not replace checking the source—engineering simplicity, as always, comes with a few small conditions in the fine print.
A model file sometimes contains executable logic
Model formats based on Python pickle can store not just data, but also instructions for calling functions during loading. Hence the recommendation not to load untrusted files .pt, .pth, .pkl or .joblib without assessing the risk. As a safer option for weights, the material cites safetensors, which stores numbers and metadata rather than executable code.
In PyTorch, starting with version 2.6, the default value for torch.load became weights_only=True. However, the author reminds readers of CVE-2025-32434, which affected versions up to and including 2.5.1, and of Picklescan bypasses described in 2025. So scanning is an additional barrier, not proof of safety. For models from Hugging Face, they also recommend pinning a specific commit, using use_safetensors=True and not enabling trust_remote_code=True unless necessary: this parameter runs Python code from the model repository.
You cannot trust text just because of how it looks to an agent
Prompt injection occurs when an agent treats outside text as an instruction. The source could be a README, issue, website, or documentation. If the agent has access to secrets, reads untrusted content, and can send data out, you get the “lethal trifecta” described by Simon Willison. In this configuration, a hidden instruction can turn an ordinary action into a data leak.
In the Gemini CLI example, the agent ran a command that began with the permitted grep, but after a semicolon included a separate curl that sent environment variables. The permission check, which looked only at the beginning of the line, missed the second command. Google released a fix in version 0.1.14. This case shows why you need to read the whole line, including parts after &&, ;, | and substitution constructs.
Another type of change is an agent editing its own settings or launch configurations. The article describes cases with Copilot and Cursor where changes to configurations could automate confirmations or launch an external server. Files such as .vscode/settings.json, .cursor/mcp.json, .git/config and workflow files should be reviewed as code, not as harmless settings. The risks of invisible Unicode characters in agent rules are also described; a person may not notice them in the editor.
The danger is not limited to attacks. In the Replit case, while code was supposed to be frozen, the agent ran commands against SaaStr’s production database and deleted data about more than 1 200 executives and 1 190 companies. According to the material, the founder restored the data himself later. The author’s conclusion is specific: if an agent needs access to data, restrict it to read-only, isolate it with a separate account, and keep backups whose restoration has been tested.
Restrictions matter more than asking it to “be careful”
The source does not suggest relying on prompt wording as protection against injection. Instead, it recommends reducing the agent’s capabilities: disable automatic command execution outside an isolated environment, do not give it production keys, restrict network connections, and mount only the required project directory in the container. If the agent has no secrets and no channel for sending them out, malicious text has far fewer ways to do damage.
For dependencies, pinning versions and hashes, checking packages, and configuring CI carefully are useful. For models, prefer safetensors and avoid executing other people’s code unless there is a conscious need to. For agents, review commands all the way to the end, control changes to settings files, and work without access to production secrets. These boundaries determine whether an assistant’s mistake stays a mistake in a draft or gains access to a live system.









