LLMs Infiltrate Your Stack: New Risks at Every Layer

The integration of large language models into enterprise technology stacks introduces a complex array of security challenges that demand a fundamental shift in how organizations approach risk. As these AI systems become embedded in core products and workflows, security leaders face pressure to adapt traditional models to address novel vulnerabilities across data handling, application behavior, and internal trust boundaries. A new framework, structured around the OWASP Top 10 for LLM Applications, provides a comprehensive risk model and reference architecture to guide teams through this evolving landscape.

One of the most significant hurdles is not purely technical but cultural. Teams frequently carry habits from experimental phases into production environments. The essential mindset shift involves unlearning the idea that the model is an intelligent brain and instead treating it as untrusted compute. This foundational change is critical for aligning engineering practices with a robust trust boundary model, ensuring security is designed into the system from the start.

Prompt injection remains a primary concern, as models can inadvertently follow malicious instructions hidden within user inputs or retrieved content. Closely related is the risk of sensitive information disclosure, where prompts, documents, outputs, logs, or even the model providers themselves can leak confidential data. The guiding principle is that both inputs and outputs must be considered untrusted until they have been properly validated. This approach is often hindered by cultural inertia, where teams may rely too heavily on prompt engineering or fine-tuning instead of enforcing strict schemas and policies through code.

Improper output handling presents a parallel challenge. Model responses can generate unsafe HTML, malformed JSON, unexpected URLs, or text that downstream systems might mistakenly execute. This effectively turns the model’s output into a new form of untrusted input, necessitating rigorous inspection, sanitization, and structural enforcement before any data is passed along.

The expanding ecosystem surrounding LLMs introduces substantial supply chain exposure. Risks now extend to model formats, servers, third-party hubs, connectors, and agent frameworks, where vulnerabilities like unsafe serialization, remote code execution flaws, and malicious dependencies can lurk. Similarly, data and model poisoning has become a tangible threat. Even small amounts of manipulated data can skew training, fine-tuning, or retrieval-augmented generation systems. A handful of poisoned documents can distort retrieval results or plant hidden triggers, making robust data provenance, versioning, and quarantine processes essential components of any secure pipeline.

Operational risks are magnified as teams adopt more autonomous, agentic patterns. Excessive agency becomes an architectural issue when a model can call tools or execute workflows; any flaw in validation or privilege design can lead to unintended and potentially harmful actions. Implementing hard boundaries, the principle of least privilege, and strict mediation of all tool calls is vital. Another common pitfall is system prompt leakage, where hidden instructions containing internal logic, secrets, or policy details are exposed, providing attackers with valuable intelligence. Such information should be moved out of prompts and into secure code or middleware.

For teams utilizing retrieval systems, vector and embedding weaknesses pose a distinct threat. Misconfigured vector stores, cross-tenant data exposure, and poisoning of the index can all compromise both the quality and security of retrieval. Defenses require strong isolation, filtering at both ingestion and retrieval points, and treating embeddings derived from sensitive content as sensitive data themselves.

The final categories of risk relate directly to product behavior and resource management. Misinformation refers to models generating incorrect or unsubstantiated answers that users may accept as fact. Mitigating this is a design problem; systems should be engineered to ground responses in retrieved context, provide supporting evidence, or decline to answer when certainty is lacking. Uncontrolled consumption covers scenarios like runaway token use, endless reasoning loops, and unbounded retries, which can skyrocket costs, degrade system availability, and trigger rate-limit failures. Implementing token-aware rate limits, strict quotas, defined retry rules, and agent action limits are recommended controls.

A practical reference architecture addresses these risks by distributing controls across policy layers, orchestrators, tool proxies, data stores, and observability systems. This design treats the LLM as untrusted compute and establishes every boundary as a critical control point. For security leaders, this provides a structured method to map each OWASP risk to the specific architectural layer responsible for mitigating it.

Organizations that standardize on such a reference architecture and centralize the management of trust boundaries under a dedicated platform or application security group find the transition significantly smoother. This centralized approach prevents individual teams from repeating the same security mistakes and emphasizes the need for continuous verification in code to ensure the deployed architecture matches its intended design.

For teams that cannot implement a full architecture immediately, the most critical first steps involve establishing two key defenses: implementing preventive security checks directly in the code that interfaces with LLMs and deploying a robust policy or guardrail layer in front of every model instance. These two layers represent the fastest and most effective way to reduce both the likelihood and potential impact of the most serious security failures in AI-driven systems.

(Source: HelpNet Security)