Patterns/Emerging Patterns

Your Tools May Be Compromised

Why the LLM ecosystem's dependencies create attack surfaces you don't control

The Conventional Framing

LLM applications depend on many components: base models, embedding models, vector databases, frameworks, plugins, and third-party tools. Each is a potential supply chain risk.

Supply chain security is a general concern that applies to LLM systems.

Why LLM Supply Chains Are Especially Risky

The LLM ecosystem is young and fast-moving. Many components are new, rapidly evolving, and haven't received extensive security scrutiny. The rush to ship AI features often bypasses careful dependency review.

Models themselves are supply chain components. A compromised embedding model, a poisoned fine-tune, or a backdoored vector database all affect your system.

The trust stack:

Base model provider → fine-tuning service → embedding model → vector DB → framework → plugins → your code. Each link is a trust assumption.

Architecture

Components:

  • Base models— foundation models from providers
  • Embedding models— models for vectorization
  • Vector databases— storage and retrieval systems
  • Frameworks— LangChain, LlamaIndex, etc.
  • Plugins/Tools— third-party integrations

Trust Boundaries

Your application depends on: Base Model: Trust OpenAI/Anthropic/etc.? Embedding Model: From HuggingFace? Who trained it? Vector DB: Hosted service? What access do they have? Framework: LangChain version? Dependencies? Vulnerabilities? Plugins: Community-maintained? Last audit: never? Each is a trust assumption. Each is an attack surface you don't control.
  1. Provider → Model — trust in model providers
  2. Open source → Your system — trust in community code
  3. Hosted services → Your data — trust in SaaS providers

Threat Surface

ThreatVectorImpact
Compromised modelsBackdoored or poisoned models from providersBuilt-in malicious behavior from the start
Malicious packagesTrojanized LLM framework packagesCode execution through dependency
Hosted service compromiseAttack on vector DB or model hosting providerData exposure or system compromise
Plugin vulnerabilitiesSecurity flaws in third-party toolsAttack via vulnerable integration

The ZIVIS Position

  • •
    Inventory your dependencies.Know what models, services, and packages you depend on. You can't secure what you don't know about.
  • •
    Vet providers carefully.Model providers, embedding services, vector DBs—all are critical dependencies. Evaluate their security posture.
  • •
    Pin versions, audit updates.Don't auto-update dependencies in production. Review changes before deploying new versions.
  • •
    Assume some compromise.Design for defense in depth. If one component is compromised, limit the blast radius.

What We Tell Clients

LLM supply chains are young and fast-moving with many trust assumptions. Each model provider, embedding service, framework, and plugin is a dependency you're trusting.

Inventory your dependencies, vet providers, pin versions, and design for defense in depth. The ecosystem moves faster than security review.

Related Patterns