Checklists 🇮🇳 Indian Mandate Aligned 16 min read

AI Security Threats & Defenses: 20 Key Lifecycle Threats (OWASP LLM & Adversarial ML Guide)

The authoritative technical guide analyzing 20 distinct threats across the enterprise Artificial Intelligence lifecycle—from pre-training dataset poisoning and Hugging Face supply chain backdoors to direct prompt injection, RAG memory tampering, and autonomous tool-call hijacking. Aligned with OWASP Top 10 for LLMs and ISO/IEC 42001.

By BlueCiphr Threat Research Desk • Lead CEH & PenTest+ Auditors

📥 Download Offline Artifact: Audit-Ready Formats

[1. Data & Pre-training] Validate hash integrity of all pre-trained weights and Hugging Face repositories using safe tensors (safetensors format only).

Mandate: AI Supply Chain Standard

[1. Data & Pre-training] Scan fine-tuning training datasets for embedded canary tokens and PII to prevent training data extraction.

Mandate: DPDP Act PII Safeguard

[2. Prompt & Inference] Deploy pre-inference semantic guardrails (e.g. NeMo Guardrails, Llama Guard) to filter jailbreak patterns and token smuggling.

Mandate: OWASP LLM01

[2. Prompt & Inference] Enforce strict separation between system instructions, contextual documents, and untrusted user prompts using XML delimiters.

Mandate: OWASP LLM01

[3. RAG Security] Sanitize and strip executable markdown, HTML, and script payloads from external documents prior to chunking and vector indexing.

Mandate: OWASP LLM02

[3. RAG Security] Enforce document-level access control within vector databases ensuring users only retrieve chunks they are authorized to access.

Mandate: BOLA Prevention in AI

[4. Output & Tool Calling] Treat all LLM output as untrusted; encode output before rendering in web browsers to eliminate Cross-Site Scripting (XSS).

Mandate: OWASP LLM02

[4. Tool Calling] Implement deterministic schema validation on function-call arguments before executing database queries or microservice calls.

Mandate: OWASP LLM06

[4. Tool Calling] Enforce Human-in-the-Loop (HITL) mandatory approval gates for irreversible actions (financial transfers, account deletions).

Mandate: Agentic AI Safety

[5. Resource Protection] Enforce strict token usage caps, rate-limiting, and timeout limits on LLM API endpoints to prevent Denial of Wallet (DoW).

Mandate: OWASP LLM04

[6. Monitoring & Audit] Log full prompt-completion transactions to an encrypted, tamper-evident audit store for post-incident forensic investigation.

Mandate: ISO 42001 Control A.9

[6. Red Teaming] Conduct regular multi-turn automated and manual red teaming targeting model hallucination, bias, and security guardrail bypasses.

Mandate: Frontier Model Audit

🤖 AI Security Taxonomy: 20 Critical AI Lifecycle Threats (OWASP Top 10 for LLMs)

Artificial Intelligence systems introduce non-deterministic attack surfaces that traditional firewalls and WAFs cannot detect. This taxonomy details the 20 primary threats across the end-to-end model lifecycle.

The 20 AI Threats Across 4 Operational Domains

Domain A: Supply Chain & Training (Threats 1–5)

1. Pre-training Data Poisoning (inserting backdoors into datasets)
2. Compromised Model Weights (pickled Python code execution)
3. Dependency Supply Chain (malicious packages in PyTorch/Transformers)
4. Training Data PII Memorization & Inversion
5. Proprietary IP Theft via Model Distillation

Domain B: Prompt Injection & Evasion (Threats 6–10)

6. Direct Prompt Injection (Jailbreaking system instructions)
7. Indirect Prompt Injection (Malicious text in ingested PDFs/URLs)
8. Multi-turn Semantic Roleplay & Cognitive Pressure
9. Token Smuggling (Base64, Unicode, and cipher obfuscation)
10. System Prompt Disclosure (Extracting proprietary instructions)

Domain C: Architecture & RAG Exploits (Threats 11–15)

11. Vector Database Chunk Poisoning
12. Cross-Tenant Information Leakage in Shared Embeddings
13. Denial of Wallet (Unbounded recursive generation)
14. Sponge Examples (Inputs engineered to maximize GPU latency)
15. Model Inversion via Repeated Query Analysis

Domain D: Agentic Action & Tool Exploits (Threats 16–20)

16. Excessive Agency & Unbounded Tool Permissions
17. Insecure Output Execution (RCE via LLM-generated code)
18. Tool-Call Parameter Tampering via Hallucination
19. Memory Poisoning Across Persistent Autonomous Sessions
20. Bypassing External Authentication via Agent Delegation

🛡️ AUDIT DEFENSE & TECHNICAL VERIFICATION

Request Audit Scoping & Consultation → ⚡ Quick Response SLA • Board-Ready Evidence

AI Security Threats & Defenses: 20 Key Lifecycle Threats (OWASP LLM & Adversarial ML Guide)

Interactive Technical Checklist (12 Verification Checkpoints)

The 20 AI Threats Across 4 Operational Domains

Need Accredited Auditors to Execute This Audit or Gap Assessment?