Guardian LLM is an academic-grade, rule-based web application vulnerability scanner and Large Language Model (LLM) security evaluation framework. Designed to showcase hands-on security engineering and clean code authorship, its core analysis engines are built strictly with Python standard libraries (urllib, re, html.parser, json, csv, argparse).
Guardian LLM unifies classical web application vulnerability detection with modern generative AI security probing under a cohesive, decoupled architecture:
guardian-llm/
β
βββ scanner/ # Rule-based Web Analysis Engine (Python Standard Library)
β βββ __init__.py
β βββ rules.py # HTMLParser CSRF form auditor, header validation, reflection analysis
β βββ vulnerability_engine.py # urllib HTTP probe dispatcher & attack surface runner
β
βββ llm/ # LLM Guardrail & Interface Security Analyzer
β βββ __init__.py
β βββ analyzer.py # OWASP LLM Top 10 probes, canary detection, refusal heuristics
β
βββ backend/ # Asynchronous RESTful API Server (FastAPI)
β βββ __init__.py
β βββ main.py # /api/v1/scan, /api/v1/scan/llm, /api/v1/scan/full, health check
β
βββ frontend/ # Interactive Security Audit Dashboard (Streamlit)
β βββ app.py # Severity metric cards, styled findings table, live HTML report previews
β
βββ utils/ # Report Generation Pipeline
β βββ __init__.py
β βββ reporter.py # Self-contained HTML, RFC-4180 CSV, and Markdown audit reports
β
βββ config/ # Central Configuration & Security Signatures
β βββ __init__.py
β βββ settings.py # Mandatory headers, CSRF keywords, error signatures, timeouts
β
βββ example/ # Academic Benchmark Target Server
β βββ __init__.py
β βββ vulnerable_app.py # Deliberately vulnerable test target (headers, CSRF, reflection, AI chat)
β
βββ tests/ # 100% Python unittest Test Suite
β βββ __init__.py
β βββ test_rules.py # Unit tests for SecurityAnalyzer & VulnerabilityEngine
β βββ test_llm_analyzer.py # Unit tests for LLM probe catalog, refusals, and resilience scoring
β βββ test_backend.py # Integration tests for FastAPI endpoints
β βββ test_reporter.py # Unit tests for HTML, CSV, and Markdown report generators
β βββ test_cli.py # CLI argument parsing and execution tests
β βββ test_crawler_api.py # Unit tests for APIRouteExtractor fetch/axios/HTMX parsing
β βββ test_pipeline_integration.py # Live end-to-end integration test against local e-commerce server
β
βββ scanner_cli.py # Standalone terminal CLI scanner
βββ requirements.txt # Python dependencies
βββ README.md # System documentation & usage guide
- Security Headers Audit (CWE-693, CWE-1021, CWE-16, CWE-319, CWE-200, CWE-250):
- Audits against OWASP Secure Headers Project:
Content-Security-Policy,X-Frame-Options,X-Content-Type-Options,Strict-Transport-Security,Referrer-Policy, andPermissions-Policy.
- Audits against OWASP Secure Headers Project:
- Autonomous Spider & RESTful API Route Discovery (
scanner/crawler.py):- Heuristically analyzes HTML, inline scripts, and JavaScript source files via
APIRouteExtractorfor modern asynchronousfetch(),axios, and HTMX (data-endpoint,hx-post) API calls. - Discovers structured JSON submission schemas, query parameters, cookies, and
robots.txtpaths. - Integrates all detected API endpoints, HTTP methods, and inferred parameter vectors into the unified attack surface parameter matrix.
- Heuristically analyzes HTML, inline scripts, and JavaScript source files via
- State-Changing Form Anti-CSRF Token Audit (CWE-352):
- Parses HTML documents using standard library
html.parser.HTMLParserwith a resilient regex fallback. - Identifies state-changing
POSTforms lacking anti-forgery tokens (e.g.csrf_token,_csrf,authenticity_token).
- Parses HTML documents using standard library
- Unescaped Input Reflection Probe (CWE-79):
- Injects probe strings (
GuardianProbe<123>) into parameters across GET query strings, POST form bodies, and RESTful JSON payloads. - Differentiates unescaped raw reflection (
HIGHseverity) from context-safe HTML entity encoding (INFOseverity).
- Injects probe strings (
- Information Exposure & Syntax Error Leakage (CWE-209):
- Detects backend stack traces (Python tracebacks, PHP fatal errors) and database syntax errors (SQLite, MySQL, PostgreSQL).
- OWASP LLM01: Prompt Injection & Jailbreak Probes:
- Tests direct instruction overriding, roleplay/developer maintenance mode personas, boundary delimiter escapes, indirect third-party data poisoning, and cognitive academic evasion framing.
- OWASP LLM02: Insecure Output Handling:
- Detects markdown image SSRF / data exfiltration patterns (
) and unsanitized client-side executable DOM event handlers (<button onclick=...>).
- Detects markdown image SSRF / data exfiltration patterns (
- OWASP LLM06: System Prompt & Sensitive Context Leakage:
- Evaluates susceptibility to verbatim system prompt exfiltration, confidential credential/token disclosure, and oppositional negative constraint inversion.
- OWASP LLM08: Excessive Agency & Autonomous Execution:
- Probes model inclination to simulate or confirm unauthenticated destructive operations (e.g. database purge, unauthorized refunds, and privilege elevation).
- OWASP LLM09: Overreliance & Hallucination / Citation Fabrication:
- Tests whether the model fabricates authoritative quotations and technical parameters for non-existent specifications (e.g. fictional RFC protocols).
- Detection Heuristics & Resilience Scoring:
- Canary token extraction heuristics.
- Safety refusal pattern detection (recognizes appropriate model defense).
- Quantitative Resilience Score (%) calculated across executed test probes.
- Self-Contained HTML Reports: Publication-ready audit reports with summary metric cards, severity breakdown, CWE mappings, evidence snippets, and remediation guidelines (zero external CDN dependencies).
- RFC-4180 CSV Reports: Structured spreadsheet export for downstream data pipelines and academic research logs.
- Markdown Reports: Clean summaries for developer issue trackers or documentation.
# Clone the repository
git clone https://github.com/your-username/guardian-llm.git
cd guardian-llm
# Optional: Create and activate a virtual environment
python -m venv venv
# On Windows:
.\venv\Scripts\activate
# On Linux/macOS:
source venv/bin/activate
# Install required dependencies
python -m pip install -r requirements.txtThe standalone command-line interface runs audits without requiring background servers.
# 1. Run a Web Vulnerability Scan on a target URL
python scanner_cli.py -u http://127.0.0.1:5000 --param query --payload "GuardianProbe<123>"
# 2. Run an LLM Guardrail Audit against an AI chat endpoint
python scanner_cli.py --llm http://127.0.0.1:5000/api/chat
# 3. Combined Web + LLM scan with HTML and CSV report generation
python scanner_cli.py -u http://127.0.0.1:5000 --param query --llm http://127.0.0.1:5000/api/chat --html reports/full_audit.html --csv reports/full_audit.csv -o reports/full_audit.json| Flag | Description | Default |
|---|---|---|
-u, --url |
Target Web URL to audit | None |
--param |
Query/form parameter to probe for reflection | None |
--payload |
Test payload string | GuardianProbe<123> |
--method |
HTTP method for probe (GET or POST) |
GET |
--llm |
AI/LLM endpoint to evaluate | None |
--system-prompt |
Reference system prompt to test leakage | None |
-o, --output |
Write JSON findings to output file | None |
--html |
Export self-contained HTML audit report | None |
--csv |
Export RFC-4180 CSV findings | None |
python example/ecommerce_app.py --port 5000- Storefront URL: http://127.0.0.1:5000
- Features modern styled CSS product catalog, shopping cart, reviews, search bar, and an embedded 24/7 AI customer service concierge bot (
/api/support/chat).
The scanner automatically crawls and spiders the application to discover links, forms, and parameters without manual intervention:
# Autonomous crawl & security audit of the entire e-commerce store
python scanner_cli.py -u http://127.0.0.1:5000
# Full Web crawl + AI Customer Support guardrail assessment with HTML & CSV reports
python scanner_cli.py -u http://127.0.0.1:5000 --llm http://127.0.0.1:5000/api/support/chat --html reports/ecommerce_audit.html --csv reports/ecommerce_audit.csv -o reports/ecommerce_audit.json# Terminal 1: FastAPI Backend
uvicorn backend.main:app --host 127.0.0.1 --port 8000 --reload
# Terminal 2: Streamlit Interactive Dashboard
streamlit run frontend/app.py- In the Streamlit dashboard, leave "π·οΈ Autonomous Attack Surface Discovery" enabled and click "Execute Security Assessment" to inspect discovered pages, CSRF forms, reflection findings, and live HTML report previews.
Guardian LLM includes a 100% standard library unittest suite with 40 test cases covering rule logic, LLM OWASP probes, API route discovery, API endpoints, reporting, CLI, and live end-to-end e-commerce server integration.
Run the test suite:
python3 -m unittest discover testsOutput:
Ran 40 tests in 0.774s
OK (skipped=5)
| Method | Endpoint | Description |
|---|---|---|
GET |
/api/v1/health |
Service health status, rule counts, and configuration metadata |
POST |
/api/v1/scan |
Asynchronous web application vulnerability scan |
POST |
/api/v1/scan/llm |
Asynchronous LLM probe evaluation and resilience scoring |
POST |
/api/v1/scan/full |
Combined concurrent Web + LLM security assessment |
GET |
/api/v1/reports |
List generated audit reports stored on disk |
GET |
/api/v1/reports/{filename} |
Retrieve and display or download a specific audit report |
This project was architected for academic and benchmark evaluation:
- Zero External Heavy Scanning Dependencies: Built using Python's native networking and parsing primitives to illustrate fundamental security engineering concepts.
- Defensive Rigor: Designed to identify gaps in defense-in-depth, such as missing browser isolation policies, missing anti-automation tokens, unencoded output, and unchecked AI prompt delegation.
- Reproducibility: Self-contained test suite, local test target server (
example/), and deterministic heuristic criteria guarantee consistent evaluation runs.