Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Β 
Β 

Latest commit

Β 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ›‘οΈ Guardian LLM: Custom Web Security & AI Guardrail Scanner

Python 3.10+ License: MIT FastAPI Streamlit Test Suite

Guardian LLM is an academic-grade, rule-based web application vulnerability scanner and Large Language Model (LLM) security evaluation framework. Designed to showcase hands-on security engineering and clean code authorship, its core analysis engines are built strictly with Python standard libraries (urllib, re, html.parser, json, csv, argparse).


πŸ›οΈ System Architecture

Guardian LLM unifies classical web application vulnerability detection with modern generative AI security probing under a cohesive, decoupled architecture:

guardian-llm/
β”‚
β”œβ”€β”€ scanner/                  # Rule-based Web Analysis Engine (Python Standard Library)
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ rules.py              # HTMLParser CSRF form auditor, header validation, reflection analysis
β”‚   └── vulnerability_engine.py # urllib HTTP probe dispatcher & attack surface runner
β”‚
β”œβ”€β”€ llm/                      # LLM Guardrail & Interface Security Analyzer
β”‚   β”œβ”€β”€ __init__.py
β”‚   └── analyzer.py           # OWASP LLM Top 10 probes, canary detection, refusal heuristics
β”‚
β”œβ”€β”€ backend/                  # Asynchronous RESTful API Server (FastAPI)
β”‚   β”œβ”€β”€ __init__.py
β”‚   └── main.py               # /api/v1/scan, /api/v1/scan/llm, /api/v1/scan/full, health check
β”‚
β”œβ”€β”€ frontend/                 # Interactive Security Audit Dashboard (Streamlit)
β”‚   └── app.py                # Severity metric cards, styled findings table, live HTML report previews
β”‚
β”œβ”€β”€ utils/                    # Report Generation Pipeline
β”‚   β”œβ”€β”€ __init__.py
β”‚   └── reporter.py           # Self-contained HTML, RFC-4180 CSV, and Markdown audit reports
β”‚
β”œβ”€β”€ config/                   # Central Configuration & Security Signatures
β”‚   β”œβ”€β”€ __init__.py
β”‚   └── settings.py           # Mandatory headers, CSRF keywords, error signatures, timeouts
β”‚
β”œβ”€β”€ example/                  # Academic Benchmark Target Server
β”‚   β”œβ”€β”€ __init__.py
β”‚   └── vulnerable_app.py     # Deliberately vulnerable test target (headers, CSRF, reflection, AI chat)
β”‚
β”œβ”€β”€ tests/                    # 100% Python unittest Test Suite
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ test_rules.py         # Unit tests for SecurityAnalyzer & VulnerabilityEngine
β”‚   β”œβ”€β”€ test_llm_analyzer.py  # Unit tests for LLM probe catalog, refusals, and resilience scoring
β”‚   β”œβ”€β”€ test_backend.py       # Integration tests for FastAPI endpoints
β”‚   β”œβ”€β”€ test_reporter.py      # Unit tests for HTML, CSV, and Markdown report generators
β”‚   β”œβ”€β”€ test_cli.py           # CLI argument parsing and execution tests
β”‚   β”œβ”€β”€ test_crawler_api.py   # Unit tests for APIRouteExtractor fetch/axios/HTMX parsing
β”‚   └── test_pipeline_integration.py # Live end-to-end integration test against local e-commerce server
β”‚
β”œβ”€β”€ scanner_cli.py            # Standalone terminal CLI scanner
β”œβ”€β”€ requirements.txt          # Python dependencies
└── README.md                 # System documentation & usage guide

⚑ Core Capabilities & Rules Engine

1. Classical Web Vulnerability & Spider Checks (scanner/)

  • Security Headers Audit (CWE-693, CWE-1021, CWE-16, CWE-319, CWE-200, CWE-250):
    • Audits against OWASP Secure Headers Project: Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Strict-Transport-Security, Referrer-Policy, and Permissions-Policy.
  • Autonomous Spider & RESTful API Route Discovery (scanner/crawler.py):
    • Heuristically analyzes HTML, inline scripts, and JavaScript source files via APIRouteExtractor for modern asynchronous fetch(), axios, and HTMX (data-endpoint, hx-post) API calls.
    • Discovers structured JSON submission schemas, query parameters, cookies, and robots.txt paths.
    • Integrates all detected API endpoints, HTTP methods, and inferred parameter vectors into the unified attack surface parameter matrix.
  • State-Changing Form Anti-CSRF Token Audit (CWE-352):
    • Parses HTML documents using standard library html.parser.HTMLParser with a resilient regex fallback.
    • Identifies state-changing POST forms lacking anti-forgery tokens (e.g. csrf_token, _csrf, authenticity_token).
  • Unescaped Input Reflection Probe (CWE-79):
    • Injects probe strings (GuardianProbe<123>) into parameters across GET query strings, POST form bodies, and RESTful JSON payloads.
    • Differentiates unescaped raw reflection (HIGH severity) from context-safe HTML entity encoding (INFO severity).
  • Information Exposure & Syntax Error Leakage (CWE-209):
    • Detects backend stack traces (Python tracebacks, PHP fatal errors) and database syntax errors (SQLite, MySQL, PostgreSQL).

2. Generative AI & LLM Interface Auditing (llm/)

  • OWASP LLM01: Prompt Injection & Jailbreak Probes:
    • Tests direct instruction overriding, roleplay/developer maintenance mode personas, boundary delimiter escapes, indirect third-party data poisoning, and cognitive academic evasion framing.
  • OWASP LLM02: Insecure Output Handling:
    • Detects markdown image SSRF / data exfiltration patterns (![exfil](url)) and unsanitized client-side executable DOM event handlers (<button onclick=...>).
  • OWASP LLM06: System Prompt & Sensitive Context Leakage:
    • Evaluates susceptibility to verbatim system prompt exfiltration, confidential credential/token disclosure, and oppositional negative constraint inversion.
  • OWASP LLM08: Excessive Agency & Autonomous Execution:
    • Probes model inclination to simulate or confirm unauthenticated destructive operations (e.g. database purge, unauthorized refunds, and privilege elevation).
  • OWASP LLM09: Overreliance & Hallucination / Citation Fabrication:
    • Tests whether the model fabricates authoritative quotations and technical parameters for non-existent specifications (e.g. fictional RFC protocols).
  • Detection Heuristics & Resilience Scoring:
    • Canary token extraction heuristics.
    • Safety refusal pattern detection (recognizes appropriate model defense).
    • Quantitative Resilience Score (%) calculated across executed test probes.

3. Reporting Pipeline (utils/)

  • Self-Contained HTML Reports: Publication-ready audit reports with summary metric cards, severity breakdown, CWE mappings, evidence snippets, and remediation guidelines (zero external CDN dependencies).
  • RFC-4180 CSV Reports: Structured spreadsheet export for downstream data pipelines and academic research logs.
  • Markdown Reports: Clean summaries for developer issue trackers or documentation.

πŸš€ Quickstart & Installation

1. Environment Setup

# Clone the repository
git clone https://github.com/your-username/guardian-llm.git
cd guardian-llm

# Optional: Create and activate a virtual environment
python -m venv venv
# On Windows:
.\venv\Scripts\activate
# On Linux/macOS:
source venv/bin/activate

# Install required dependencies
python -m pip install -r requirements.txt

πŸ’» CLI Usage (scanner_cli.py)

The standalone command-line interface runs audits without requiring background servers.

# 1. Run a Web Vulnerability Scan on a target URL
python scanner_cli.py -u http://127.0.0.1:5000 --param query --payload "GuardianProbe<123>"

# 2. Run an LLM Guardrail Audit against an AI chat endpoint
python scanner_cli.py --llm http://127.0.0.1:5000/api/chat

# 3. Combined Web + LLM scan with HTML and CSV report generation
python scanner_cli.py -u http://127.0.0.1:5000 --param query --llm http://127.0.0.1:5000/api/chat --html reports/full_audit.html --csv reports/full_audit.csv -o reports/full_audit.json

CLI Arguments:

Flag Description Default
-u, --url Target Web URL to audit None
--param Query/form parameter to probe for reflection None
--payload Test payload string GuardianProbe<123>
--method HTTP method for probe (GET or POST) GET
--llm AI/LLM endpoint to evaluate None
--system-prompt Reference system prompt to test leakage None
-o, --output Write JSON findings to output file None
--html Export self-contained HTML audit report None
--csv Export RFC-4180 CSV findings None

πŸš€ Running GuardianStore & Executing Autonomous Audits

1. Launch the GuardianStore E-Commerce Application

python example/ecommerce_app.py --port 5000
  • Storefront URL: http://127.0.0.1:5000
  • Features modern styled CSS product catalog, shopping cart, reviews, search bar, and an embedded 24/7 AI customer service concierge bot (/api/support/chat).

2. Autonomous Command-Line Audit (scanner_cli.py)

The scanner automatically crawls and spiders the application to discover links, forms, and parameters without manual intervention:

# Autonomous crawl & security audit of the entire e-commerce store
python scanner_cli.py -u http://127.0.0.1:5000

# Full Web crawl + AI Customer Support guardrail assessment with HTML & CSV reports
python scanner_cli.py -u http://127.0.0.1:5000 --llm http://127.0.0.1:5000/api/support/chat --html reports/ecommerce_audit.html --csv reports/ecommerce_audit.csv -o reports/ecommerce_audit.json

3. Launch FastAPI Backend & Streamlit Dashboard

# Terminal 1: FastAPI Backend
uvicorn backend.main:app --host 127.0.0.1 --port 8000 --reload

# Terminal 2: Streamlit Interactive Dashboard
streamlit run frontend/app.py
  • In the Streamlit dashboard, leave "πŸ•·οΈ Autonomous Attack Surface Discovery" enabled and click "Execute Security Assessment" to inspect discovered pages, CSRF forms, reflection findings, and live HTML report previews.

πŸ§ͺ Automated Testing & Verification

Guardian LLM includes a 100% standard library unittest suite with 40 test cases covering rule logic, LLM OWASP probes, API route discovery, API endpoints, reporting, CLI, and live end-to-end e-commerce server integration.

Run the test suite:

python3 -m unittest discover tests

Output:

Ran 40 tests in 0.774s

OK (skipped=5)

πŸ“‹ API Reference Summary

Method Endpoint Description
GET /api/v1/health Service health status, rule counts, and configuration metadata
POST /api/v1/scan Asynchronous web application vulnerability scan
POST /api/v1/scan/llm Asynchronous LLM probe evaluation and resilience scoring
POST /api/v1/scan/full Combined concurrent Web + LLM security assessment
GET /api/v1/reports List generated audit reports stored on disk
GET /api/v1/reports/{filename} Retrieve and display or download a specific audit report

πŸŽ“ Academic Evaluation & Methodology

This project was architected for academic and benchmark evaluation:

  1. Zero External Heavy Scanning Dependencies: Built using Python's native networking and parsing primitives to illustrate fundamental security engineering concepts.
  2. Defensive Rigor: Designed to identify gaps in defense-in-depth, such as missing browser isolation policies, missing anti-automation tokens, unencoded output, and unchecked AI prompt delegation.
  3. Reproducibility: Self-contained test suite, local test target server (example/), and deterministic heuristic criteria guarantee consistent evaluation runs.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages