SOTERIA - Crash Detection Demo

SOTERIA is a Streamlit dashboard for crash detection in dashcam footage. It pairs a YOLOv8 object detector with a custom image-classification head exported to ONNX, then layers collision heuristics to surface clear, human-readable crash reports in real time.

Engine Overview

• Upload a video or point to a URL (YouTube/Shorts, Google Drive, RTMP/RTSP/HLS, direct .mp4), then watch frame-by-frame crash probabilities plotted live.
• Uses YOLOv8 (yolov8n.pt) to track vehicles and a lightweight ONNX classifier (best.onnx) to score crash likelihood.
• Applies collision reasoning to flag likely crash type (rear-end, side impact, multi-vehicle), highlights involved vehicles, and snapshots the peak frame.
• Auto-discovers the newest exports/*/best.onnx so you can drop in fresh models without code changes.
• Ships with data-prep, training, and export scripts to reproduce or improve the classifier.

Repo- Layout

• app/ — Streamlit UI and crash-analysis logic (app/app.py).
• crash_classifier/data/ — CCD dataset helpers: frame extraction (extract_frames.py), flag-based separation (separate_by_flag.py, separate_frames.py), and train/val/test split (split_dataset.py).
• crash_classifier/training/train.py — YOLOv8 classification training with sensible defaults for a single GPU.
• crash_classifier/models/export.py — Export a trained checkpoint to ONNX/TFLite/TorchScript into exports/.
• crash_classifier/cli/ — Small utilities (e.g., infer_one.py for quick checkpoint sanity checks).
• exports/ — Place ONNX models here (one subfolder per version); the app picks the most recent.
• runs/ — YOLO training outputs (auto-generated by Ultralytics).
• requirements.txt — Python dependencies (Streamlit, onnxruntime, ultralytics, OpenCV, yt-dlp, etc.).

Requirements

• Python 3.9+.
• ffmpeg on your PATH (needed for frame extraction scripts and some video handling).
• GPU optional; CPU works for inference. Training benefits from CUDA.

Starting the App

cd path/to/SOTERIA
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt

# ensure an ONNX model is present (see below), then run:
streamlit run app/app.py

Open http://localhost:8501 and choose either:

• Upload Video File: drop an .mp4 from your dashcam.
• Direct Video URL: paste a YouTube link, YouTube Short, Google Drive share link, direct .mp4, or RTMP/RTSP/HLS stream. The app downloads Shorts/YouTube via yt-dlp, with friendly errors for private/age-gated videos.

Use the sidebar slider to control the analysis FPS; the main panel shows the live probability chart, processed frames, and a textual crash summary with involved vehicles.

Model Placement (required)

The UI stops with an error if no best.onnx is found.

• Create an exports subfolder for each model version and drop the ONNX inside:

SOTERIA/
└── exports/
    └── final-model/
        └── best.onnx

• The newest exports/*/best.onnx is loaded automatically. A best.torchscript can sit beside it (unused by the UI but kept for other targets).

Training the Classifier

The crash_classifier folder holds a full CCD pipeline. Typical flow:

1. Extract frames from CCD videos (normal clips): python crash_classifier/data/extract_frames.py --fps 2 --dst data/ccd/frames_normal
2. Separate CCD frames by crash flag (impact vs pre vs normal): python crash_classifier/data/separate_frames.py --link or python crash_classifier/data/separate_by_flag.py if you only need impact/pre split.
3. Build train/val/test folders (hard-links by default): python crash_classifier/data/split_dataset.py --link This writes data/images/{train,val,test}/{crash,normal}/.
4. Train YOLOv8 classifier (adjust config in the script for epochs/batch/gpu): python crash_classifier/training/train.py
5. Export to ONNX/TFLite/TorchScript (auto-picks newest run if no arg): python crash_classifier/models/export.py
6. Serve the model: move or copy the resulting exports/<run>/best.onnx into SOTERIA/exports/<name>/best.onnx for the Streamlit app.

Quick CLI sanity check on a single image (uses the PyTorch checkpoint):

python crash_classifier/cli/infer_one.py path/to/image.jpg

Troubleshooting Notes

• If the app says “No best.onnx found,” confirm the path exports/*/best.onnx exists and is readable.
• YouTube Shorts or age/geo-restricted videos may fail to download; try a different source or a direct .mp4 URL.
• Training scripts assume CCD-style filenames (C_/N_ prefixes) and data/ccd/Crash_Table.csv; adjust paths inside the scripts if your dataset differs.
• Live streams use RTMP/RTSP/HLS URLs; ensure the stream is reachable from your machine.