An edge-ready computer vision safety system designed to prevent accidents through real-time fatigue detection and automated emergency response.
- Overview & System Features
- Video Demonstration
- System Architecture & Tech Stack
- In-Depth Mechanism & Mathematics
- Emergency Action Protocol
- Prerequisites & Installation
- Usage Guide
- Future Roadmap
SafeDrive AI is a complete, beginner-friendly telematics and computer vision system designed to enhance profile value for engineering applications. By combining multi-modal structural tracking with defensive cloud routing pipelines, it provides high-impact real-world usability.
- Multi-Modal Fatigue Profiling: Monitors both eye separation (PERCLOS tracking) and wide yawns (Mouth Opening Ratio) continuously via facial landmarks.
- Asynchronous Alarm Pipeline: Spins off a local cabin warning siren using Pygame onto an independent background thread, preventing frame lockup.
- Network Geolocation Mapping: Automatically queries global internet routers to pull live city names and coordinates within 3 seconds, injecting coordinates seamlessly into emergency payloads.
- Comprehensive Linear Continuity: Pins the local warning, device isolation loop, network ping, and Twilio voice API dispatches under a single controlled logical umbrella block.
Click the image above to watch the live simulation of the fatigue detection and emergency SOS routing!
| Technology | Purpose in Project |
|---|---|
| Python | The core programming language powering the multi-threaded logic and state machines. |
| OpenCV (cv2) | Handles video feed ingestion, RGB color conversion, and drawing the dynamic Heads-Up Display (HUD). |
| MediaPipe | Deploys two simultaneous neural networks: FaceMesh (for high-precision eye/lip tracking) and Pose (for overall body/head alignment). |
| NumPy | Performs rapid vector mathematics to calculate L2 Euclidean distances between facial landmarks. |
| Twilio API | Routes programmatic voice calls over global cellular networks during critical failures. |
| Pygame & Threading | Handles asynchronous audio playback (alarm.wav) to prevent thread-locking during emergency events. |
The accuracy of SafeDrive AI relies on geometric calculations mapped over continuous time-buffers. Here is how the detection algorithms work under the hood.
The MediaPipe FaceMesh network is activated with refine_landmarks=True to map extra high-precision points around the eyes. We isolate:
- Node 159: Top Eyelid
- Node 145: Bottom Eyelid
We calculate the exact distance between these points using the L2 Euclidean Norm:
Distance = √((x2 - x1)² + (y2 - y1)²)
- Threshold Logic: If the normalized distance drops below
0.015, the eye is classified as "closed". - Time-Buffering: A single blink should not trigger the alarm. The system increments a
CLOSED_COUNTERonly during consecutive closed frames. If the counter reaches90 frames(approximately 3 seconds at 30 FPS), the Sleep Emergency Protocol is triggered. Opening the eyes instantly resets the counter to zero.
Using a similar methodology, we track the inner lips:
- Node 13: Top Inner Lip
- Node 14: Bottom Inner Lip
If the Euclidean distance between these points exceeds 0.040, a yawn is registered. Sustaining this distance for 45 consecutive frames triggers a critical fatigue alert.
MediaPipe's Pose network tracks the driver's overall skeletal structure. We isolate the Nose Landmark.
- In the normalized MediaPipe coordinate space,
Y=0is the top of the frame andY=1.0is the bottom. - If the Nose drops below
Y = 0.75, it mathematically indicates the driver's head has slumped severely downward toward the steering wheel. - A buffer of
60 frames(~2 seconds) is required to verify a crash posture, filtering out quick glances at the dashboard.
When any of the three danger thresholds are breached, the script locks into an emergency state machine executing the following sequence:
- Multithreaded Siren: Spawns a background daemon thread utilizing
pygame.mixerto continuously loop an audio file (alarm.wav) to jolt the driver awake. - Resource Reallocation: Shuts down the webcam feed and OpenCV windows to free up CPU cycles for network communications.
- Dynamic Geolocation: Queries
https://ipapi.co/json/via therequestslibrary to fetch the vehicle's live latitude, longitude, and region. A 3-second timeout prevents the system from hanging in dead zones. - Cellular Dispatch: Constructs a dynamic TwiML (Twilio Markup Language) script containing the telemetry data, utilizing the Amazon Polly "Amy" voice.
- Outbound Call: Bypasses local hardware limitations by routing the SOS call through Twilio's cloud infrastructure directly to a target phone number.
- A functional webcam positioned on the dashboard or steering column.
- Python 3.8+ installed on your local machine or edge device.
- A Twilio Developer account (for cellular SOS routing).
- An audio file named
alarm.wavin the root directory.
1. Clone the Source Code
git clone https://github.com/SuryanshOps/Driver_Minitoring_System.git
cd Driver_Monitoring_System2. Virtual Environment Setup
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate3. Install Dependencies
pip install opencv-python mediapipe numpy twilio requests pygame4. Configure Telematics (Twilio)
Open main2.py and replace the placeholder credentials with your Twilio dashboard keys:
TWILIO_ACCOUNT_SID = "ACxxxxxxxxxxxxxxxxxxxxxxxx"
TWILIO_AUTH_TOKEN = "your_auth_token_here"
TWILIO_PHONE_NUM = "+1234567890"
TARGET_PHONE_NUM = "+91XXXXXXXXXX"- Place your
alarm.wavsound file in the same folder as the script. - Launch the monitoring system:
python main2.py
- A HUD window will appear displaying live tracking metrics for your eyes and posture.
- To Test Sleep Detection: Close your eyes for 3 seconds.
- To Test Crash Posture: Duck your head below the bottom 25% of the camera frame for 2 seconds.
- To Exit: Press the
qkey on your keyboard to safely break the monitoring loop and release hardware resources.
- Infrared Camera Support: Integrate IR spectrum analysis for flawless nighttime and low-light tracking inside the cabin.
- GPS Hardware Integration: Replace IP-based geolocation with serial communication to a dedicated GPS module for extreme precision in dead zones.
- Gaze Tracking: Monitor pupil direction to detect dangerous levels of distraction (e.g., staring at a phone instead of the road).
Advancing vehicular safety through edge computing and artificial intelligence.