Phase 11.7 — Email Retry System (Production-Ready)

[vimal-java-dev]

Context:

• Spring Boot backend (Docker, VPS deployed with blue-green)
• Async email already implemented (ThreadPoolTaskExecutor)
• EmailService using interface + SMTP implementation
• PostgreSQL available
• Redis NOT used
• Goal: Add a retry mechanism with failure tracking

Requirements:

• Store failed emails in DB
• Retry using the scheduler
• Track status (PENDING, FAILED, SENT)
• Clean integration with the existing EmailService
• Production-ready design (no overengineering)

Proceed step-by-step.

[vimaltech-dev]

Assign me.

[vimal-java-dev]

If any Error at the VPS level, do below 1st,

docker stop vimaltech-contact-api-blue
docker rm vimaltech-contact-api-blue

docker image rm ghcr.io/vimal-java-dev/vimaltech-contact-api:latest

docker pull ghcr.io/vimal-java-dev/vimaltech-contact-api:latest
./deploy.sh

---

👉 Ensure it shows MAIL_HOST (NOT SPRING_MAIL_HOST)

docker exec -it vimaltech-contact-api-green printenv | grep MAIL

MAIL_HOST=smtp.zoho.in
MAIL_PORT=587
MAIL_USERNAME=vimal.patel@vimaltech.dev
MAIL_PASSWORD=xxxxxxxxxxxxxxx
APP_MAIL_FROM=vimal.patel@vimaltech.dev
APP_ADMIN_EMAIL=vimal.patel@vimaltech.dev

---

DEBUG at VPS Level

Run:

docker logs vimaltech-contact-api-green -f

docker logs vimaltech-contact-api-green | grep EmailService

Then hit the API.

[vimaltech-dev]

🧠 Final Architecture (Clean & Production-Ready)

Flow becomes:

API Request
↓
Async EmailService
↓
DB (PENDING)
↓
SMTP Send
↓
SENT or FAILED
↓
Scheduler
↓
Retry FAILED
↓
Final SENT or STOP after max retries

---

✅ What We Already Did Correctly

Our current system already has:

✅ Interface-based email abstraction
✅ Async execution isolated in the SMTP layer
✅ Dedicated thread pool
✅ DTO separation (EmailRequest)
✅ Contact persistence independent from email sending
✅ Proper logging
✅ Reply-To support
✅ Profile-based SMTP activation

This means:
👉 We should extend, not redesign.

---

✅ Correct Design For YOUR Existing System

We should NOT:

• Modify ContactService heavily
• Create Kafka-like complexity
• Add Redis
• Introduce queues

Instead:

🔷 We Will Add

Component	Purpose
EmailLog entity	persistent email tracking
EmailStatus enum	SENT / FAILED / PENDING
EmailLogRepository	retry querying
EmailRetryScheduler	periodic retry
small refactor in SmtpEmailService	central retry handling

[vimaltech-dev]

From your project root:

docker compose down

Then rebuild + start:

docker compose up --build

---

✅ Recommended (Detached Mode)

Usually better:

docker compose up --build -d

✅ Then Check Logs

Watch startup logs:

docker compose logs -f contact-api

OR if your service name is different:

docker compose ps

to see the actual service name.

✅ What You Should See

Hibernate should create:

email_logs

Look for logs similar to:

create table email_logs

or

alter table email_logs

---

✅ Then Verify Inside PostgreSQL Container

Enter DB container:

docker exec -it vimaltech-postgres psql -U contactuser -d contactdb

If your container name differs:

docker ps

[vimaltech-dev]

✅ Create a DEV Email Implementation
✅ Current Status

Component	Status
EmailStatus enum	✅
EmailLog entity	✅
EmailLogRepository	✅
PostgreSQL schema	✅
Docker integration	✅
Dev profile separation	✅
Email bean architecture	✅

---

⚠️ IMPORTANT ARCHITECTURAL CHANGE

Right now, your SMTP service:

• only sends mail
• does NOT persist state

We will now make it:

• persistence-aware
• retry-aware
• production-safe

WITHOUT breaking existing callers.

---

🔷 STEP 1 — Update EmailService Interface
🔷 STEP 2 — Update DevEmailService

---

🔥 NOW: Refactor SmtpEmailService

This is the core of Phase 11.7.

We will do this carefully in small, controlled steps.

================

⚠️ IMPORTANT GOAL

Current behavior:

send email
↓
if fails
↓
log only

New behavior:

create EmailLog row
↓
attempt SMTP send
↓
update SENT / FAILED
↓
scheduler retries FAILED later

---

We’ve now completed:

Phase 11.7 core implementation

• Flyway integration
• retry scheduler
• production-safe schema management
• logging cleanup
• retry validation end-to-end

And our latest logs confirm Step 3 succeeded correctly, including safe idempotent migrations with:

already exists, skipping.

which is expected because of IF NOT EXISTS.

[vimaltech-dev]

Phase 11.7 — Step 4

Optimize Retry Scheduler Efficiency

Production Benefits

This is a real enterprise-grade improvement.
We gain:

✅ lower DB scans
✅ faster scheduler execution
✅ reduced CPU usage
✅ cleaner retry queue
✅ operational clarity
✅ easier monitoring/analytics
✅ future dashboard support

---

Expected Final Flow

Create Email
↓
PENDING
↓
Send Success → SENT

OR

Send Failure
↓
FAILED
↓
Scheduler Retry
↓
Success → SENT

OR

Retries Exhausted
↓
FAILED_PERMANENT

---

This Is a Strong Production Feature

Most beginner retry systems stop at:

FAILED forever

My implementation now demonstrates:

• retry lifecycle management
• queue hygiene
• operational optimization
• scheduler efficiency
• production-aware state transitions

This is genuinely strong backend engineering for a portfolio project.

---

After Completing Step 4

Our retry system will now include:

✅ async email sending
✅ retry mechanism
✅ exponential/backoff-ready architecture
✅ retry counting
✅ scheduler retry processing
✅ terminal failure state
✅ retry queue optimization
✅ production-grade lifecycle management

That becomes a very credible enterprise backend feature set.

[vimaltech-dev]

Check My PR. #51

[vimal-java-dev]

Check My PR. #51

PR #51 closed and merged,