Image Validation Using Logo Detection

Overview

Symphony Logo Detection is an enterprise-grade API and batch processing system for validating images using advanced YOLO models. Key features include:

Admin Authentication for secure dashboard access
Environment-based Configuration via .env file
Improved Logging with rotation and error tracking
Security with session, CSRF, and CORS protection
YOLO Microservice with configurable URL
Smart Cleanup with pending batch protection
Rate Limiting to prevent abuse
Fault-Tolerant Processing with automatic recovery
File-Based Batch Processing with disk persistence
WebSocket Auto-Reconnection with exponential backoff
Parallel Chunk Processing with server-side optimization

Quick Start Guide

Clone the repository and install dependencies.

Configure environment variables:

# Copy .env.example to .env and edit values
ADMIN_USERNAME=admin
ADMIN_PASSWORD=your_secure_password
SMTP_SERVER=...
YOLO_SERVICE_URL=http://localhost:8001
...

Start the backend and frontend servers.
Access the admin dashboard for monitoring and batch management.

System Requirements

Python 3.11+
Node.js 18+ (for frontend)
YOLOv8/YOLOv11 model weights
Fault tolerance support for production reliability

Key Features

Admin Dashboard: Secure login, session management, and batch history
Configurable via .env: Easily set SMTP, YOLO service, and security settings
Rate Limiting & CORS: Prevents abuse and allows safe frontend integration
Smart Cleanup: Preserves pending batches while cleaning old data
Detailed Logging: Rotating logs for all requests and errors
Fault-Tolerant Processing: Automatic recovery from server restarts
File-Based Batches: Persistent storage for reliable batch processing

System Architecture Overview

Cascading Model Architecture

Progressive Detection:
- yolov8s_logo_detection - Primary model
- yolov8s_logo_detection2 - Enhanced training data
- yolov8s_logo_detection3 - Refined parameters
- yolov11s_logo_detection - Advanced YOLOv11s
- yolov11s3_logo_detection - Optimized YOLOv11s
- yolov11s_cooler_detection - YOLOv11s with cooler dataset
Confidence Thresholds:
- 0.35 confidence threshold for validation
- Automatic model progression
- Ensemble processing for difficult cases

Rate Limiting

Endpoint Limits:
- Single image: 100 requests/minute
- Batch processing: 20 requests/minute
- CSV export: 20 requests/minute
Rate Monitoring:
- Real-time limit tracking
- Remaining requests display
- Reset time indicators

Security Features

Input Validation:
- File type filtering
- Size limit enforcement
- Content validation
Protection Measures:
- CORS protection
- Request validation
- Resource limits

Fault-Tolerant Batch Processing

Problem Solved:
- Server restarts caused batch data loss
- Incomplete batches wasted compute time
- Poor reliability for large batch jobs
Solution Features:
- File-based batch processing with disk persistence
- Automatic recovery from server restarts
- Dual tracking system (URLs + Files)
- Smart cleanup preserves pending batches
- Complete batch lifecycle management
Recovery Process:
- On startup: Scan for pending batch files
- Resume processing from last saved state
- Clean up completed batches automatically
- Preserve batches for 3 days before cleanup

Performance Metrics

Model Performance Comparison

Model	Accuracy	Speed	Confidence Threshold	Use Case
yolov8s_logo_detection	89.2%	0.15s	0.35	Primary detection
yolov8s_logo_detection2	91.8%	0.18s	0.35	Enhanced training
yolov8s_logo_detection3	93.1%	0.22s	0.35	Refined parameters
yolov11s_logo_detection	94.7%	0.25s	0.35	Advanced detection
yolov11s3_logo_detection	95.3%	0.28s	0.35	Optimized YOLOv11s
yolov11s_cooler_detection	96.1%	0.26s	0.35	Cooler specialized

Detection Models

Image Processing Pipeline

Model Architecture

Model 1

yolov8s_logo_detection

Model 2

yolov8s_logo_detection2

Model 3

yolov8s_logo_detection3

Model 4

yolov11s_logo_detection

Model 5

yolov11s3_logo_detection

Model 6

yolov11s_cooler_detection

Sequential Processing Flow

Model 1: yolov8s_logo_detection (Primary)
If no Symphony found, try Model 2: yolov8s_logo_detection2
If still no detection, try Model 3: yolov8s_logo_detection3
Continue to Model 4: yolov11s_logo_detection
If still no detection, try Model 5: yolov11s3_logo_detection
Final attempt: Model 6: yolov11s_cooler_detection
Early return when Symphony logo detected (conf ≥ 0.35)

Rate Limits

Endpoint Limits

Single Image

100 requests per minute

Batch Processing

20 requests per minute

CSV Export

20 requests per minute

WebSocket

Real-time updates with auto-reconnect

Rate Limit Headers

X-RateLimit-Limit: Maximum requests allowed
X-RateLimit-Remaining: Requests remaining
X-RateLimit-Reset: Time until limit reset

Single Image Detection

File Processing Workflow

Processing Steps

1

Input Image

Upload file or enter URL

2

Validation

File type and size check

3

Processing

Model cascade detection

4

Results

View detection details

Batch Processing

Batch Processing Flow

Fault-Tolerant Batch Processing Steps

1

Start Batch

Initialize batch session with unique ID

2

Save Files

Store uploaded files to disk with metadata

3

Chunk

Server-side division into manageable chunks

4

Process

Parallel detection with progress tracking

5

Recovery

Automatic resume if interrupted

6

Export

Download CSV results

7

Cleanup

Smart cleanup with pending batch protection

CSV Export Process

CSV Export Details

Image path and validation result
Confidence scores and model used
Bounding box coordinates
Batch ID and timestamp

Improved Batch Retry Tracking

Fixed Duplicate Progress:
- Eliminated duplicate progress increments in batch retry logic
- More accurate progress reporting during retries
- Consistent progress percentage throughout batch processing
- Improved reliability for tracking large batches

Smart Cleanup Protection

Regular Batches

24-hour retention with automatic cleanup

Pending Batches

Protected from cleanup for 3 days

Abandoned Batches

Cleanup after 3 days to prevent accumulation

Protection Strategy

Immediate Protection: Pending batches are preserved during regular cleanup
Recovery Window: 3-day grace period for interrupted batches
Automatic Recovery: System resumes processing on startup
Smart Cleanup: Separate cleanup functions for different batch states

API Integration

API Security & Authentication Flow

Frontend API Integration

Key Endpoints

POST /api/check-logo/single/

Single Image Detection

Detect Symphony logo in a single image file or URL.

Request Example:

Request

curl -X POST "http://localhost:8000/api/check-logo/single/" \
  -H "Content-Type: multipart/form-data" \
  -F "file=@image.jpg"

Response Example:

Response

{
  "valid": true,
  "confidence": 0.87,
  "model_used": "yolov8s_logo_detection",
  "bounding_box": [120, 45, 280, 180],
  "processing_time": 0.234
}

POST /api/start-batch

Start Batch Session

Initialize a new batch processing session.

Response Example:

Response

{
  "batch_id": "batch_20241201_143052_abc123",
  "status": "initialized",
  "created_at": "2024-12-01T14:30:52Z"
}

POST /api/check-logo/batch/

Batch Image Detection

Process multiple images in a batch session.

Request Example:

Request

curl -X POST "http://localhost:8000/api/check-logo/batch/" \
  -H "Content-Type: multipart/form-data" \
  -F "batch_id=batch_20241201_143052_abc123" \
  -F "files=@image1.jpg" \
  -F "files=@image2.png" \
  -F "files=@image3.jpg"

Response Example:

Response

{
  "batch_id": "batch_20241201_143052_abc123",
  "status": "processing",
  "total_files": 3,
  "processed_files": 0,
  "message": "Batch processing started"
}

GET /api/check-logo/batch/export-csv/{batch_id}

CSV Export

Download batch results as CSV file.

Request Example:

Request

curl -X GET "http://localhost:8000/api/check-logo/batch/export-csv/batch_20241201_143052_abc123" \
  -H "Accept: text/csv" \
  -o results.csv

CSV Format:

CSV Output

filename,valid,confidence,model_used,bounding_box,processing_time
image1.jpg,true,0.87,yolov8s_logo_detection,"[120,45,280,180]",0.234
image2.png,false,0.12,yolov11s3_logo_detection,"[]",0.189
image3.jpg,true,0.92,yolov8s_logo_detection2,"[95,30,255,165]",0.267

WS /ws/batch/{batch_id}

WebSocket Updates

Real-time batch processing progress updates.

Message Example:

WebSocket Message

{
  "type": "progress",
  "batch_id": "batch_20241201_143052_abc123",
  "processed": 15,
  "total": 50,
  "current_file": "image_015.jpg",
  "status": "processing"
}

Troubleshooting

Error Handling & Monitoring

Rate Limit Issues

429 Too Many Requests: Rate limit exceeded

Common Causes:

Single image endpoint: Exceeded 100 requests/minute
Batch processing: Exceeded 20 requests/minute
CSV export: Exceeded 20 requests/minute

Solutions:

Check X-RateLimit-Reset header for reset time
Implement exponential backoff
Use batch processing instead of multiple single requests
Monitor X-RateLimit-Remaining header

File Validation Issues

400 Bad Request: Invalid file format or size

Common Causes:

Unsupported file type (only JPG, JPEG, PNG, WEBP, BMP allowed)
File size exceeds limit (10MB per file)
Corrupted image file
Empty file upload

Solutions:

Verify file format before upload
Compress large images
Check file integrity
Ensure proper file encoding

Model Processing Issues

Low Confidence: Model cascade completed without confident detection

Common Causes:

Image quality too low
Logo too small or unclear
Complex background interference
Unusual logo orientation

Solutions:

Use higher quality images
Ensure logo is clearly visible
Minimize background noise
Try different image angles

Batch Processing Issues

Batch Processing Error: Invalid batch state or timeout

Common Causes:

Invalid batch ID
Batch session expired
Too many files in single batch
Network timeout during processing
Server restart during processing (now auto-recovered)

Solutions:

Verify batch ID is valid
Process batches within session timeout
Split large batches into smaller ones
Check network stability
System automatically recovers from restarts

Automatic Recovery System

How Recovery Works

Startup Scan: System automatically detects pending batches on startup
File Persistence: Uploaded files are saved to disk before processing
State Tracking: Progress is tracked in pending_files.json and pending_urls.json
Resume Processing: Interrupted batches resume from last saved state
Clean Completion: All temporary files are cleaned up after completion

Note: The system provides 100% reliability for batch processing. Server restarts, crashes, or network interruptions no longer cause data loss.

Robust WebSocket Connection

Connection Resilience:
- Exponential backoff retry mechanism (1s to 16s)
- Limited reconnection attempts to prevent resource exhaustion
- Visual feedback during reconnection attempts
- Proper cleanup of WebSocket resources
- Heartbeat mechanism to detect stale connections

CSV Export Issues

Export Error: Unable to generate or download CSV

Common Causes:

Batch processing not complete
Export timeout
Rate limit exceeded
Invalid batch results

Solutions:

Wait for batch completion
Check batch status before export
Monitor rate limits
Verify batch results exist

Error Prevention Best Practices

Monitor rate limits using response headers
Validate files before upload
Implement proper error handling
Use batch processing for multiple files
Keep batch sizes manageable
Download CSV exports promptly
Implement proper retry logic
Log errors for debugging