# Order Processing Module — Developer Guide

## Overview

The `orderprocessing` module automates the fetching, storing, and notification of court case orders. It:

1. Builds a queue of cases that need order processing
2. Fetches orders from parsed court data, uploads PDFs to S3, extracts text
3. Sends email and WhatsApp notifications to users, clients, and POC contacts

---

## Directory Structure

```
modules/orderprocessing/
├── Orderprocessing.php              # Module bootstrap
├── config/
│   └── main.php                     # Module parameters (batchSize, numberoftries)
├── commands/
│   └── ProcessQueueController.php   # All CLI cron actions (1575 lines)
├── controllers/
│   └── OrderprocessingController.php # Web dashboard controller
├── services/
│   └── OrderProcessor.php           # Core order processing logic
├── components/
│   ├── QueueLogger.php              # Dual logging (console + DB)
│   └── Utility.php                  # S3 upload, PDF-to-text, notifications
├── models/
│   ├── Queue.php                    # ActiveRecord → mjbk_case_order_processing_queue
│   ├── OrderProcessingLog.php       # ActiveRecord → mjbk_order_processing_log
│   ├── CaseOrderProcessingQueueSearch.php
│   └── OrderProcessingLogSearch.php
├── views/orderprocessing/
│   ├── index.php                    # Queue dashboard (live stats via JS polling)
│   └── logs.php                     # Log viewer
└── migrations/
    └── m260321_000001_create_order_processing_log_table.php
```

---

## Database Tables

### `mjbk_case_order_processing_queue`

Tracks each case queued for order processing.

| Column | Type | Description |
|---|---|---|
| `id` | int PK | Auto-increment |
| `case_id` | int | Foreign key → mjbk_cases.id |
| `status` | int | 0=Pending, 1=Processing, 2=Done, 3=Failed |
| `attempts` | int | Number of processing attempts |
| `error` | text | Error/success message from last run |
| `info` | text | Human-readable date comparison info (next_hearing_date vs latest order date) |
| `locked_at` | datetime | Timestamp when item was locked for processing |
| `order_processing_finish_at` | datetime | Timestamp when processing completed |
| `created_at` | datetime | When the queue item was created |

**Status lifecycle:** `0 (Pending)` → `1 (Processing)` → `2 (Done)` or `3 (Failed)`

Items with `status=3` and `attempts < numberoftries` are retried automatically.
Items with `status=1` and `locked_at` older than 30 minutes are reclaimed (stale lock recovery).

### `mjbk_order_processing_log`

Stores structured logs for every module action.

| Column | Type | Description |
|---|---|---|
| `id` | int PK | Auto-increment |
| `action` | varchar(50) | Action name: `build`, `delete`, `process`, `email-orders-for-cases`, etc. |
| `level` | varchar(20) | `info`, `warning`, `error`, `success` |
| `message` | text | Log message |
| `case_id` | int | Related case ID (nullable) |
| `queue_id` | int | Related queue item ID (nullable) |
| `filters` | text | JSON-encoded filters used |
| `details` | json | Additional context |
| `query` | text | Raw SQL query (if applicable) |
| `created_by` | varchar(255) | `system` or user identifier |
| `created_at` | datetime | Auto-set timestamp |

---

## Configuration

**`config/main.php`**

```php
'params' => [
    'orderprocessing.batchSize'      => 5,   // Items per processing batch
    'orderprocessing.numberoftries'  => 1,   // Max retry attempts for failed items
]
```

These are merged into `Yii::$app->params` at module bootstrap via `Orderprocessing::init()`.

---

## CLI Commands (Cron Actions)

All commands are in `ProcessQueueController`. The Yii route prefix is `cron-process-queue`.

### 1. `cron-process-queue/build` — Build the Queue

Populates `mjbk_case_order_processing_queue` with eligible cases.

**Mandatory condition:** `mjbk_cases.parse_flag = 1`

**Filters (all optional, combinable):**

| Option | Alias | Description |
|---|---|---|
| `--user-id` | `-u` | Comma-separated user IDs |
| `--court-id` | `-c` | Comma-separated court IDs |
| `--high-court-id` | `-h` | Comma-separated high court IDs |
| `--case-status` | `-s` | Comma-separated case statuses |
| `--uc-settings-id` | | Comma-separated UC settings IDs |
| `--date-from` | | Case created_at >= (format: YYYY-MM-DD) |
| `--date-to` | | Case created_at <= (format: YYYY-MM-DD) |
| `--skip-existing` | | Skip cases already in queue (default: true) |

**Usage examples:**
```bash
php yii cron-process-queue/build --user-id=1,2,3
php yii cron-process-queue/build --court-id=2 --case-status=2
php yii cron-process-queue/build --uc-settings-id=5399
php yii cron-process-queue/build --user-id=996 --court-id=2 --high-court-id=14 --case-status=2
php yii cron-process-queue/build --date-from=2026-01-01 --date-to=2026-03-31
```

**Smart date filtering (`casesFilterQuery`):**
When `--case-status` is provided, the build action applies an additional filter to avoid reprocessing cases where the latest order is already newer than the next hearing/status date:
- For `status=1` (running): only includes cases where `next_hearing_date > latest order date`
- For `status=2` (closed): only includes cases where `status_date > latest order date`
- Cases with no existing orders are always included

**High court 12 special rule:** Cases with `high_court_id=12` are only included if `is_cnr=1`.

---

### 2. `cron-process-queue/index` — Process the Queue

Main processing loop. Runs continuously until the queue is empty.

```bash
php yii cron-process-queue/index
```

**Processing flow:**
1. Opens a DB transaction and fetches `batchSize` pending/retryable items using `FOR UPDATE SKIP LOCKED` (prevents concurrent duplicate processing)
2. Marks fetched items as `status=1` (Processing) with `locked_at` timestamp
3. Commits the transaction
4. For each item, calls `OrderProcessor::process($case)`
5. Updates each item to `status=2` (Done) or `status=3` (Failed) with message
6. Exits after 6 consecutive empty queue checks (~12 seconds idle)

**Stale lock recovery:** Items stuck in `status=1` with `locked_at` older than 30 minutes are automatically re-queued.

---

### 3. `cron-process-queue/delete` — Delete or Reset Queue Items

```bash
php yii cron-process-queue/delete --status=3          # Reset all failed items
php yii cron-process-queue/delete --case-id=456 --force  # Permanently delete
php yii cron-process-queue/delete --date=2026-03-21   # Reset items from a date
```

**Options:** `--id`, `--case-id`, `--status`, `--date`, `--force`

- Without `--force`: resets items to `status=0, attempts=0` (ready to reprocess)
- With `--force`: permanently deletes matching rows
- At least one filter is required (safety guard)
- Prompts for confirmation before executing

---

### 4. `cron-process-queue/process-pdf-to-text` — Extract Text from PDFs

Processes orders that have a `file_name` but no `order_text`.

```bash
php yii cron-process-queue/process-pdf-to-text
```

- Fetches `batchSize` orders from `mjbk_orders` where `order_text` is null/empty
- Calls `Yii::$app->utility->getPdfToText($file_name, true)` for each
- Updates `order_text` on success

---

### 5. `cron-process-queue/email-orders-for-cases` — Queue Email Notifications

Finds orders created/updated in the last 24 hours and queues notifications.

```bash
php yii cron-process-queue/email-orders-for-cases
```

**Criteria:**
- Order `date_of_order >= case created_at` (no stale orders)
- Order `created_at >= yesterday 00:00:00`
- Order `date_of_order >= 30 days ago` (no very old orders)
- User `status` in `[1, 3]` (active or blocked)

For each matching case, it calls:
- `sendUserItemToQueue()` — queues notifications for team members (`type_record_process_order_cases`)
- `sendClientPOCItemToQueue()` — queues notifications for clients and POC contacts

---

### 6. `cron-process-queue/blast-order-cases-emails` — Dispatch Notification Batches

Orchestrates batch sending by spawning sub-commands for each recipient type.

```bash
php yii cron-process-queue/blast-order-cases-emails
php yii cron-process-queue/blast-order-cases-emails --debug-email-flag=1
```

Processes three queue types in parallel cycles:

| Queue Type | Group By | Sub-command |
|---|---|---|
| `type_record_process_order_cases` | `user_id` | `bulk-send-order-cases-notification` |
| `type_record_process_order_cases_clients` | `client_id` | `bulk-send-client-order-cases-notification` |
| `type_record_process_order_cases_POC` | `POC_client_id` | `bulk-send-poc-client-order-cases-notification` |

---

### 7–9. Bulk Send Actions

Process the queued notifications and send emails/WhatsApp.

```bash
php yii cron-process-queue/bulk-send-order-cases-notification {offset} {limit} {debug_flag}
php yii cron-process-queue/bulk-send-client-order-cases-notification {offset} {limit} {debug_flag}
php yii cron-process-queue/bulk-send-poc-client-order-cases-notification {offset} {limit} {debug_flag}
```

Each follows the same pattern:
1. Fetch distinct recipient IDs with `status=pending` (paginated by offset/limit)
2. Bulk-update their queue items to `status=processing`
3. For each recipient: build HTML email content, send email + WhatsApp
4. Update queue items to `status=processed`
5. Log activity to `cron-order-cases-log`

With `debug_email_flag=1`, emails are sent to `Yii::$app->params['DEBUG_EMAIL']` instead of actual recipients.

---

## Core Service: `OrderProcessor::process($case)`

Located at `services/OrderProcessor.php`. Called by `actionIndex` for each queued case.

**Flow:**
1. Fetches parse data for the case from the court parsing system
2. Determines the CNR (uses `cases.cnr` if `is_cnr=1`, else `case_unique_number`)
3. Fetches existing orders from `mjbk_orders` for deduplication
4. For each new order found:
   - Validates URL and date
   - Uploads PDF to S3 via `Utility::uploadToS3FromUrl()`
   - Extracts text from PDF via `Utility::getPdfToText()`
   - Creates push notifications via `Utility::SendPushNotificationForOrders()`
5. Returns `['status' => true/false, 'message' => '...']`

---

## Components

### `QueueLogger`

Dual-destination logger. All module actions use this for consistent output.

```php
QueueLogger::info('build', 'Found 100 cases', $filters, $details, $caseId, $queueId, $rawSql);
QueueLogger::error('process', 'Failed to process case', null, null, $caseId);
QueueLogger::success('delete', '5 items deleted');
QueueLogger::warning('build', 'Skipping case with no CNR');
```

**Log destinations** (controlled by `$logtype` param):
- `'both'` (default): console output + DB insert into `mjbk_order_processing_log`
- `'yii'`: Yii logger only (file/DbTarget)
- `'db'`: DB only
- `'none'`: console only

Console output is ANSI color-coded: green (success), red (error), yellow (warning), default (info).

### `Utility` (registered as `Yii::$app->utility`)

| Method | Description |
|---|---|
| `uploadToS3FromUrl($url, $s3Key, $retry, $caseId)` | Downloads from URL, uploads to S3. Exponential backoff retry. Returns `['success', 'url', 'error']` |
| `getPdfToText($file_url, $batch)` | Calls internal Python PDF parsing service. Returns extracted text string |
| `SendPushNotificationForOrders($order_id, $date_of_order, $modelCase)` | Creates push notifications for team members |
| `isValidURL($url)` | Validates http/https URL |
| `isValidDate($date)` | Validates date, must be after 1971-01-01. Returns `Y-m-d` or null |
| `getErrorDetails($e, $msg, $file, $line, $trace)` | Standardized exception formatting |
| `ObjToArrayWithRename($data)` | Recursively converts objects to arrays, renames `order_date`→`date`, `judgement`→`order_name` |

---

## Web Dashboard

**Access:** Superadmin only (controlled by `Yii::$app->params['superadmin_access']` array of user IDs).

| Route | Description |
|---|---|
| `/orderprocessing` | Queue grid with live stats (polls `/orderprocessing/stats` every 2 seconds) |
| `/orderprocessing/stats` | JSON endpoint: total/pending/processing/done/failed counts |
| `/orderprocessing/logs` | Log viewer with filters by action, level, case_id, message |

---

## Recommended Cron Schedule

```
# Step 1: Build queue daily (adjust filters as needed)
0 1 * * * php /var/www/html/yii cron-process-queue/build --case-status=1,2

# Step 2: Process queue (run multiple workers in parallel)
*/5 * * * * php /var/www/html/yii cron-process-queue/index
*/5 * * * * php /var/www/html/yii cron-process-queue/index

# Step 3: PDF text extraction
*/10 * * * * php /var/www/html/yii cron-process-queue/process-pdf-to-text

# Step 4: Queue email notifications (run after orders are processed)
0 8 * * * php /var/www/html/yii cron-process-queue/email-orders-for-cases

# Step 5: Send queued notifications
15 8 * * * php /var/www/html/yii cron-process-queue/blast-order-cases-emails
```

---

## Notification Queue Types

| Type constant | Description | Grouped by |
|---|---|---|
| `type_record_process_order_cases` | Team member notifications | `user_id` |
| `type_record_process_order_cases_clients` | Client notifications | `client_id` |
| `type_record_process_order_cases_POC` | Point-of-contact notifications | `POC_client_id` |

Items expire at `next day 00:00:00` (`item_exp_time`).

---

## Error Handling Patterns

- All cron actions wrap logic in `try/catch (\Throwable $e)` — failures are logged, not re-thrown
- `actionIndex` uses `FOR UPDATE SKIP LOCKED` for safe concurrent execution
- Failed items (`status=3`) are retried up to `numberoftries` times automatically
- Stale locks (processing > 30 min) are reclaimed on next run
- `actionDelete` requires at least one filter to prevent accidental full-table wipes

---

## Adding a New Processing Action

1. Add your action method to `ProcessQueueController`
2. Register it in `options()` if it needs CLI option flags
3. Use `QueueLogger::info/error/success()` for all logging
4. Wrap the entire action body in `try/catch (\Throwable $e)`
5. Return `0` on success, `1` on failure (Yii console convention)