# UI Breakage Prevention Measures
**Date:** 2026-02-09  
**Related:** `ui-breakage-root-cause.md`

## Immediate Fixes Implemented

### 1. Code Fix - Null Safety
**File:** `/home/aria/projects/symph-agency/src/app/api/discovery/opportunities/route.ts`

Added defensive checks for undefined field values:
```typescript
// Skip entries without title
if (!scanOpp.title) {
  continue;
}

// Null-safe duplicate check
const isDuplicate = allOpportunities.some(
  (existing) => {
    if (!existing.name || !opp.name) return false;
    // ... safe toLowerCase() calls
  }
);
```

### 2. Startup Script - Port Conflict Prevention
**File:** `/home/aria/projects/symph-agency/start.sh`

Created startup script that:
- Kills zombie processes on port 4000
- Verifies port is free before starting
- Performs health check after startup
- Logs PID for future management

**Usage:**
```bash
cd /home/aria/projects/symph-agency
./start.sh
```

### 3. Process Management - No PM2 (For Now)
**Why:** PM2 was causing a crash loop due to npm wrapper exit behavior. Port 4000 would be occupied by previous instance while PM2 tried to restart.

**Current approach:** Simple background process with nohup + PID tracking

**Future:** Re-evaluate PM2 after testing stability for 48h. If stable, can reconsider PM2 with proper configuration.

## Longer-Term Prevention

### A. Scan File Validation
**Priority:** HIGH  
**Effort:** 1-2 hours

Create validation schema for scan YAML files:
```typescript
// agency/discovery/lib/scan-validator.ts
interface ValidScanOpportunity {
  title: string;           // REQUIRED
  source: string;          // REQUIRED
  signal: string;          // REQUIRED
  insight: string;         // REQUIRED
  tags?: string[];         // optional
  score: number;           // required, 1-5
}
```

Add validation to scan scripts BEFORE writing YAML files.

### B. API Error Boundaries
**Priority:** MEDIUM  
**Effort:** 2-3 hours

Wrap scan file parsing in try-catch to isolate failures:
```typescript
for (const file of scanFiles) {
  try {
    // parse and process
  } catch (err) {
    console.error(`Skipping malformed scan file ${file}:`, err);
    continue; // Don't crash entire endpoint
  }
}
```

### C. Health Monitoring
**Priority:** MEDIUM  
**Effort:** 4-6 hours

Implement health check endpoint and monitoring:
1. `/api/health` endpoint (CPU, memory, uptime, error rates)
2. Hourly health checks via cron
3. Auto-alert to Discord #aria if errors > threshold
4. Auto-restart if health check fails 3x in a row

### D. Deploy Script Safety
**Priority:** HIGH  
**Effort:** 2-3 hours

Update deploy script with:
- Pre-deploy health check
- Graceful shutdown (not kill -9)
- Post-deploy verification
- Auto-rollback if new deploy fails health check
- Port conflict detection before start

**Example flow:**
```bash
# In deploy.sh
check_health_or_exit
kill_gracefully_with_timeout
verify_port_free
npm run build
start_new_instance
sleep 10
verify_health_or_rollback
```

### E. Type Safety at Runtime
**Priority:** LOW  
**Effort:** 3-4 hours

Use Zod or similar for runtime validation:
```typescript
import { z } from 'zod';

const ScanOpportunitySchema = z.object({
  title: z.string().min(1),
  source: z.string(),
  // ...
});

// In API route
const scanData = ScanOpportunitySchema.safeParse(rawData);
if (!scanData.success) {
  console.warn(`Invalid scan file ${file}:`, scanData.error);
  continue;
}
```

## Testing Checklist

Before marking this issue as "resolved for good":

- [ ] App runs for 48h without restarts (check daily)
- [ ] Discovery page loads without errors
- [ ] API endpoint `/api/discovery/opportunities` returns 200
- [ ] No `TypeError` in logs
- [ ] New scan files with missing fields don't crash the app
- [ ] Startup script successfully prevents port conflicts

## Rollout Timeline

| Phase | Action | Timeline |
|-------|--------|----------|
| ✅ Phase 1 | Fix code bug + startup script | DONE (2026-02-09) |
| ⏳ Phase 2 | Monitor stability 48h | 2026-02-09 → 02-11 |
| 🔜 Phase 3 | Scan file validation | 2026-02-11 |
| 🔜 Phase 4 | Health monitoring + deploy safety | 2026-02-12 |

## Monitoring

**Where to check:**
- **Logs:** `/home/aria/projects/.aria-logs/symph-agency.log`
- **PID:** `/home/aria/projects/symph-agency.pid`
- **Process:** `ps aux | grep $(cat /home/aria/projects/symph-agency.pid)`
- **Port:** `lsof -i :4000`
- **Health:** `curl http://localhost:4000/api/health` (once implemented)

**Red flags:**
- Multiple next-server processes on port 4000
- PID file doesn't match running process
- Log file shows repeated startups
- API returns 500 errors
