# ARIA Scaling Architecture
## Multi-Instance Deployment Plan

> **Author:** ARIA  
> **Date:** 2026-02-10  
> **Status:** Draft — Awaiting Review  
> **Audience:** Symph Engineering Team

---

## 1. Executive Summary

This document outlines a 3-phase plan to scale ARIA from a single-instance deployment to a multi-agent architecture. The goal: handle more concurrent work (builds, discovery, comms) without bottlenecks, while maintaining the file-based simplicity that works today.

**Current state:** 1 VPS, 1 ARIA, file-based state, 16 concurrent sessions  
**Target state:** N specialized ARIA siblings, shared state, job-based dispatch

---

## 2. Current Architecture

```
┌─────────────────────────────────────────────────┐
│                  VPS (Single)                    │
│                                                  │
│  ┌──────────┐  ┌──────────┐  ┌───────────────┐  │
│  │ OpenClaw │  │  PM2     │  │    Caddy       │  │
│  │ Gateway  │  │ (Agency  │  │ (Reverse Proxy)│  │
│  │          │  │  UI:4000)│  │ *.dev.apps...  │  │
│  └────┬─────┘  └──────────┘  └───────────────┘  │
│       │                                          │
│  ┌────┴─────┐                                    │
│  │  ARIA    │                                    │
│  │ (main)   │──── Claude API (Opus 4.6)          │
│  │          │                                    │
│  │ Sessions:│    ┌─────────────────────┐         │
│  │ • Discord│    │   Workspace (files) │         │
│  │ • Cron   │    │                     │         │
│  │ • Builds │    │ • MEMORY.md         │         │
│  │ • Subs   │    │ • agency/products/* │         │
│  └──────────┘    │ • agency/People/*   │         │
│                  │ • memory/*.md       │         │
│                  │ • agency/jobs/*     │         │
│                  └─────────────────────┘         │
│                                                  │
│  ┌──────────────────────────────────┐            │
│  │  Product Apps (PM2 / direct)     │            │
│  │  • AI Teacher Assistant :3003    │            │
│  │  • Sthics :3011                  │            │
│  │  • MetCabinet :3012              │            │
│  │  • ... (ports 3001-3100)         │            │
│  └──────────────────────────────────┘            │
└─────────────────────────────────────────────────┘
         │
    ┌────┴────┐
    │ Discord │  (primary comms channel)
    │ GitHub  │  (code repos)
    └─────────┘
```

### Current Bottlenecks

| Bottleneck | Impact | Frequency |
|-----------|--------|-----------|
| **Session cap (16)** | Long builds block other work | Daily during active builds |
| **CPU contention** | Next.js builds + Claude Code + app hosting compete | During builds |
| **Single API quota** | Heavy days exhaust Anthropic rate limits | Weekly |
| **File write races** | Concurrent subagents editing same YAML | Occasional |
| **Single point of failure** | VPS down = everything down | Rare but catastrophic |

---

## 3. Phase 1: Dual-Instance Split (This Week)

**Goal:** Offload builds to a dedicated second instance. Immediate relief.

```
┌─────────────────────────┐    ┌─────────────────────────┐
│     VPS-1 (Comms)       │    │    VPS-2 (Builder)      │
│                         │    │                         │
│  ┌──────────┐           │    │  ┌──────────┐           │
│  │ ARIA-1   │           │    │  │ ARIA-2   │           │
│  │ "Comms"  │           │    │  │ "Builder" │           │
│  └────┬─────┘           │    │  └────┬─────┘           │
│       │                 │    │       │                 │
│  Handles:               │    │  Handles:               │
│  • Discord messages     │    │  • Claude Code builds   │
│  • Heartbeats & cron    │    │  • QA testing           │
│  • Discovery research   │    │  • App deployments      │
│  • Quick tasks          │    │  • Long-running jobs    │
│  • Job dispatch         │    │                         │
│                         │    │  ┌──────────────────┐   │
│  ┌──────────────────┐   │    │  │ Product Apps     │   │
│  │ Symph Agency UI  │   │    │  │ (all dev apps)   │   │
│  │ (PM2, port 4000) │   │    │  │ ports 3001-3100  │   │
│  └──────────────────┘   │    │  └──────────────────┘   │
│                         │    │                         │
│  Model: Opus 4.6        │    │  Model: Sonnet 4.5      │
│  (complex reasoning)    │    │  (fast builds, cheaper)  │
└────────────┬────────────┘    └────────────┬────────────┘
             │                              │
             │    ┌──────────────────┐      │
             └────┤   GitHub Repos   ├──────┘
                  │  (shared state)  │
                  └──────────────────┘
```

### 3.1 How It Works

**Job Dispatch (ARIA-1 → ARIA-2):**
1. ARIA-1 receives a build request (e.g., "Build GaragePro")
2. ARIA-1 writes a job file to `agency/jobs/build-garagepro.yaml` with `status: pending`
3. ARIA-1 commits + pushes to GitHub
4. ARIA-2's heartbeat picks up pending jobs (polls every 5-10 min)
5. ARIA-2 executes the build, updates job status → `in_progress` → `complete`
6. ARIA-2 commits + pushes results
7. ARIA-1's heartbeat sees completion, notifies Discord

**Alternatively (faster, no polling):**
- ARIA-1 uses `sessions_send` or a webhook to ping ARIA-2 directly
- Requires both instances to be on the same network or have API access to each other

### 3.2 State Sync Strategy

| Data Type | Sync Method | Conflict Risk |
|-----------|-------------|---------------|
| Product artifacts (discovery, design, build reports) | Git push/pull | Low — write-once |
| Product config (`product.yaml`) | Git push/pull | Medium — both may edit |
| Memory (`MEMORY.md`, `memory/*.md`) | Git push/pull | Low — different daily files |
| People profiles | Git push/pull | Low — rarely concurrent |
| Job queue (`agency/jobs/*.yaml`) | Git push/pull | Medium — status transitions |
| Cost tracking | Git push/pull | Low — append-only |
| Chat messages | Stay on VPS-1 only | None |

**Conflict Resolution Rules:**
- Always `git pull --rebase` before writing
- Job files use status transitions: `pending` → `in_progress` → `complete/failed`
- Only the assigned instance writes to a job once claimed (set `assigned_to: aria-2`)
- Product artifacts: builder writes, comms reads — no conflict

### 3.3 Setup Steps

1. **Provision VPS-2** (4+ CPU, 8GB+ RAM for builds)
2. **Install OpenClaw** on VPS-2 with separate config
3. **Clone workspace** from GitHub on VPS-2
4. **Configure ARIA-2:**
   - Different Discord bot token (or no Discord — just job runner)
   - Model: `claude-sonnet-4-5` (cost-effective for builds)
   - Workspace: same repo, different clone
   - Heartbeat: check `agency/jobs/` for pending work
5. **Move product apps** to VPS-2 (Caddy + PM2)
6. **Update DNS** — point `*.dev.apps.symph.co` to VPS-2
7. **Test:** ARIA-1 dispatches a build, ARIA-2 picks it up and executes

### 3.4 Cost Estimate

| Item | Monthly Cost |
|------|-------------|
| VPS-2 (4 CPU, 8GB RAM) | ~$24-48/mo |
| Additional API usage (Sonnet 4.5 for builds) | ~$50-100/mo |
| **Total additional** | **~$75-150/mo** |

---

## 4. Phase 2: Database-Backed State (2-4 Weeks)

**Goal:** Replace file-based state for hot data with Firestore. Eliminate race conditions.

```
┌──────────┐  ┌──────────┐  ┌──────────┐
│ ARIA-1   │  │ ARIA-2   │  │ ARIA-N   │
│ (Comms)  │  │ (Builder)│  │ (Future) │
└────┬─────┘  └────┬─────┘  └────┬─────┘
     │             │             │
     └──────┬──────┴──────┬──────┘
            │             │
     ┌──────┴──────┐ ┌────┴──────┐
     │  Firestore  │ │  GitHub   │
     │  (hot state)│ │  (cold    │
     │             │ │  artifacts│
     │ • products  │ │           │
     │ • jobs      │ │ • reports │
     │ • people    │ │ • designs │
     │ • memory    │ │ • code    │
     │ • costs     │ │ • docs    │
     └─────────────┘ └───────────┘
```

### 4.1 What Moves to Firestore

| Collection | Source | Why |
|-----------|--------|-----|
| `aria_products` | `product.yaml` per product | Concurrent reads/writes from multiple agents |
| `aria_jobs` | `agency/jobs/*.yaml` | Real-time job dispatch, no polling needed |
| `aria_people` | `agency/People/*.md` | Profile lookups during conversations |
| `aria_memory` | `MEMORY.md` + daily files | Shared memory across instances |
| `aria_costs` | `costs.yaml` per product | Append-heavy, needs atomics |

### 4.2 What Stays in Git

- Discovery reports, PRDs, user stories (write-once markdown)
- Design assets and brand guidelines (binary + markdown)
- Build plans and summaries (write-once)
- Code repositories (obviously)
- SKILL.md files, AGENTS.md, SOUL.md (config)

### 4.3 Migration Path

1. Create Firestore collections with schema matching current YAML structure
2. Write import script: YAML → Firestore (one-time migration)
3. Update OpenClaw workspace tools or add a custom skill for DB reads/writes
4. Update Symph Agency UI to read from Firestore instead of filesystem
5. Keep git as backup/export — nightly dump Firestore → YAML → git push

### 4.4 Job Queue (Real-Time)

```yaml
# Firestore: aria_jobs/{jobId}
{
  id: "build-garagepro-20260210",
  type: "build",
  product_slug: "motor-vehicle-maintenance",
  status: "pending",        # pending → claimed → in_progress → complete/failed
  assigned_to: null,        # ARIA instance that claimed it
  created_by: "aria-1",
  created_at: "2026-02-10T04:30:00Z",
  claimed_at: null,
  completed_at: null,
  result: null,
  priority: "normal"
}
```

- Agents listen to Firestore `onSnapshot` — instant job pickup, no polling
- Claim uses Firestore transaction (atomic, no double-claim)
- Status updates visible to all instances immediately

---

## 5. Phase 3: Specialized Multi-Agent (1-2 Months)

**Goal:** Purpose-built siblings with domain expertise.

```
                    ┌─────────────────┐
                    │   Dispatcher    │
                    │  (Router/LB)    │
                    └───────┬─────────┘
                            │
          ┌─────────┬───────┼───────┬─────────┐
          │         │       │       │         │
     ┌────┴───┐ ┌───┴──┐ ┌─┴────┐ ┌┴──────┐ ┌┴───────┐
     │Scout   │ │Design│ │Build │ │Market │ │Operate │
     │(Disc.) │ │Agent │ │Agent │ │Agent  │ │Agent   │
     │        │ │      │ │      │ │       │ │        │
     │Haiku   │ │Sonnet│ │Sonnet│ │Sonnet │ │Haiku   │
     │4.5     │ │4.5   │ │4.5   │ │4.5    │ │4.5     │
     └────────┘ └──────┘ └──────┘ └───────┘ └────────┘
          │         │       │       │         │
          └─────────┴───────┼───────┴─────────┘
                            │
                    ┌───────┴─────────┐
                    │   Shared State  │
                    │  Firestore + Git│
                    └─────────────────┘
```

### 5.1 Agent Roles

| Agent | Model | Responsibility | Concurrency |
|-------|-------|---------------|-------------|
| **ARIA Prime** | Opus 4.6 | Discord comms, coordination, complex reasoning, human interaction | 1 instance |
| **Scout** | Haiku 4.5 | Market research, discovery, competitor analysis | 2-3 parallel |
| **Designer** | Sonnet 4.5 | Brand guidelines, screen generation, Figma automation | 1 instance |
| **Builder** | Sonnet 4.5 | Claude Code builds, QA, deployments | 2-3 parallel |
| **Marketer** | Sonnet 4.5 | Landing pages, content, campaigns | 1 instance |
| **Operator** | Haiku 4.5 | Health checks, monitoring, alerts | 1 instance |

### 5.2 Dispatch Logic

```
User Request → ARIA Prime
  │
  ├── "Build this app" → Builder Agent (VPS-2)
  ├── "Research this market" → Scout Agent (serverless/VPS-2)
  ├── "Design the brand" → Designer Agent (VPS-2)
  ├── "Create landing page" → Marketer Agent (VPS-2)
  ├── "Is the app healthy?" → Operator Agent (VPS-1)
  └── Everything else → ARIA Prime handles directly
```

### 5.3 Cost Optimization

| Model | Cost/MTok (blended) | Use Case |
|-------|-------------------|----------|
| Opus 4.6 | ~$45 | Human conversation, complex decisions |
| Sonnet 4.5 | ~$9 | Builds, design, marketing (5x cheaper) |
| Haiku 4.5 | ~$2.40 | Research, monitoring, simple tasks (19x cheaper) |

**Estimated monthly cost at scale:**
- ARIA Prime (Opus): ~$200-400/mo (human interactions only)
- 3 Builders (Sonnet): ~$150-300/mo
- 2 Scouts (Haiku): ~$20-40/mo
- 1 Designer + 1 Marketer (Sonnet): ~$50-100/mo
- 1 Operator (Haiku): ~$10-20/mo
- **Total API: ~$430-860/mo**
- **Infra (2-3 VPS): ~$75-150/mo**
- **Grand total: ~$500-1000/mo for a full AI agency**

---

## 6. Risk Mitigation

| Risk | Mitigation |
|------|-----------|
| Git merge conflicts | Claim-based ownership — only assigned agent writes to a job/product |
| Stale state | Phase 2 Firestore eliminates this; Phase 1 uses pull-before-read |
| API quota exhaustion | Model routing (Opus only for complex); separate API keys per instance |
| VPS-2 failure | Builds queue up; ARIA-1 can still do everything (just slower) |
| Data inconsistency | Nightly reconciliation job — compare Firestore ↔ git, flag mismatches |
| Runaway costs | Per-agent daily cost caps; alert if any agent exceeds $50/day |

---

## 7. Implementation Timeline

| Week | Milestone | Effort |
|------|-----------|--------|
| **Week 1** | Phase 1: Provision VPS-2, install OpenClaw, clone workspace | 1 day |
| **Week 1** | Phase 1: Configure ARIA-2 as build runner, test job dispatch | 1 day |
| **Week 1** | Phase 1: Move product apps to VPS-2, update DNS | Half day |
| **Week 2** | Phase 1: Stabilize, monitor, fix edge cases | Ongoing |
| **Week 3-4** | Phase 2: Firestore schema + migration script | 2-3 days |
| **Week 3-4** | Phase 2: Update Agency UI to read from Firestore | 2-3 days |
| **Week 3-4** | Phase 2: Real-time job queue with Firestore listeners | 1-2 days |
| **Month 2** | Phase 3: Specialized agents, dispatcher, cost routing | 1-2 weeks |

---

## 8. Decision Points

Before proceeding, the team should decide:

1. **Phase 1 scope:** Just a build runner, or should ARIA-2 also handle discovery?
2. **VPS provider:** Same provider as current? (Hetzner, DigitalOcean, etc.)
3. **Separate Discord bot?** ARIA-2 could have its own bot for build notifications, or just relay through ARIA-1
4. **API key strategy:** Same Anthropic org, or separate accounts for quota isolation?
5. **Timeline priority:** Start Phase 1 this week, or wait for a quieter period?

---

## 9. Success Metrics

| Metric | Current | Phase 1 Target | Phase 3 Target |
|--------|---------|----------------|----------------|
| Concurrent builds | 1-2 | 3-4 | 6-8 |
| Build queue wait time | 0-30 min | < 5 min | < 1 min |
| Response time (Discord) | 5-15s (blocked during builds) | < 5s always | < 3s always |
| Monthly API cost | ~$300-500 | ~$400-600 | ~$500-1000 |
| Uptime | ~99% (single VPS) | ~99.5% (degraded if VPS-2 down) | ~99.9% |
| Products built/week | 2-3 | 5-8 | 10-15 |

---

*This is a living document. Updated as decisions are made and phases are implemented.*

*— ARIA, 2026-02-10*
