Deployment

The supported V03 production path is a Linux VPS with host PostgreSQL 17, PM2, and Caddy. A web-only deployment is incomplete because durable tasks, browser isolation, fulfillment, Push, and retention require long-running worker services.

Before the first deploy

Point public, admin, and three Demo hostnames at the VPS.
Create bucket-scoped R2 credentials and a Polar product/webhook.
Generate independent database, auth, API, browser, voter, Push, and download secrets.
Create separate database migration and forced-RLS runtime roles.
Assemble and sign the release artifact; retain the previous signed release.
Record Node, pnpm, PostgreSQL, PM2, and Caddy versions in release evidence.

Store runtime secrets in a root-owned 0600 environment file outside the immutable release directory. Never expose the migration URL to PM2.

Install and verify the release

git checkout --detach <release-commit>
if [[ -f .gitmodules ]]; then git submodule update --init --recursive; fi
corepack enable
corepack prepare pnpm@11.9.0 --activate
pnpm install --frozen-lockfile
pnpm check
pnpm test
pnpm build

Take and validate a custom-format database backup before upgrading. Apply the checked-in migrations and idempotent import with the migration owner:

source "${AWP_RUNTIME_ENV:?set AWP_RUNTIME_ENV to the root-owned runtime env file}"
export NEXT_PUBLIC_SITE_URL NEXT_PUBLIC_SITE_NAME SITE_SLUG
export COMMERCE_DEMO_HOST EDUCATION_DEMO_HOST HEALTH_DEMO_HOST
export DATABASE_URL="$MIGRATION_DATABASE_URL"
export APP_DATABASE_ROLE=aiwc_app
bash scripts/ops/migrate-and-grant.sh
V03_DATA_MODE=upgrade bash scripts/ops/bootstrap-v03-data.sh

Use V03_DATA_MODE=fresh for an empty database. Use reconcile only after a completed V03 import. The runtime URL and Demo host values must be present for bootstrap so verified production domains become canonical; local defaults are development-only. Never run prisma db push.

Start the five PM2 roles

Load the runtime environment containing the least-privilege DATABASE_URL, then run:

mkdir -p logs
export AWP_RELEASE_DIR="$PWD"
export AWP_LOG_DIR="$PWD/logs"
export AWP_NODE_BINARY=/opt/node-v24.15.0/bin/node
pm2 startOrReload ecosystem.config.cjs --update-env
EXPECTED_BUILD_ID="$BUILD_ID" EXPECTED_NODE_MAJOR=24 pnpm ops:health
pm2 save

Install the encrypted backup service after creating a dedicated read-only BYPASSRLS database role, a private backup bucket, a bucket-only R2 token, and /etc/aiworkflowcenter/backup.env with mode 0600:

ln -sfn "$AWP_RELEASE_DIR" /opt/awp-v03-current
install -m 0644 ops/systemd/awp-v03-backup.service /etc/systemd/system/
install -m 0644 ops/systemd/awp-v03-backup.timer /etc/systemd/system/
systemctl daemon-reload
systemctl enable --now awp-v03-backup.timer
systemctl start awp-v03-backup.service
systemctl status awp-v03-backup.service awp-v03-backup.timer

The timer runs every ten minutes, below the 15-minute RPO. A release is not ready until an encrypted object and manifest exist off-host, their hashes pass, and a fresh isolated restore meets the documented RTO. Pin AWP_PG_DUMP_BIN and AWP_PG_RESTORE_BIN to the PostgreSQL 17 client pair; do not rely on a distribution wrapper that can dispatch the two commands to different majors.

The health gate covers:

3301  awp-v03-web             /api/health
3302  awp-v03-mcp             /api/health
3303  awp-v03-worker          /healthz
3304  awp-v03-scheduler       /healthz
3310  awp-v03-browser-worker  /healthz

All ports bind to loopback. Install PM2 startup integration for a dedicated, non-root service account.

Configure host Caddy

Install the checked-in Caddyfile, provide its five domain environment variables through systemd, and validate before reload:

caddy validate --config /etc/caddy/Caddyfile
systemctl reload caddy

The admin host exposes only /admin* and /api/auth/*. Public and Demo hosts reject that surface. MCP/WebMCP routes go to port 3302; other public traffic goes to port 3301.

Health and smoke checks

curl -fsS https://$DOMAIN/api/health
curl -fsS https://$DOMAIN/.well-known/mcp.json
curl -fsS https://$DOMAIN/robots.txt
curl -fsS https://$DOMAIN/sitemap.xml
curl -fsS https://$ADMIN_DOMAIN/admin/login

Verify all three Demo hosts resolve distinct Site identity and data. The health Demo must be noindex, use synthetic data, and state that it is not medical advice. Run one controlled model task and confirm its model, tokens, cost, provenance, review state, and kill-switch behavior.

Task scheduling

The Scheduler reads versioned Site schedules and writes deterministic task.execution.requested Outbox events. The Worker leases each event, validates it, executes the registered task, records an Inbox receipt, and retries with bounded backoff. Legacy /api/cron/* routes return 410 Gone.

Manual and REST triggers use /api/tasks/{taskId}/trigger. They enqueue work and return 202; they do not run long tasks in the request process.

Rollback and recovery

Application rollback switches AWP_RELEASE_DIR to the preceding signed release and runs pm2 startOrReload followed by all five probes. Do not reverse an irreversible migration during an incident. V03 uses expand, backfill, verify, cut over, observe, then contract; preserve the prior readable shape throughout the rollback window.

Back up PostgreSQL and R2 independently. Restore into an isolated environment, then verify Site counts, relations, RLS, ledgers, entitlements, artifact hashes, queue reconciliation, and download revocation. Record achieved RPO and RTO in the release evidence. Code rollback is not data rollback.

Search AI Workflow Center