0019. AWS single-instance deployment for the beta

Design document, published from the repository at docs/adr/0019-aws-single-instance-beta.md.

  • Status: Accepted
  • Date: 2026-11-18

Context

The beta needs the website, API, proof worker, indexer, PostgreSQL and Redis running on a public HTTPS address at the lowest reasonable cost. Traffic is expected to be small. The images in infrastructure/docker/ already run the whole stack under Docker Compose. Mainnet is out of scope (the release gate requires an external audit); the beta uses Arbitrum Sepolia (421614).

Decision

One ARM (Graviton) EC2 instance in us-east-1 runs the stack with Docker Compose (infrastructure/aws/compose.yaml), defined in CloudFormation (infrastructure/aws/stack.yaml) and operated with infrastructure/aws/deploy.mjs:

  • The edge has two modes (UseCloudFront). direct (the default, because new AWS accounts cannot create CloudFront distributions until AWS verifies them): Caddy on the instance serves HTTPS with a Let's Encrypt certificate for <elastic-ip>.sslip.io (or a custom domain) and redirects HTTP. cloudfront: CloudFront terminates HTTPS and caches only /_next/static/*; the instance accepts port 80 only from CloudFront's origin-facing prefix list, and nginx refuses any request without this deployment's X-Origin-Verify secret.
  • Only the website is public. The browser reaches the API through the website's backend-for-frontend (/api/ql); the API, worker and indexer have no public ports.
  • PostgreSQL data lives on a separate encrypted EBS volume, snapshotted daily (7 kept); Redis holds only transient data.
  • Secrets are generated once by deploy.mjs into SSM Parameter Store (SecureString, AWS managed key) and rendered on the instance into per-service env files. The web container never sees the KEK; the backend never sees the session secret.
  • Images are built for arm64, pushed to ECR (immutable tags, 5 kept) and deployed with SSM Run Command. There is no SSH access.
  • The backend runs with QUANTLAYER_ENVIRONMENT=staging, so the API's deployed-environment checks apply.

Consequences

  • Low fixed cost and one place to look; no NAT gateway, load balancer or managed database.
  • Single point of failure: an instance or Availability Zone failure means downtime until the instance is replaced and the data volume is reattached or restored from a snapshot.
  • In direct mode there is no CDN: every request reaches the instance. In cloudfront mode, traffic between CloudFront and the instance is plain HTTP over AWS's network until a custom domain gives the origin its own certificate.
  • sslip.io is a third-party DNS service that maps the host name to the IP; a custom domain removes that dependency.
  • The public API for SDK users is not exposed yet; adding it is a CloudFront behaviour and an nginx location.
  • Moving to managed services (ECS Fargate, RDS, ElastiCache) reuses the same images and env variables.