RaftDB Infrastructure & Route 53

The final two modules provide the DNS/TLS foundation for the split-domain architecture and the RaftDB staging infrastructure — an optional second CloudFormation stack used for qualification, recovery drills, and member replacement practice.

Module overview

1. Route 53 & ACM Certificate

The Route 53 module imports an existing hosted zone and creates a wildcard ACM certificate covering all subdomains.

export function createRoute53AndCertificates(
  scope: Construct,
  input: Route53AndCertInput
): Route53AndCertOutput {
  const { domainName, hostedZoneId } = input;

  // Import existing hosted zone (must exist before deployment)
  const hostedZone = route53.HostedZone.fromHostedZoneAttributes(
    scope, 'HostedZone',
    { hostedZoneId, zoneName: domainName }
  );

  // Wildcard certificate covering *.domainName
  const wildcardCert = new acm.Certificate(scope, 'WildcardCertificate', {
    domainName: `*.${domainName}`,
    validation: acm.CertificateValidation.fromDns(hostedZone),
  });

  return { hostedZone, wildcardCert };
}

a) Certificate consumers

The wildcard certificate is used by two modules:

Separate TLS for Amplify: The root domain (domain.com) has its TLS managed by Amplify Hosting — Amplify provisions and manages its own ACM certificate. The wildcard certificate only covers subdomains.

b) Why import the hosted zone?

route53.HostedZone.fromHostedZoneAttributes(scope, 'HostedZone', {
  hostedZoneId,
  zoneName: domainName,
});

The hosted zone is imported, not created by CDK. It must exist before the first deployment. This is common for production domains where the hosted zone is created manually or by a separate bootstrap stack. The CDK stack only creates DNS records within the existing zone — it never creates or deletes the zone itself.

c) DNS validation

validation: acm.CertificateValidation.fromDns(hostedZone)

CDK uses DNS validation (not email validation) for the ACM certificate. It automatically creates the required _validation CNAME records in the hosted zone, waits for them to propagate, and cleans them up after validation completes. This means certificate renewal is fully automated — ACM re-validates before expiry without manual intervention.

2. RaftDB Staging Stack

The RaftDbStagingStack is an optional second CloudFormation stack created when ENABLE_RAFTDB=true. It provides a disposable, isolated environment for RaftDB qualification, recovery drills, and member replacement practice.

This stack is NEVER used in production. The production RaftDB runs as a sidecar in the main ECS task (see 5.4.4 ECS Compute).

a) Stack creation (conditional)

if (process.env.ENABLE_RAFTDB === 'true') {
  const stagingStack = new RaftDbStagingStack(app, 'RaftDbStagingStack', {
    env,
    imageDigest: process.env.RAFTDB_IMAGE_DIGEST,
    dataGeneration: process.env.RAFTDB_DATA_GENERATION,
    restoreFromS3: parseRestoreFromS3(process.env.RAFTDB_RESTORE_FROM_S3),
    nodeCount: parseNodeCount(process.env.RAFTDB_NODE_COUNT),
  });
  stagingStack.addDependency(mainStack,
    'RaftDB staging imports the shared ECR repository');
}

b) Stack configuration parameters

c) VPC configuration by node count

The single-node mode opens port 9100 to the internet for qualification tests. The three-node mode is fully isolated — only Raft peer traffic on port 9101, no public access.

d) Staging stack resources

e) Monitoring: CloudWatch alarms and dashboard

The CloudWatch dashboard (lib/dashboard.ts) adds Raft-specific widget groups:

Dashboard Widgets
├── Raft State       → Leader/Follower/Candidate transitions per member
├── Log Replication  → Applied index, commit index, last log index
├── Snapshots        → Snapshot age, size, count
├── Network          → RUDP retransmissions, TCP connection count
└── WAL              → WAL errors, bytes written, segment count

These custom metrics are emitted by the RaftDB C++ process and provide deep visibility into Raft consensus health — essential for qualification and recovery drills.

f) Member replacement procedure

The staging stack is designed to support the documented member replacement procedure:

Step 1: REMOVE  → Remove old member from Raft configuration
                  (committed membership change via raftdb-cli)

Step 2: STOP    → Stop old ECS task (desiredCount=0)

Step 3: PROVISION → Bump dataGeneration (e.g., staging-1 → staging-2)
                    Creates fresh EFS access point → clean WAL

Step 4: START   → Start replacement task (desiredCount=1)
                  Empty WAL, clean catalog

Step 5: INSTALL → Pull snapshot + WAL tail from a healthy peer

Step 6: ADD     → Add member back to Raft cluster

Step 7: VERIFY  → Confirm quorum reached + applied index converged

Safety rules enforced by process:

• Never remove two voters at once — would cause loss of quorum and cluster unavailability
• Never reuse a member ID while another task with that ID could still run
• Emergency quorum changes require written approval + verified backup
• dataGeneration must change on every replacement** — prevents accidental WAL reuse

3. RaftDB Cluster Factory

The lib/raftdb.ts module provides reusable factory functions for creating RaftDB clusters:

These factories are used by the staging stack and can be composed for production multi-node deployments if needed. The factory pattern keeps the staging logic testable — the raftdb contract tests synthesize templates with different cluster configurations (1-member, 3-member, restore mode) and validate the generated CloudFormation.

4. Complete Infrastructure Diagram

Here is the complete infrastructure deployed by the CDK stacks:

┌────────────────────────────────────────────────────────────────────┐
│  Route 53 Hosted Zone (domain.com)                                 │
│  ├── domain.com          → Amplify (managed TLS)                   │
│  ├── api.domain.com      → API Gateway (wildcard cert)             │
│  └── ws.domain.com       → ALB (wildcard cert)                     │
│                                                                    │
│  ACM Wildcard Certificate (*.domain.com)                           │
│  ├── api.domain.com (API Gateway custom domain)                    │
│  └── ws.domain.com  (ALB HTTPS listener)                           │
│                                                                    │
│  VPC (2 AZs, public subnets only, no NAT Gateway)                  │
│  ├── ALB (internet-facing, HTTPS 443 → ECS:8980)                   │
│  │   ├── Health check: GET /health (30s interval)                  │
│  │   └── Idle timeout: 3600s (WebSocket support)                   │
│  ├── ECS Fargate (1 task: 1024 CPU / 2048 MiB)                     │
│  │   ├── RaftDB sidecar (:9100, non-root 10001)                    │
│  │   │   └── EFS mount: /data/raftdb (encrypted)                   │
│  │   └── Go App (:8980)                                            │
│  │       └── depends_on: RaftDB HEALTHY                            │
│  └── Security Groups:                                              │
│      ├── ALB SG: 0.0.0.0/0 → :80, :443                             │
│      └── ECS SG: ALB SG → :8980                                    │
│                                                                    │
│  DynamoDB (4 tables, on-demand billing, composite PK/SK)           │
│  ├── Config (PK=CONFIG, SK=SINGLETON)                              │
│  ├── Bans (PK=BAN#type#id, SK=BAN)                                 │
│  ├── Milestones (PK=MILESTONE, GSI: TriggerAtIndex)                │
│  └── History (PK=HISTORY, GSI: XOriginalIndex)                     │
│                                                                    │
│  S3 Buckets                                                        │
│  ├── awsplace-canvas-{account} (versioned, canvas binary, 32 MiB)  │
│  ├── awsplace-exports-{account} (PNG exports)                      │
│  └── RaftDB snapshots (versioned, 35-day retention)                │
│                                                                    │
│  ECR Repository (awsplace-ecs, keep 10 images, raftdb-* immutable) │
│                                                                    │
│  Lambda (Node.js 24, 512 MB, 30s timeout, NO VPC)                  │
│  └── API Gateway HTTP API v2 ($default → Lambda)                   │
│                                                                    │
│  Amplify Hosting (manual zip deploy, SPA rewrite + /admin.html)    │
│                                                                    │
│  Secrets Manager: awsplace/app-secrets (SESSION_SECRET, etc.)      │
│                                                                    │
│  IAM Roles (3 roles, least-privilege scoped)                       │
│  ├── ECS Task Execution (ECR pull + CW Logs + Secrets Manager)     │
│  ├── ECS Task (DynamoDB CRUD + S3 R/W + EFS mount)                 │
│  └── Lambda Execution (DynamoDB CRUD + S3 PutObject + CW Logs)     │
│                                                                    │
│  [Optional: RaftDbStagingStack — ENABLE_RAFTDB=true]               │
│  ├── Separate VPC (1 AZ or 3 AZs)                                  │
│  ├── ECS Cluster + NLB + Cloud Map                                 │
│  ├── EFS + S3 (per-member storage)                                 │
│  └── CloudWatch Dashboard + Alarms                                 │
└────────────────────────────────────────────────────────────────────┘