Storage: ECR & S3

The storage layer consists of two independent modules: ECR (createEcr) for Docker image storage and S3 (createStorage) for application binary data. A third module, createRaftDbApplicationStorage, provides durable storage for the RaftDB sidecar via EFS and S3.

Module overview

1. ECR Repository

The ECR module creates a single Elastic Container Registry repository named awsplace-ecs. This repository stores both the Go application image and the RaftDB image — two different image tags/digests in the same repository.

export function createEcr(scope: Construct): EcrOutput {
  const repository = new ecr.Repository(scope, 'ApplicationRepository', {
    repositoryName: 'awsplace-ecs',
    imageScanOnPush: true,
    imageTagMutability: ecr.TagMutability.MUTABLE_WITH_EXCLUSION,
    imageTagMutabilityExclusionFilters: [
      ecr.ImageTagMutabilityExclusionFilter.wildcard('raftdb-*'),
    ],
  });

  repository.applyRemovalPolicy(RemovalPolicy.RETAIN);

  repository.addLifecycleRule({
    maxImageCount: 10,
    rulePriority: 1,
    tagStatus: ecr.TagStatus.ANY,
  });

  return { repository };
}

a) Configuration breakdown

b) Why RemovalPolicy.RETAIN?

The ECR repository must survive cdk destroy cycles. The sequence matters:

CI pipeline pushes image → CDK deploy creates/updates stack → ECS pulls image

If cdk destroy deleted the repository and a fresh cdk deploy re-created it, all existing image references would break. The CI pipeline would need to re-push images before the new stack could deploy. With RETAIN:

1. cdk destroy leaves the repository (and its images) intact.
2. cdk deploy creates a new stack with the same repository name.
3. ECR allows creating a repository with the same name as a retained one — CDK auto-imports it.

c) Image chain of custody for RaftDB

RaftDB images follow a stricter chain of custody than the application image:

Build (no AWS credentials) → Export artifact → Verify identity → OIDC auth → Push to ECR

RaftDB images are always referenced by immutable SHA256 digest in the ECS task definition (see 5.4.4 ECS Compute). This is enforced at two levels:

1. The ECR tag mutability exclusion prevents overwriting raftdb-* tags.
2. The parseRaftDbImageDigest() function in the entry point rejects non-digest references.

d) Tag strategy comparison

2. S3 Storage (Imported Buckets)

The storage module imports (does not create) two S3 buckets by name. This is the “import-only” pattern discussed in the design principles.

export function createStorage(scope: Construct, props?: StorageProps): StorageOutput {
  const account = props?.account ?? '';

  const canvasBucket = s3.Bucket.fromBucketName(
    scope, 'ImportedCanvasBucket',
    `awsplace-canvas-${account}`
  );

  const exportsBucket = s3.Bucket.fromBucketName(
    scope, 'ImportedExportsBucket',
    `awsplace-exports-${account}`
  );

  return { canvasBucket, exportsBucket };
}

a) Import-only pattern

b) Bucket purposes

c) Why S3 for canvas instead of DynamoDB?

DynamoDB is excellent for structured metadata (config, bans, milestones, history) but fundamentally unsuitable for binary blobs beyond 400 KB. S3 is the natural choice.

3. RaftDB Application Storage (EFS + S3)

The createRaftDbApplicationStorage module creates three resources for the RaftDB sidecar’s durable storage:

export function createRaftDbApplicationStorage(
  scope: Construct,
  props: { vpc: ec2.Vpc; taskRole: iam.Role },
): RaftDbApplicationStorage {
  const snapshotBucket = new s3.Bucket(scope, 'RaftDbApplicationSnapshots', {
    versioned: true,
    encryption: s3.BucketEncryption.S3_MANAGED,
    blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
    enforceSSL: true,
    minimumTLSVersion: 1.2,
    removalPolicy: RemovalPolicy.DESTROY,
    autoDeleteObjects: true,
    lifecycleRules: [{
      noncurrentVersionExpiration: Duration.days(35),
      abortIncompleteMultipartUploadAfter: Duration.days(1),
    }],
  });

  const fileSystem = new efs.FileSystem(scope, 'RaftDbApplicationFileSystem', {
    vpc: props.vpc,
    vpcSubnets: { subnetType: ec2.SubnetType.PUBLIC },
    encrypted: true,
    removalPolicy: RemovalPolicy.DESTROY,
  });

  const accessPoint = fileSystem.addAccessPoint('ApplicationAccessPoint', {
    path: '/raftdb/production/member-1',
    createAcl: { ownerUid: '10001', ownerGid: '10001', permissions: '0750' },
    posixUser: { uid: '10001', gid: '10001' },
  });

  // Task role needs EFS mount + write permissions scoped to this access point
  props.taskRole.addToPrincipalPolicy(new iam.PolicyStatement({
    actions: ['elasticfilesystem:ClientMount', 'elasticfilesystem:ClientWrite'],
    resources: [fileSystem.fileSystemArn],
    conditions: {
      StringEquals: {
        'elasticfilesystem:AccessPointArn': accessPoint.accessPointArn,
      },
    },
  }));

  return { fileSystem, accessPoint, snapshotBucket };
}

a) Storage resource summary

b) EFS access point enforcement

The access point is the critical security control for RaftDB storage:

EFS Access Point
  ├── path: /raftdb/production/member-1
  ├── POSIX user: 10001:10001 (RaftDB process)
  ├── ACL: ownerUid=10001, ownerGid=10001, permissions=0750
  │
  └── IAM condition: elasticfilesystem:AccessPointArn == this access point ARN

Even if another process in the ECS task tried to access the EFS filesystem, it would be denied — the IAM policy is conditioned on the specific access point ARN, and the access point enforces UID/GID 10001.

c) Snapshot bucket lifecycle

d) Why different removal policies?