S3 Bucket Keys

Ref: https://learn.cantrill.io/courses/1820301/lectures/41301481

Scaling Problems with S3 SSE-KMS

In normal SSE-KMS architecture, each individual object requires generating a DEK (with a kms:GenerateDataKey action) when being uploaded to S3 → one whole API call to KMS for each object
- 👎 Requests to KMS incur costs… what happens when uploading 30k objects per sec?
Besides, kms:GenerateDataKey operation has throttling limits
- 👎 Scaling limit for PUTs per second per KMS key
- Can only generate a finite number of DEKs per second across regions

🔧 KMS key generates a temporary bucket key that generates DEKs
- Helps S3 scale & reduce costs when using SSE-KMS
👍 Greatly reduces API calls to KMS and costs
- One API call to generate bucket key, then bucket key generates a lot of DEKs without needing to call KMS
- ‼️ AWS CloudTrail will not log operations per-object any more, only per-bucket!!
  - API call to KMS will only show bucket ARN, not object ARNs
  - CloudTrail will register much fewer logs and events
❗ NOT retroactive!
- Won't encrypt existing objects in the bucket when enabled
- Objects in bucket encrypted before (without bucket keys) won't benefit from cost savings
👍 Works with S3 Replication (both SRR and CRR)
- Object encryption configuration is maintained in target bucket
- 💡 NOTE: before, when storing in S3 in plaintext was allowed, there was a quirk with Entity Tags (ETAGs) in replicated S3 objects:
  - If a plaintext object was replicated to a different bucket when using bucket keys, it was the bucket keys in the destination bucket (and NOT the source bucket) that were used → ETAG changed during replication
  - This is an extreme nuance that will rarely come up these days, since plaintext storing is no longer allowed and SSE enforced, but just in case it appears in an exam question…
Doc Ref: https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html