fix(s3): one write path for side effects; replication, encryption and cross-service reads match AWS

Merged
VI
5d ago
+6059-1156

Summary

Follow-ups to #2725: every new object gets the same S3 side effects, and replication / encryption behave as on AWS.

One write path. PutObject, POST, CopyObject, CompleteMultipartUpload and objects other services write (put_delivered_object: Firehose, Logs, ELB logs, Athena, DynamoDB export, ElastiCache, ACM PCA, SES, RDS bridge, access logs, inventory, CodeBuild) all go through side_effects::commit_new_object:

  • bucket default encryption;
  • replication;
  • s3:ObjectCreated:* notifications, sent after the S3 lock is released.

The delivery bus, KMS hook and IAM mode live on the shared S3 state, and a server reset keeps them.

Replication (S3 User Guide: replication-requirements, replication-what-is-isnot-replicated; API Reference: ReplicationRule):

  • Multipart uploads replicate like PutObject (version id, multipart ETag, parts, checksum, lock).
  • Null versions are never replicated.
    • Suspending versioning on a replication source returns InvalidBucketState.
    • A destination without versioning marks the source FAILED.
  • Rules:
    • One replica per destination: the matching rule with the highest priority is chosen first, then its SSE-KMS opt-in and delete-marker settings apply.
    • Duplicate priorities are rejected.
    • A tag filter combined with delete-marker replication is rejected.
    • SseKmsEncryptedObjects requires ReplicaKmsKeyID.
    • V1 Prefix is kept, and so are the other rule elements the old normalizer dropped.
    • Tag and And filters, destination StorageClass, and SSE-KMS replica re-encryption are implemented.
  • Not replicated: replicas themselves, and Glacier Flexible Retrieval / Deep Archive objects.
  • DeleteObject/DeleteObjects replicate delete markers.
  • A replica failure marks the source FAILED instead of failing a write that already succeeded.

Encryption:

  • Multipart uploads get the bucket default.
  • SSE-KMS multipart bodies are stored encrypted like PutObject, and the SSE headers are returned on Create/UploadPart/UploadPartCopy/Complete.
  • UploadPartCopy decrypts KMS sources.
  • aws:kms with no key uses and reports the account's aws/s3 key.
  • PutBucketEncryption accepts an aws:kms default with no KMSMasterKeyID, as AWS does.
  • DynamoDB export honors S3SseAlgorithm/S3SseKmsKeyId.

Cross-service reads. DynamoDB ImportTable, Athena, CloudFormation templates, Route 53 Resolver domain lists, the RDS S3 bridge and the delivery bus read through one decrypting reader (body_io.rs). A KMS refusal fails the operation with an accurate error; it is not reported as a missing object.

Startup. Interrupted Logs exports resume after S3 is wired, so they get the same side effects.

Docs. website/content/docs/services/s3.md and dynamodb.md are updated. No SDK or count changes.

Test plan

  • cargo fmt --check, cargo +stable clippy --workspace --all-targets -D warnings
  • lib: s3 (652), dynamodb (649), cloudformation (632), athena (48), route53resolver (24), persistence, server bin (161)
  • e2e: s3, s3_sse_kms (incl. MPU + default/explicit SSE-KMS, disk mode, keyless default), s3_persistence, s3_write_side_effects (new: MPU replication, delivery -> notification + replication, default encryption), dynamodb (KMS export -> ImportTable round trip), multi_account, firehose (DSSE default), logs, athena, cloudformation_execute_change_set (template from an SSE-KMS bucket), serverlessrepo
  • s3 conformance probe 4068/4068

Summary by cubic

Unifies S3 write side effects so every new object gets the same treatment, and aligns replication, encryption, and cross-service reads with AWS behavior, without SDK or conformance-count changes.

One write path

  • PutObject, POST Object, CopyObject, CompleteMultipartUpload, and objects written by other services (Firehose, Logs exports, Athena, ELB/access logs, inventory, CodeBuild, SES, ACM PCA, RDS bridge, DynamoDB/ElastiCache exports) all commit through side_effects::commit_new_object.
  • Each new object gets the bucket default encryption, is replicated, and fires s3:ObjectCreated:* notifications after the S3 lock is released.
  • The delivery bus, KMS hook, and IAM mode live on shared S3 state and survive a full reset; interrupted Logs exports resume after the S3 service installs them.

Replication, encryption, and cross-service reads

  • Multipart uploads replicate like PutObject, take the bucket default encryption, and store SSE-KMS bodies encrypted; UploadPartCopy decrypts KMS sources.
  • Replication matches AWS: highest-priority rule per destination, V1 prefixes and V2 tag/And filters, destination storage class, SSE-KMS opt-in with replica re-encryption, and delete-marker replication; replicas, null versions, and Glacier objects are not replicated, and a replica failure marks the source FAILED instead of failing the write.
  • A destination without versioning marks the source FAILED, suspending versioning on a replication source returns InvalidBucketState, and duplicate rule priorities or tag+delete-marker rules are rejected.
  • Cross-service reads (DynamoDB import/export, Athena, CloudFormation, Route 53 Resolver, RDS bridge, delivery bus) go through one decrypting reader; a KMS refusal fails the operation with an accurate error instead of reporting a missing object.
  • PutBucketEncryption accepts aws:kms with no KMSMasterKeyID; the account's aws/s3 key is used and reported.