fix(firehose): reject direct puts to Kinesis-source streams

Merged
CA
3d ago
+478-15

Summary

Closes #2710.

  • Reject PutRecord and PutRecordBatch on KinesisStreamAsSource delivery streams with AWS's HTTP 400 InvalidArgumentException and exact message: This operation is not permitted on KinesisStreamAsSource delivery stream type.
  • Put the guard in shared account/region-scoped admission, before destination/backend checks or S3 writes. Internal Firehose delivery hooks use the same guard.
  • Route empty/all-invalid batches through shared admission so they cannot bypass the source-type restriction.
  • Add regressions for both public APIs, internal hooks, absent backends/destinations, scoped lookup, no in-memory/durable S3 side effects, and successful DirectPut controls.

This is a standalone defect fix against upstream main, with no dependency on the Firehose S3 buffering/compression series. It does not add Kinesis-source polling.

Test plan

  • cargo test -p fakecloud-firehose — 36 unit tests passed; six new regressions failed before the fix.
  • cargo build --bin fakecloud followed by cargo test -p fakecloud-e2e --test firehose — 9 official-SDK E2Es passed.
  • cargo test -p fakecloud-conformance --test firehose — 6 passed.
  • cargo test -p fakecloud-conformance --lib — 94 passed.
  • cargo clippy --workspace --all-targets -- -D warnings.
  • cargo fmt --all --check and git diff --check.
  • Upstream CI-equivalent workspace test gate passed, with the single local WSL exclusion below.
  • Independent code review completed with no findings.

Local verification limitations

cargo test --workspace \
  --exclude fakecloud-e2e --exclude fakecloud-conformance \
  --exclude fakecloud-tfacc --exclude fakecloud-parity \
  -- --skip data_plane::tests::vpc_link_proxy_resolves_target_nlb_and_attempts_connection

The excluded API Gateway test fails identically on untouched upstream 041ddc563 on this WSL host: unused localhost ports time out (504) rather than immediately refusing the connection (expected 502). This is a command-line exclusion for local verification only; no tests were disabled in the PR.

The unrestricted workspace run was attempted too. It encountered Podman prerequisite failures, an unrelated RDS readiness timeout, and the two-hour tool timeout while running Terraform acceptance tests. These are outside the normal CI workspace test job. The affected Firehose suites and the scoped conformance tests passed.

Summary by cubic

Closes #2710. Delivery streams created with KinesisStreamAsSource now reject PutRecord and PutRecordBatch with AWS's HTTP 400 InvalidArgumentException and the exact message, instead of delivering the records to the destination.

  • The guard runs in shared account- and region-scoped admission, before destination and backend checks, so it also covers internal Firehose delivery hooks and any S3 writes.
  • Empty and all-invalid batches now go through shared admission and can no longer bypass the source-type restriction.
  • DirectPut streams keep their existing success and partial-failure behavior; no Kinesis-source polling is added.
  • Adds regressions for both public APIs, internal hooks, absent backends and destinations, scoped lookup, no S3 side effects, and DirectPut controls; all Firehose unit, E2E, and conformance suites pass.