CFNv2: minor parity assessment and triaging #12871

simonrw · 2025-07-17T14:05:06Z

Motivation

While triaging the outstanding work for the CFn v2 provider project, I have made some minor changes (typically minor fixes) to gain us more parity with the v1 provider.

Changes

Recursively fetch nested property values (mostly for nested stacks)
Validate the stack name is provided in describe_stack_events
Don't assume a previous update model is present, for the case of calling update_stack after create_stack (or another update_stack)
Handle stacks with no changes
Correctly model stack change based on additional template properties
Update tests that have been updated in the v1 directory
Unskip 22 tests and rescope 13 tests
Explicitly skip the v2 directory in the main integration test step
- the markers in the individual files are clearly not enough to correctly skip the tests, and running these tests just slows down the main pipeline run

localstack-bot · 2025-07-17T14:05:18Z

Currently, only patch changes are allowed on main. Your PR labels (semver: minor) indicate that it cannot be merged into the main at this time.

github-actions · 2025-07-17T14:15:39Z

Test Results - Preflight, Unit

21 983 tests ±0 20 249 ✅ ±0 6m 33s ⏱️ +17s
1 suites ±0 1 734 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 2c6ffd1. ± Comparison against base commit 1775743.

♻️ This comment has been updated with latest results.

github-actions · 2025-07-17T14:23:42Z

LocalStack Community integration with Pro

2 files ±0 2 suites ±0 1h 45m 21s ⏱️ +14s
4 938 tests - 1 4 167 ✅ +5 771 💤 - 6 0 ❌ ±0
4 940 runs - 1 4 167 ✅ +5 773 💤 - 6 0 ❌ ±0

Results for commit 2c6ffd1. ± Comparison against base commit 1775743.

This pull request removes 1 test.

tests.aws.services.cloudformation.v2.ported_from_v1.api.test_stacks.TestStacksApi ‑ test_list_stack_resources_for_removed_resource

♻️ This comment has been updated with latest results.

github-actions · 2025-07-17T14:24:43Z

Test Results (amd64) - Acceptance

7 tests ±0 5 ✅ ±0 3m 25s ⏱️ +12s
1 suites ±0 2 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 2c6ffd1. ± Comparison against base commit 1775743.

♻️ This comment has been updated with latest results.

github-actions · 2025-07-17T14:29:28Z

Test Results (amd64) - Integration, Bootstrap

5 files 5 suites 2h 19m 45s ⏱️
4 839 tests 4 359 ✅ 480 💤 0 ❌
4 845 runs 4 359 ✅ 486 💤 0 ❌

Results for commit 2c6ffd1.

♻️ This comment has been updated with latest results.

Should we keep the behaviour the test is testing?

pinzon

Great changes! 🎉

pinzon · 2025-07-24T16:01:29Z

localstack-core/localstack/services/cloudformation/v2/provider.py

+class StackNotFoundError(ValidationError):
+    def __init__(self, stack_name_or_id: str):
+        if is_stack_arn(stack_name_or_id):
+            super().__init__(f"Stack with id {stack_name_or_id} does not exist")
+        else:
+            super().__init__(f"Stack [{stack_name_or_id}] does not exist")


comment: Thanks for the refactoring

pinzon · 2025-07-24T16:02:18Z

localstack-core/localstack/services/cloudformation/v2/provider.py

@@ -242,18 +242,14 @@ def create_change_set(
                    request_payload=request,
                    template=structured_template,
                    template_body=template_body,
+                    initial_status=StackStatus.REVIEW_IN_PROGRESS,


comment: Glad that we now how the correct initial status

pinzon · 2025-07-24T16:07:51Z

tests/aws/cdk_templates/SecretGeneration/SecretGenerationbc794fce.json

question: is this file intentional? I didn't see any unskipped test that uses CDK

pinzon · 2025-07-24T16:08:32Z

localstack-core/localstack/services/kinesis/resource_providers/aws_kinesis_streamconsumer.py

@@ -66,7 +66,7 @@ def create(
            response = kinesis.register_stream_consumer(
                StreamARN=model["StreamARN"], ConsumerName=model["ConsumerName"]
            )
-            model["ConsumerARN"] = response["Consumer"]["ConsumerARN"]
+            model["Id"] = model["ConsumerARN"] = response["Consumer"]["ConsumerARN"]


Comment: nice small fix 👍

github-actions · 2025-07-25T12:32:54Z

Test Results (MA/MR) - Preflight, Unit

21 983 tests 20 249 ✅ 6m 25s ⏱️
1 suites 1 734 💤
1 files 0 ❌

Results for commit 2c6ffd1.

github-actions · 2025-07-25T12:40:41Z

Test Results (amd64, MA/MR) - Acceptance

7 tests 5 ✅ 3m 6s ⏱️
1 suites 2 💤
1 files 0 ❌

Results for commit 2c6ffd1.

k-a-il · 2025-07-25T12:51:57Z

.github/workflows/aws-tests.yml

@@ -394,7 +394,7 @@ jobs:
        env:
          # add the GitHub API token to avoid rate limit issues
          GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          PYTEST_ARGS: "${{ env.TINYBIRD_PYTEST_ARGS }}${{ env.TESTSELECTION_PYTEST_ARGS }} --splits 4 --group ${{ matrix.group }} --store-durations --clean-durations --ignore=tests/unit/ --ignore=tests/bootstrap"
+          PYTEST_ARGS: "${{ env.TINYBIRD_PYTEST_ARGS }}${{ env.TESTSELECTION_PYTEST_ARGS }} --splits 4 --group ${{ matrix.group }} --store-durations --clean-durations --ignore=tests/unit/ --ignore=tests/bootstrap --ignore=tests/aws/services/cloudformation/v2"


Thank you for adding the V2 provider-specific tests to the ignored list 👍

github-actions · 2025-07-25T13:12:21Z

Test Results (amd64, MA/MR) - Integration, Bootstrap

5 files 5 suites 2h 20m 5s ⏱️
4 839 tests 4 359 ✅ 480 💤 0 ❌
4 845 runs 4 359 ✅ 486 💤 0 ❌

Results for commit 2c6ffd1.

simonrw added the semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases label Jul 17, 2025

simonrw added 21 commits July 23, 2025 23:18

Implement nested structure lookup

602f8db

Fix update after create/update behaviour

0336b58

Unskip cdk bootstrap test

301777f

Unskip events test

8f67cb8

Use custom message for describing non-existent change sets

6e33ffb

update more skips

10f2501

Add comment for test

d14d661

Should we keep the behaviour the test is testing?

Remove comment

b0fd3ad

Fix AWS::Kinesis::StreamConsumer physical resource id

e1cacdd

Unskip 3 tests

5c916b6

Stacks can take initial status

2a0b120

Update logic for validating stack states

eb9404d

Classify more tests

ad9fb2e

Handle creating empty change set

9ddf740

push further with skips and recategorisation

22c4bb6

Only show non-deleted resources in status endpoint

6ddac81

Update tests

ccd39d4

Update opensearch test

3e36634

Make events instance scoped

ac74ed7

Implement update_termination_protection

611c5fc

Correctly model change type

4959841

simonrw force-pushed the cfn/v2/parity-assessment branch from 99634d3 to 4959841 Compare July 23, 2025 22:20

Unite stack not found errors

20ba17a

simonrw marked this pull request as ready for review July 23, 2025 23:16

simonrw requested review from dominikschubert and pinzon as code owners July 23, 2025 23:16

simonrw modified the milestones: Playground, 4.8 Jul 24, 2025

Merge branch 'main' into cfn/v2/parity-assessment

669b731

pinzon approved these changes Jul 24, 2025

View reviewed changes

simonrw added 5 commits July 24, 2025 21:05

Fix lint errors

79f623d

Remove incorrectly tracked file

e34284e

Merge branch 'main' into cfn/v2/parity-assessment

ffb49f2

Fix linting error

53f5d31

Merge remote-tracking branch 'origin/main' into cfn/v2/parity-assessment

4cdd95d

simonrw force-pushed the cfn/v2/parity-assessment branch from 09ae1f3 to 4cdd95d Compare July 25, 2025 10:22

simonrw added 2 commits July 25, 2025 11:59

No need to process before state

69a5cf5

Don't run cfnv2 tests under normal integration test run

2c6ffd1

simonrw requested review from alexrashed, dfangl, silv-io and k-a-il as code owners July 25, 2025 12:22

k-a-il reviewed Jul 25, 2025

View reviewed changes

simonrw merged commit 34049aa into main Jul 25, 2025
61 checks passed

simonrw deleted the cfn/v2/parity-assessment branch July 25, 2025 13:26

This was referenced Jul 28, 2025

[Testing] Update test durations #12915

Merged

CFNv2: implement describe-stack-resource #12912

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

CFNv2: minor parity assessment and triaging #12871

CFNv2: minor parity assessment and triaging #12871

Uh oh!

simonrw commented Jul 17, 2025 •

edited

Loading

Uh oh!

localstack-bot commented Jul 17, 2025

Uh oh!

github-actions bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

pinzon left a comment

Uh oh!

pinzon Jul 24, 2025

Uh oh!

pinzon Jul 24, 2025

Uh oh!

pinzon Jul 24, 2025

Uh oh!

pinzon Jul 24, 2025

Uh oh!

github-actions bot commented Jul 25, 2025

Uh oh!

github-actions bot commented Jul 25, 2025

Uh oh!

k-a-il Jul 25, 2025

Uh oh!

github-actions bot commented Jul 25, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CFNv2: minor parity assessment and triaging #12871

CFNv2: minor parity assessment and triaging #12871

Uh oh!

Conversation

simonrw commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Uh oh!

localstack-bot commented Jul 17, 2025

Uh oh!

github-actions bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results - Preflight, Unit

Uh oh!

github-actions bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LocalStack Community integration with Pro

Uh oh!

github-actions bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (amd64) - Acceptance

Uh oh!

github-actions bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Results (amd64) - Integration, Bootstrap

Uh oh!

pinzon left a comment

Choose a reason for hiding this comment

Uh oh!

pinzon Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

pinzon Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

pinzon Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

pinzon Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 25, 2025

Test Results (MA/MR) - Preflight, Unit

Uh oh!

github-actions bot commented Jul 25, 2025

Test Results (amd64, MA/MR) - Acceptance

Uh oh!

k-a-il Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jul 25, 2025

Test Results (amd64, MA/MR) - Integration, Bootstrap

Uh oh!

Uh oh!

Uh oh!

simonrw commented Jul 17, 2025 •

edited

Loading

github-actions bot commented Jul 17, 2025 •

edited

Loading

github-actions bot commented Jul 17, 2025 •

edited

Loading

github-actions bot commented Jul 17, 2025 •

edited

Loading

github-actions bot commented Jul 17, 2025 •

edited

Loading