Skip to content

CFn: add v2 tests to CI #12556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Apr 29, 2025
Merged

CFn: add v2 tests to CI #12556

merged 5 commits into from
Apr 29, 2025

Conversation

simonrw
Copy link
Contributor

@simonrw simonrw commented Apr 24, 2025 •
edited
Loading

Motivation

We should start testing the new implementation in CI so we have some confidence while reviewing PRs that the implementation is correct.

Changes

  • Add a new CI test stage
  • Copy the WIP change set tests into the new v2 directory for easy test selection
  • Assign change set id to stack to conform to describe_stacks output

@simonrw simonrw added the semver: patch Non-breaking changes which can be included in patch releases label Apr 24, 2025
@simonrw simonrw marked this pull request as draft April 24, 2025 16:37
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 24, 2025 •
edited
Loading

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 47m 14s ⏱️ - 2m 56s
4 384 tests ±0  4 021 ✅  - 1  362 💤 +1  1 ❌ ±0 
4 386 runs  ±0  4 021 ✅  - 1  364 💤 +1  1 ❌ ±0 

For more details on these failures, see this check.

Results for commit 61ac17f. ± Comparison against base commit e8d2029.

This pull request removes 14 and adds 14 tests. Note that renamed tests count towards both. 
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_dynamic]
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_parameter_for_condition_create_resource]
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_unrelated_property]
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_unrelated_property_not_create_only]
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_base_mapping_scenarios[update_string_referencing_resource]
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_conditions
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_direct_update
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_dynamic_update
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_execute_with_ref
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_mappings_with_parameter_lookup
…

tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_dynamic]
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_parameter_for_condition_create_resource]
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_unrelated_property]
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_unrelated_property_not_create_only]
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_base_mapping_scenarios[update_string_referencing_resource]
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_conditions
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_direct_update
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_dynamic_update
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_execute_with_ref
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_mappings_with_parameter_lookup
…
This pull request removes 14 skipped tests and adds 14 skipped tests. Note that renamed tests count towards both. 
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_dynamic]
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_parameter_for_condition_create_resource]
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_unrelated_property]
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_unrelated_property_not_create_only]
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_base_mapping_scenarios[update_string_referencing_resource]
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_conditions
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_direct_update
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_dynamic_update
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_execute_with_ref
tests.aws.services.cloudformation.api.test_changesets.TestCaptureUpdateProcess ‑ test_mappings_with_parameter_lookup
…

tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_dynamic]
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_parameter_for_condition_create_resource]
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_unrelated_property]
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_base_dynamic_parameter_scenarios[change_unrelated_property_not_create_only]
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_base_mapping_scenarios[update_string_referencing_resource]
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_conditions
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_direct_update
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_dynamic_update
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_execute_with_ref
tests.aws.services.cloudformation.v2.test_change_sets.TestCaptureUpdateProcess ‑ test_mappings_with_parameter_lookup
…
This pull request skips 1 test. 
tests.aws.services.cloudformation.v2.test_change_sets ‑ test_single_resource_static_update

♻️ This comment has been updated with latest results.

@simonrw simonrw changed the title Add CFn v2 tests to CI CFn: add v2 tests to CI Apr 24, 2025
@simonrw simonrw marked this pull request as ready for review April 24, 2025 21:55
@simonrw simonrw requested a review from pinzon as a code owner April 24, 2025 21:55
@simonrw simonrw requested review from MEPalma and removed request for dfangl and pinzon April 24, 2025 21:55
alexrashed
alexrashed reviewed Apr 25, 2025
View reviewed changes
Copy link
Member

@alexrashed alexrashed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The introduction to the alternative step is straight forward! 💯 I'll leave the review for the tests to @MEPalma!
/cc to @silv-io and @k-a-il since this adds an additional provider test step to the CircleCI workflow (which needs to be considered in a potential migration to from CircleCI to GitHub actions)

@k-a-il
Copy link
Contributor

k-a-il commented Apr 25, 2025

/cc to @silv-io and @k-a-il since this adds an additional provider test step to the CircleCI workflow (which needs to be considered in a potential migration to from CircleCI to GitHub actions)

@alexrashed , thank you for tagging us. I will migrate this test to GH Actions as well 👍

@alexrashed
Copy link
Member

Nice! @k-a-il Maybe it would make sense to add you and silvio as CODEOWNERS for the GitHub actions and CircleCI files to make sure you don't miss anything ;)

@k-a-il k-a-il mentioned this pull request Apr 25, 2025
Merged
@simonrw simonrw added this to the Playground milestone Apr 25, 2025
MEPalma
MEPalma approved these changes Apr 28, 2025
View reviewed changes
Copy link
Contributor

@MEPalma MEPalma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for setting this up!

@simonrw
Copy link
Contributor Author

simonrw commented Apr 29, 2025

The failing test is a known flake, and unrelated to this PR.

@simonrw simonrw merged commit d2ba498 into master Apr 29, 2025
32 of 34 checks passed
@simonrw simonrw deleted the cfn/v2/add-ci branch April 29, 2025 11:32
@yashuatla yashuatla mentioned this pull request Jun 23, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexrashed alexrashed alexrashed left review comments

@MEPalma MEPalma MEPalma approved these changes

@dominikschubert dominikschubert Awaiting requested review from dominikschubert

Assignees
No one assigned
Labels
semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Milestone
Playground
Development

Successfully merging this pull request may close these issues.
4 participants
@simonrw @k-a-il @alexrashed @MEPalma