Skip to content

Step Functions: Surface Support for Mocked Responses #12525

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 17, 2025

Conversation

MEPalma
Copy link
Contributor

@MEPalma MEPalma commented Apr 15, 2025

Motivation

These changes introduce base support for executing AWS Step Functions state machines in a mock-enabled mode, designed to streamline testing and simulation workflows. The primary motivation is to enable users to execute state machines using mock configuration files, allowing simulated task execution by sampling predefined responses from a mock oracle. This capability is particularly valuable for validating control flow logic and response handling in a deterministic manner without invoking actual downstream services AWS Docs

The changes in this PR surface support for all critical components necessary to enable this workflow. Future changes will refine and expand on each aspect introduced in this PR

Changes

  • Parsing mock configuration files that define the oracle of mocked responses
  • Selecting specific test cases through standard state machine execution APIs by appending #test-case-name to the execution ARN
  • Introducing a mock mode in the environment
  • Sampling the mocked response at runtime based on configuration and execution context
  • Injecting the mocked response in place of real task logic
  • Support for Return and Throw response types
  • Snapshot testing utils to compare mocked executions against real AWS executions

@MEPalma MEPalma added the semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases label Apr 15, 2025
@MEPalma MEPalma added this to the 4.4 milestone Apr 15, 2025
@MEPalma MEPalma self-assigned this Apr 15, 2025
Copy link

LocalStack Community integration with Pro

    2 files      2 suites   1h 51m 54s ⏱️
4 358 tests 4 012 ✅ 346 💤 0 ❌
4 360 runs  4 012 ✅ 348 💤 0 ❌

Results for commit 376a73d.

Copy link
Contributor

@gregfurman gregfurman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Some very small nits/questions. Otherwise, I think this is good to go 👍

Comment on lines +112 to +113
range_part_start = definition_parts[0]
range_part_end = definition_parts[1]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be checking that range_part_start > range_part_end?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We check when trying to construct the mock responses from these values localstack.services.stepfunctions.mocking.mock_config.StateMockedResponses.__init__



def _parse_mocked_response_range(string_definition: str) -> tuple[int, int]:
definition_parts = string_definition.strip().split("-")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to validate these values at all? Like are there valid bounds?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes indeed, we do this after parsing the values and constructing the StateMockedResponses object for the runtime later localstack.services.stepfunctions.mocking.mock_config.StateMockedResponses.__init__

Accepts any fields.
"""

model_config = {"extra": "allow", "frozen": True}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should we be using a pydantic.dataclass here instead? It contains a frozen attribute out-the-box so am wondering about the BaseModel approach

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely a good point you raise! In this case we still need a few BaseModel features, I can see:

  • extra="allow" on RawReturnResponse
  • the root model on RawTestCase
  • the model_validator in RawResponseModel
    Pydantic’s dataclass doesn’t support any of those as far as I can see, and we'd also lose the model_dump() (although this may not be an issue by itself if we were using records)

Comment on lines +92 to +93
@lru_cache(maxsize=1)
def _read_sfn_raw_mock_config(file_path: str, modified_epoch: int) -> Optional[RawMockConfig]: # noqa
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: this will only cache a single call. I imagine we wont be reading in multiple mock files?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct, for now we'll be loading one file only

@MEPalma MEPalma merged commit 052eedf into master Apr 17, 2025
31 checks passed
@MEPalma MEPalma deleted the MEP-SFN-mocked_responses_happy_path_and_preprocessing branch April 17, 2025 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants