-
-
Notifications
You must be signed in to change notification settings - Fork 4.2k
Secret Manager: Solve the issue for rotate secret after sub-sequent r… #12391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Secret Manager: Solve the issue for rotate secret after sub-sequent r… #12391
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Welcome to LocalStack! Thanks for raising your first Pull Request and landing in your contributions. Our team will reach out with any reviews or feedbacks that we have shortly. We recommend joining our Slack Community and share your PR on the #community channel to share your contributions with us. Please make sure you are following our contributing guidelines and our Code of Conduct.
All contributors have signed the CLA ✍️ ✅ |
I have read the CLA Document and I hereby sign the CLA |
"tests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_rotate_secret_with_lambda_success[False]": { | ||
"recorded-date": "15-03-2025, 08:34:38", | ||
"recorded-content": { | ||
"rotate_secret_immediately_1": { | ||
"ARN": "arn:<partition>:secretsmanager:<region>:111111111111:secret:<SecretId-0idx><ArnPart-0idx>", | ||
"Name": "<SecretId-0idx>", | ||
"VersionId": "<version_uuid:2>", | ||
"ResponseMetadata": { | ||
"HTTPHeaders": {}, | ||
"HTTPStatusCode": 200 | ||
} | ||
}, | ||
"describe_secret_rotated_1": { | ||
"ARN": "arn:<partition>:secretsmanager:<region>:111111111111:secret:<SecretId-0idx><ArnPart-0idx>", | ||
"CreatedDate": "datetime", | ||
"Description": "testing rotation of secrets", | ||
"LastChangedDate": "datetime", | ||
"Name": "<SecretId-0idx>", | ||
"NextRotationDate": "datetime", | ||
"RotationEnabled": true, | ||
"RotationLambdaARN": "<lambda-arn:1>", | ||
"RotationRules": { | ||
"AutomaticallyAfterDays": 1 | ||
}, | ||
"VersionIdsToStages": { | ||
"<version_uuid:1>": [ | ||
"AWSCURRENT" | ||
] | ||
}, | ||
"ResponseMetadata": { | ||
"HTTPHeaders": {}, | ||
"HTTPStatusCode": 200 | ||
} | ||
}, | ||
"list_secret_versions_rotated_1_1": { | ||
"ARN": "arn:<partition>:secretsmanager:<region>:111111111111:secret:<SecretId-0idx><ArnPart-0idx>", | ||
"Name": "<SecretId-0idx>", | ||
"Versions": [ | ||
{ | ||
"CreatedDate": "datetime", | ||
"KmsKeyIds": [ | ||
"DefaultEncryptionKey" | ||
], | ||
"VersionId": "<version_uuid:1>", | ||
"VersionStages": [ | ||
"AWSCURRENT" | ||
] | ||
} | ||
], | ||
"ResponseMetadata": { | ||
"HTTPHeaders": {}, | ||
"HTTPStatusCode": 200 | ||
} | ||
} | ||
} | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I don't think this is a valid test snapshot. I can't see where this function was run with the parameter False
in the code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you are right, I think I missed this while I was doing some refactoring and forgot to change it back again. It should not be False, it should be None
indeed, I can update this
"tests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_rotate_secret_with_lambda_success": { | ||
"last_validated_date": "2024-03-15T08:12:22+00:00" | ||
}, | ||
"tests/aws/services/secretsmanager/test_secretsmanager.py::TestSecretsManager::test_rotate_secret_with_lambda_success[False]": { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As per my previous comment, can we remove this validation if it is not a currently tracked test combination?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as I mentioned before, you are right. This should be None
not False. The test for False
is already existed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR @mabuaisha 🚀
The PR looks good to me overall - I just have some minor concerns/suggestion.
VersionStage="AWSPENDING", | ||
RemoveFromVersionId=token, | ||
) | ||
logger.info( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: we can use logger.debug
instead of logger.info
here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reason why I choose the info
is because in the same function for AWSCURRENT
and others log levels we are also using info
so I thought this was added in purpose, so I stick to use info in this case
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is only in the lambda function body though right @macnev2013?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that’s right @simonrw. This can be ignored.
@@ -782,7 +788,7 @@ def backend_rotate_secret( | |||
pass | |||
|
|||
secret.rotation_lambda_arn = rotation_lambda_arn | |||
secret.auto_rotate_after_days = rotation_rules.get(rotation_days, 0) | |||
secret.auto_rotate_after_days = rotation_period |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
better to add default value in case rotation_period
is not set
secret.auto_rotate_after_days = rotation_period | |
secret.auto_rotate_after_days = rotation_period or 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added this in purpose without or 0
because usually the default value for https://github.com/localstack/moto/blob/localstack/moto/secretsmanager/models.py#L118 based on this logic usually is 0 in case its not set.
For extra check, we can still also fallback to 0
secret.auto_rotate_after_days = rotation_period or 0
I will update this
sm_snapshot.add_transformer( | ||
sm_snapshot.transform.key_value("RotationLambdaARN", "lambda-arn") | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: we could move this to localstack.testing.snapshots.transformer_utility.TransformerUtility.secretsmanager_api
since it's a geneic transformer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Definitely, I can move it there to make it a generic transformer since RotationLambdaARN
could be user later on with multiple tests. define it there could be useful later on
else: | ||
# Resolve auto_rotate_after_days and fallback to previous value | ||
# if its missing from the current request | ||
rotation_period = secret.auto_rotate_after_days |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this could cause 'rotation_period' variable to be referenced before assignment here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah thats right, I will init rotation_period = 0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it always 0
or can we move the rotation_period = secret.auto_rotate_after_days
above the if
block and have that as the default, then remove the else
branch?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can remove the else
branch and make it the default one. However, I will fallback to 0
like this
rotation_period = secret.auto_rotate_after_days or 0
That way even if previous value for for auto_rotate_after_days
that is part of secret is not integer, we are make sure always if the value is not set the default one is 0
raise InvalidRequestException( | ||
"An error occurred (InvalidRequestException) when calling the RotateSecret operation: You tried to \ | ||
perform the operation on a secret that's currently marked deleted." | ||
) | ||
# Resolve rotation_lambda_arn and fallback to previous value if its missing | ||
# from the current request | ||
rotation_lambda_arn = rotation_lambda_arn or secret.rotation_lambda_arn |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if the secret is being rotated first time and we haven't specified the rotation_lambda_arn
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, this was an existing behavior even before I introduce my fix. In case for the first time rotation we did not specify rotation_lambda_arn
then it will raise error, check this code here
try:
lm_client = connect_to(region_name=self.region_name).lambda_
lm_client.get_function(FunctionName=rotation_lambda_arn)
except Exception:
raise ResourceNotFoundException("Lambda does not exist or could not be accessed")
This will raise error whenever we invoked it from the cli
An error occurred (ResourceNotFoundException) when calling the RotateSecret operation: Lambda does not exist or could not be accessed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What error message does AWS return in this case? Does it match the error we raise in this circumstance? I suspect not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you are right. I will update the code because I need to add a check earlier to raise an error that match AWS one
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The error mainly look like this from AWS
"Error": {
"Code": "InvalidRequestException",
"Message": "No Lambda rotation function ARN is associated with this secret."
},
assert response["VersionIdsToStages"][initial_secret_version] == ["AWSPREVIOUS"] | ||
assert response["VersionIdsToStages"][rot_res["VersionId"]] == ["AWSCURRENT"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggestion: we usually don't use assertion in the helper methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it a pattern you followed when you write tests ? or you are talking in general ?. The reason why I add this method is to avoid duplication for multiple tests and I thought usually this could be re-used only within multiple tests for assertions. However, I'm open to change this if you think this could create confusion or violates any sort of patterns/convention you usually followed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: we normally prefer two things:
- assertions in the test body, not in helper methods
- fixtures over helper methods
I personally find it clearer for the assertion failure to occur in the test body so it's clear what's being tested at first glance in the test code itself, rather than having to go diving into helper methods. It also makes the helper methods more flexible, as the assertions may not be consistent from test to test.
For the second item, we often create fixtures that look like:
def my_fixture(aws_client, snapshot): # other fixtures
def inner(name: str, *args, **kwargs): # variables taken from the test body
# do something with both arguments and fixtures, e.g.:
return aws_client.sts.get_caller_identity()
return inner
then in the test we only need to include the fixture and use the "inner" function, which simplifies the usage of that fixture:
def test_foo(my_fixture):
identity = my_fixture("name")
See how we could call AWS methods without passing in the aws_client
.
These are just preferences and I'm not asking you to refactor the helper methods into fixtures, but perhaps you could consider moving the assertions to the test bodies?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make sense, I think you raised a valid point here. I will do some refactoring to this
def test_rotate_secret_multiple_times_with_lambda_success( | ||
self, | ||
sm_snapshot, | ||
secret_name, | ||
create_secret, | ||
create_lambda_function, | ||
aws_client, | ||
set_rotation_on_second_run, | ||
): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be worth testing - what happens if the function_arn
is not provided for the first time rotation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure thing, I can add this as a new test since it will fail if we did not specify the function_arn
. We do not have a test for this case but the rotation backend can handle that as expected based on my previous comment. I will add a test to catch this
1e21cf0
to
1d2726b
Compare
@macnev2013 Thanks a lot for taking the time to review my PR. I replied to your comments and pushed changes as a result of your comments. Let me know if need to discuss this further |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I hope have added some clarity and context to some of the points from @macnev2013 particularly around the use of helper methods and assertions.
I have an issue with test_first_rotate_secret_with_missing_lambda_arn
: you are skipping verification of the two fields that matter for this test. It is specifically around capturing the error that is presented, but you have skipped the wording.
I think this may cause confusion to our users who get an invalid response. I would like to see this implemented correctly.
raise InvalidRequestException( | ||
"An error occurred (InvalidRequestException) when calling the RotateSecret operation: You tried to \ | ||
perform the operation on a secret that's currently marked deleted." | ||
) | ||
# Resolve rotation_lambda_arn and fallback to previous value if its missing | ||
# from the current request | ||
rotation_lambda_arn = rotation_lambda_arn or secret.rotation_lambda_arn |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What error message does AWS return in this case? Does it match the error we raise in this circumstance? I suspect not.
assert response["VersionIdsToStages"][initial_secret_version] == ["AWSPREVIOUS"] | ||
assert response["VersionIdsToStages"][rot_res["VersionId"]] == ["AWSCURRENT"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: we normally prefer two things:
- assertions in the test body, not in helper methods
- fixtures over helper methods
I personally find it clearer for the assertion failure to occur in the test body so it's clear what's being tested at first glance in the test code itself, rather than having to go diving into helper methods. It also makes the helper methods more flexible, as the assertions may not be consistent from test to test.
For the second item, we often create fixtures that look like:
def my_fixture(aws_client, snapshot): # other fixtures
def inner(name: str, *args, **kwargs): # variables taken from the test body
# do something with both arguments and fixtures, e.g.:
return aws_client.sts.get_caller_identity()
return inner
then in the test we only need to include the fixture and use the "inner" function, which simplifies the usage of that fixture:
def test_foo(my_fixture):
identity = my_fixture("name")
See how we could call AWS methods without passing in the aws_client
.
These are just preferences and I'm not asking you to refactor the helper methods into fixtures, but perhaps you could consider moving the assertions to the test bodies?
assert "RotationEnabled" not in describe_secret | ||
assert "RotationRules" not in describe_secret | ||
assert "RotationLambdaARN" not in describe_secret |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: these assertions are not required giving we are matching the snapshot above
|
||
@markers.snapshot.skip_snapshot_verify(paths=["$..Error", "$..Message"]) | ||
@markers.aws.validated | ||
def test_first_rotate_secret_with_missing_lambda_arn( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test is a start, however it does not execute the rotation itself, therefore testing the error message given by AWS if a lambda function arn is not specified. This will help clarify the error handling here.
Here is a demonstration of the confusion that extracting these helper methods causes. I (incorrectly) believed that you hadn't tried to rotate the lambda function, but when I looked in the snapshot I saw that the _setup_invalid_rotation_secret
method tries to rotate a secret without specifying a function ARN.
I would find it much clearer if the test was more like:
- create secret (using our
create_secret
fixture) - try to rotate secret, wrapped in
pytest.raises(...)
- capturing a snapshot of the error message
- (optional) a describe secret call, however this is covered by other more specific tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now I have been more lean toward refactor this since it created this confusion. Thanks for raising this
"AWSPREVIOUS" | ||
"AWSCURRENT" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
q: do you understand why this changed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My assumption for this. The list is not deterministic as I noticed that sometimes version1 could be the previous version and version 2 is the current version and vice versa. Thats the reason why sometimes version 1 could be previous and sometimes it could be current. The same thing for version 2 as well
|
||
@markers.snapshot.skip_snapshot_verify(paths=["$..Error", "$..Message"]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
issue: you have skipped the key part of this test => the error message. Can you remove this verify and implement the correct behaviour?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure thing, I will remove the check it. The only thing that need to be updated is to add a check that raise an error InvalidRequestException
that match AWS one
VersionStage="AWSPENDING", | ||
RemoveFromVersionId=token, | ||
) | ||
logger.info( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is only in the lambda function body though right @macnev2013?
b8e3ea6
to
b14f7f2
Compare
…equests without providing rotation configuration
5baccec
to
9aaf9c2
Compare
… missing function arn for the first time rotation
9aaf9c2
to
8b3a1a3
Compare
Motivation
This PR is added in order to address the issue raised by our community 12144. The issue happens when making subsequent rotate-secret calls without specifying RotationLambdaARN and RotationRules cause ResourceNotFoundException
Changes
Testing
In order to test this you can run localstack locally and run the following command
Create role for lambda function
Attach policy to role
Create lambda function
Create secret
First rotation
Second rotation