Skip to content

[SPARK-52889][PYTHON] Implement the current_time function in PySpark #51584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from

Conversation

uros-db
Copy link
Contributor

@uros-db uros-db commented Jul 20, 2025

What changes were proposed in this pull request?

Implement the current_time function in PySpark & PySpark Connect API.

Why are the changes needed?

Expand API support for the current_time function.

Does this PR introduce any user-facing change?

Yes, the new function is now available in Python API.

How was this patch tested?

Added appropriate Python function tests.

  • pyspark.sql.tests.test_functions
  • pyspark.sql.tests.connect.test_parity_functions

Was this patch authored or co-authored using generative AI tooling?

No.

@uros-db uros-db marked this pull request as draft July 20, 2025 23:03
Copy link
Contributor Author

@uros-db uros-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is waiting for the corresponding Scala functions PR: #51574.

@uros-db uros-db marked this pull request as ready for review July 27, 2025 18:25
@uros-db uros-db changed the title [DRAFT][SPARK-52888][PYTHON] Implement the current_time function in PySpark [WIP][SPARK-52888][PYTHON] Implement the current_time function in PySpark Jul 27, 2025
@uros-db
Copy link
Contributor Author

uros-db commented Jul 27, 2025

Scala function PR is merged: #51574. Continuing the work on PySpark.

@uros-db uros-db changed the title [WIP][SPARK-52888][PYTHON] Implement the current_time function in PySpark [WIP][SPARK-52889][PYTHON] Implement the current_time function in PySpark Jul 27, 2025
@dongjoon-hyun dongjoon-hyun marked this pull request as draft July 27, 2025 21:05
@dongjoon-hyun
Copy link
Member

Thank you, @uros-db . According to the PR title, I converted this to Draft status because GitHub Draft status provides more features like preventing merge mistake. Please make it ready when the PR is ready.

@uros-db uros-db changed the title [WIP][SPARK-52889][PYTHON] Implement the current_time function in PySpark [SPARK-52889][PYTHON] Implement the current_time function in PySpark Jul 27, 2025
@uros-db uros-db marked this pull request as ready for review July 27, 2025 21:38
@uros-db
Copy link
Contributor Author

uros-db commented Jul 27, 2025

@zhengruifeng Please review. cc @MaxGekk

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @uros-db .

@dongjoon-hyun
Copy link
Member

cc @peter-toth

@uros-db uros-db requested a review from zhengruifeng July 28, 2025 07:06
@uros-db
Copy link
Contributor Author

uros-db commented Jul 29, 2025

Conflicts resolved. The CI failure (test_crossvalidator_on_pipeline) doesn't look related to my changes. Ready to merge. @dongjoon-hyun @HyukjinKwon @zhengruifeng

@zhengruifeng
Copy link
Contributor

merged to master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants