26 Pull requests opened by 23 people

• [SPARK-52819][SQL] Making KryoSerializationCodec serializable to fix java.io.NotSerializableException errors in various use cases

  #51615 opened Jul 22, 2025

• [SPARK-51920][SS][PYTHON] Fix composite/nested type in value state for python

  #51621 opened Jul 22, 2025

• [SPARK-52921][SQL] Specify outputPartitioning for UnionExec for same output partitoning as children operators

  #51623 opened Jul 22, 2025

• [SPARK-52146][SQL] Detect cyclic function references in SQL UDFs

  #51626 opened Jul 23, 2025

• [SPARK-52922][PS] Avoid CAST_INVALID_INPUT of "astype" in ANSI mode

  #51627 opened Jul 23, 2025

• [SPARK-52923][CORE] Allow ShuffleManager to control push merge during shuffle registration

  #51629 opened Jul 23, 2025

• [SPARK-52926][SQL] Added SQLMetric for remote schema fetching time duration.

  #51633 opened Jul 23, 2025

• [SPARK-52943][PYTHON] Enable arrow_cast for all pandas UDF eval types

  #51635 opened Jul 23, 2025

• [SPARK-52844][PYTHON][TESTS] Update Python test dependencies

  #51636 opened Jul 23, 2025

• [SPARK-52936][INFRA][TESTS] Benchmark result update automation

  #51643 opened Jul 24, 2025

• [SPARK-52937][SDP] Sinks

  #51644 opened Jul 24, 2025

• [SPARK-52942][YARN][BUILD] YARN External Shuffle Service jar should include `scala-library`

  #51650 opened Jul 24, 2025

• [SPARK-52930][CONNECT] Use DataType.Array/Map for Array/Map Literals

  #51653 opened Jul 24, 2025

• [SPARK-52952][PYTHON] Add PySpark UDF Type Coercion Dev Script

  #51663 opened Jul 25, 2025

• [SPARK-52953][SQL] Incorrect parameter order in some ExpressionEvalHelper.checkResult() method invocations

  #51664 opened Jul 25, 2025

• [SPARK-52954][PYTHON] Arrow UDF support return type coercion

  #51665 opened Jul 25, 2025

• [SPARK-52956][SQL] Preserve alias metadata when collapsing projects

  #51667 opened Jul 25, 2025

• [SPARK-52945][SQL][TESTS] Split `CastSuiteBase#checkInvalidCastFromNumericType` into three methods and guarantee assertions are valid

  #51668 opened Jul 25, 2025

• [SPARK-52931][Core] Restrict declare variable naming

  #51669 opened Jul 25, 2025

• [WIP] Exclude pandas numpy operand in __eq__

  #51670 opened Jul 25, 2025

• [SPARK-52960][SQL] Show subtree string in LogicalQueryStage toString

  #51674 opened Jul 26, 2025

• [SPARK-52967][BUILD] Upgrade ORC to 2.2.0

  #51676 opened Jul 26, 2025

• [SPARK-52944][CORE][SQL][YARN][TESTS][3.5] Fix invalid assertions in tests

  #51677 opened Jul 27, 2025

• [SPARK-52968][SS] Emit additional state store metrics

  #51679 opened Jul 28, 2025

• [SPARK-52853][TESTS][FOLLOW-UP] Import SDP module when connect dependencies are available

  #51680 opened Jul 28, 2025

• [SPARK-52970][CONNECT][INFRA] Upgrade PyArrow version in 3.5 client <> master server build

  #51681 opened Jul 28, 2025

33 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-51415][SQL] Support the time type by make_timestamp()

  #51543 commented on Jul 27, 2025 • 28 new comments

• [SPARK-51554][SQL] Add the time_trunc() function

  #51547 commented on Jul 27, 2025 • 26 new comments

• [SPARK-52772][SQL] Inconsistent table attribute handling during updates

  #51466 commented on Jul 26, 2025 • 11 new comments

• [SPARK-52689][SQL] Send DML Metrics to V2Write

  #51377 commented on Jul 26, 2025 • 10 new comments

• [SPARK-52407][SQL] Add support for Theta Sketch

  #51298 commented on Jul 28, 2025 • 9 new comments

• [SPARK-51400] Replace ArrayContains nodes to InSet

  #50170 commented on Jul 21, 2025 • 4 new comments

• [SPARK-52798] [SQL] Add function approx_top_k_combine

  #51505 commented on Jul 24, 2025 • 3 new comments

• [SPARK-52889][PYTHON] Implement the current_time function in PySpark

  #51584 commented on Jul 28, 2025 • 3 new comments

• [SPARK-52444][SQL][CONNECT] Add support for Variant/Char/Varchar Literal

  #51215 commented on Jul 26, 2025 • 3 new comments

• [SPARK-52646][PS] Avoid CAST_INVALID_INPUT of `__eq__` in ANSI mode

  #51370 commented on Jul 24, 2025 • 3 new comments

• [SPARK-52888][PYTHON] Implement the make_time function in PySpark

  #51582 commented on Jul 28, 2025 • 2 new comments

• [SPARK-52807][SDP] Proto changes to support analysis inside Declarative Pipelines query functions

  #51502 commented on Jul 21, 2025 • 1 new comment

• [SPARK-52449][CONNECT][PYTHON][ML] Make datatypes for Expression.Literal.Map/Array optional

  #51473 commented on Jul 23, 2025 • 1 new comment

• [SPARK-52777][SQL] Enable shuffle cleanup mode configuration in Spark SQL

  #51458 commented on Jul 26, 2025 • 1 new comment

• [SPARK-52617][SQL] Cast TIME to/from TIMESTAMP_NTZ

  #51381 commented on Jul 24, 2025 • 1 new comment

• [SPARK-52593][PS] Avoid CAST_INVALID_INPUT of `Series.dot` and `DataFrame.dot` in ANSI mode

  #51310 commented on Jul 25, 2025 • 1 new comment

• [WIP][SPARK-52764][PYTHON][ML][CONNECT][TESTS] Retry flaky tests in `test_parity_classification`

  #51535 commented on Jul 24, 2025 • 0 new comments

• [SPARK-52621][SQL] Cast TIME to/from VARIANT

  #51553 commented on Jul 22, 2025 • 0 new comments

• [SPARK-52868][SQL] CBO: OOM-risky stats underestimation for some filters and sources

  #51558 commented on Jul 26, 2025 • 0 new comments

• [DRAFT][DO-NOT-REVIEW][SPARK-51XXX][SQL] Enable implicit cast from STRING to TIME type

  #51583 commented on Jul 27, 2025 • 0 new comments

• [SPARK-52890][SPARK-52891][PYTHON] Implement the to_time and try_to_time functions in PySpark

  #51585 commented on Jul 28, 2025 • 0 new comments

• [SPARK-52892][SPARK-52893][SPARK-52894][PYTHON] Support TIME in the hour, minute, and second functions in PySpark

  #51586 commented on Jul 28, 2025 • 0 new comments

• [WIP][SPARK-51169] Set up a daily job for Python 3.14

  #51532 commented on Jul 23, 2025 • 0 new comments

• [SPARK-52767][SQL] Optimize the performance of maxRows for join and union

  #51451 commented on Jul 21, 2025 • 0 new comments

• approx_top_k_combine

  #51393 commented on Jul 24, 2025 • 0 new comments

• [SPARK-52580][PS] Avoid CAST_INVALID_INPUT of `replace` in ANSI mode

  #51297 commented on Jul 24, 2025 • 0 new comments

• [SPARK-52544][SQL] Allow configuring Json datasource string length limit through SQLConf

  #51235 commented on Jul 25, 2025 • 0 new comments

• [SPARK-52858][INFRA] Retry SBT compilation when OOM

  #51149 commented on Jul 21, 2025 • 0 new comments

• Increase report interval of spaming logs to 10 seconds

  #51012 commented on Jul 25, 2025 • 0 new comments

• [SPARK-51554][SQL] Add the time_trunc() function for TIME datatype

  #50607 commented on Jul 22, 2025 • 0 new comments

• [SPARK-51585][SQL] Oracle dialect supports pushdown datetime functions

  #50353 commented on Jul 28, 2025 • 0 new comments

• [SPARK-51756][CORE] Computes RowBasedChecksum in ShuffleWriters

  #50230 commented on Jul 23, 2025 • 0 new comments

• [SPARK-51359][CORE][SQL] Set INT64 as the default timestamp type for Parquet files

  #50215 commented on Jul 24, 2025 • 0 new comments