Open Source

key each scan's planning data by its plan node in a native block

check Iceberg DPP pruning by planned file tasks

reject a file without field ids at any depth whether or not id matching is on

report task input metrics after the native iterator closes and add to Spark's counters

match Spark's duplicate field and field id semantics in parquet field lookup

let decimal SUM recover from an intermediate overflow like Spark

gate the regr_r2 degenerate-case swap on the Spark patch release

guard page skipping in the native Iceberg scan

run length on binary input natively

drop the redundant width_bucket shim registrations and guard serde uniqueness

cache parsed plan data across a stage's tasks

reuse zstd compression contexts across shuffle blocks

decline structs with duplicate field names before they reach Java Arrow

compile user regex patterns once per planned expression

label pull requests by changed paths and title prefix

expand object store option references, uniquify constant metadata names, drop dead parquet JNI

reuse per-partition scratch in the shuffle write path

skip wasted per-batch work in TopK insert_batch

SPARK-54593: enable DPP for materialized filtering sides

revisions for spark sql performance tuning to improve readability and grammar

updated some grammar and a missed period in the tuning doc

expose parse on the package for config trees

release 0.2.1

Python 3.10 support

Scala-style sealed trait support via dataclasses

type hints use Text instead of str