Databricks Never Skips a Page. Its Writer Does Not Even Write Them

Wed, 19 Aug 2026 00:19:00 +0700

Two posts ago I found that EMR and Glue never read parquet column indexes. Last post I gave DataFusion Comet a native Delta read path so the fastest reader in that investigation could prune the format I actually run. One vendor was missing from the scoreboard: Databricks. So I created a workspace, pointed the same instrumentation at it, and reran the series benchmark with the platform’s own clustering in the ring: classic OPTIMIZE ZORDER BY and liquid clustering, against the learned Hilbert curve from the earlier posts.

Photon on Dustin Smith's Online Resume

Databricks Never Skips a Page. Its Writer Does Not Even Write Them