Comet Falls Back to Spark on Every Delta Table. I Wrote the PR That Fixes It

Mon, 17 Aug 2026 07:00:00 +0700

My last post ended with a scoreboard: EMR and Glue page-blind, stock Iceberg page-blind, and two working exits, stock Spark 4.2 for parquet and Delta, Comet 1.0.0 for parquet and Iceberg. There’s a hole in that scoreboard. Comet, the fastest reader in the entire investigation, covers parquet and Iceberg. The format I actually run, Delta, isn’t on its list.

The reason is one line of code. Comet’s isFileFormatSupported check requires exactly ParquetFileFormat, and Delta’s DeltaParquetFileFormat is a subclass. So Comet declines every Delta table and hands the query back to Spark. No native execution, no row-group pruning, no page skipping, on what is probably the most widely deployed table format in the Spark ecosystem. And a Delta table is just parquet files with a transaction log on top. The 3.4x reader was sitting right there.

Rust on Dustin Smith's Online Resume

Comet Falls Back to Spark on Every Delta Table. I Wrote the PR That Fixes It