Your Parquet Column Indexes Are Being Ignored on EMR and Glue

Mon, 10 Aug 2026 01:30:00 +0700

I was benchmarking data-clustering layouts on Delta tables. Locally, everything was skipping beautifully, box queries touched about 10% of rows. Then I ran the identical benchmark on EMR 7.13 and those queries scanned 93% of the table. Same code, same data shape, same queries.

Either my layouts were broken on EMR, or something in the read path was. This is the investigation that followed. It ends with a four-environment A/B test proving that the AWS Spark runtime’s vectorized parquet reader, on both EMR 7.13 and Glue 5.0, performs row-group pruning but never applies parquet column indexes. Stock Apache Spark 3.5.6, the exact version EMR ships, applies them fully against the same files.

Emr on Dustin Smith's Online Resume

Your Parquet Column Indexes Are Being Ignored on EMR and Glue