Pay Once, Not Per Task: Four Repeated Costs Comet 1.1.0 Stopped Paying
Apache DataFusion Comet 1.1.0 is out, with 14 of my PRs in it. Four of them fix the same kind of waste: work that gives the same answer every time but was being redone for every task, batch or shuffle block. Here is what each one changed, what the benchmarks really show, and the correctness fix I'd point to first.
Read more