Qiegang Long
Work notes on Lakehouse.
Variant Benchmark: Open-Sourced, Two Spark Optimizer Fixes, and a ~4x TPC-DS Speedup
We open-sourced variant-conformance-benchmark, used it to surface two Spark optimizer problems, and measured a ~4x TPC-DS speedup after the fixes — Variant finally beats string JSON on analytical queries.
Preliminary Notes on Open-Source Variant Performance
Early benchmark results comparing JSON-as-string vs Variant on GitHub Archive (GHA) data with Spark 4.