After twenty years of building enterprise data warehouses, I have learned that data is never evenly distributed. Customer tables are dominated by one country. Transaction tables are dominated by one product line. Event tables are dominated by one event type.
This is skew. And skew is what separates a data layout strategy that works in theory from one that works in production.
Z-ordering and liquid clustering both organize data to support skipping. Their benefit depends on keys, skew, file layout, maintenance and predicates; this article does not establish that either method universally fails or succeeds on skewed data.
This article explains why, with illustrations that make the mechanism visible.
The setup: a skewed dataset
Consider a transaction table with five countries: US, DE, ES, IT, and FR. Eighty percent of the records come from the United States. The remaining twenty percent is split evenly among the other four countries.
This is not an unusual distribution. In most enterprise datasets I have worked with, a single value dominates at least one important column.
What Z-ordering does with skewed data
Z-ordering uses a Z-order space-filling curve to sort data across multiple columns simultaneously. When you run OPTIMIZE my_table ZORDER BY (country, date), the engine processes the applicable optimization scope, interleaves the bits of the country and date values using the Z-curve, sorts the records, and writes them into new files.
The critical property of the Z-curve is that it treats all values uniformly. It does not know or care that US appears eighty times more often than FR. It zigzags through the two-dimensional space in a fixed pattern.
A Z-order curve alone does not imply that every file contains every country. Actual min/max ranges depend on sorted values and file boundaries. Measure files read for each predicate; the former illustrated layout was not generated by a Databricks test.
What Liquid Clustering does differently
Liquid Clustering uses a Hilbert space-filling curve instead of a Z-order curve. The Hilbert curve has a property that matters enormously for skewed data: better locality preservation. Points that are close together in multi-dimensional space stay closer together in the one-dimensional ordering.
A fixed Hilbert curve is not inherently adaptive to data density. Locality properties alone do not establish the files produced by liquid clustering or its advantage for this dataset.
The former five-equal-file illustration was inconsistent with 80% US rows: three of five equal-sized files represent 60%, not 80%. Without measured file sizes and boundaries it cannot establish a four-of-five skipping result, and that illustration is removed.
The skew advantage: both majority and minority queries benefit
Here is the insight that most explanations miss. Liquid Clustering does not merely help queries on minority values. It helps queries on the majority value as well.
The maintenance difference
There is a second dimension to this comparison that matters in production: what happens when new data arrives.
ZORDER optimization is incremental and can be scoped to partitions; it does not always rewrite the entire table. New data can change skipping effectiveness, so measure maintenance costs and query benefits.
Liquid clustering supports incremental maintenance, but no guarantee limits a run to one or two files or leaves every minority file untouched. When predictive optimization is not used, schedule OPTIMIZE as documented. Maintenance consumes resources.
Summary
Comparative file counts and scan percentages require a reproducible Databricks workload; no measured comparison is available here.
Select clustering from measured predicates, file statistics, maintenance cost and runtime support. Neither method guarantees improvement for every query or a universal remedy for skew. No Databricks runtime benchmark was performed in this review.
Deciding what to change in your data platform?
Compare optimisation, modernisation and migration against your workload and business requirements.
