DWHPro Insights

The Medallion Architecture Is Not New. We Just Called It Something Else.

By Roland Wenzlofsky ·

The Medallion Architecture Is Not New. We Just Called It Something Else.

Why data warehouse professionals have been doing “bronze, silver, gold” for over 20 years.

medal

If you have been working in data warehousing for any length of time, the first time you heard about the “medallion architecture” you probably had one reaction: We already do this.

You were right.

The medallion architecture, popularized by Databricks as a core design pattern for their lakehouse platform, defines three layers: bronze for raw data, silver for cleansed and conformed data, and gold for business-ready aggregates. If that sounds familiar, it should. The data warehousing community has followed this exact pattern since the late 1990s.

What Databricks did was give an old idea a catchy name. That is nothing, but it is not the revolution it was marketed as either.

The Pattern We Have Always Used

Many data warehouses I have worked with follow a layered approach to data processing. The names vary, the concept does not.

In a traditional data warehouse, data first lands in a staging area. This is the point of entry. Data arrives from source systems with minimal transformation. The goal is speed: get the data in, validate the basics, and make it available for the next step.

From staging, data moves into an integration layer (sometimes called the cleansed layer, the ODS, or the core warehouse). Here, data quality rules are enforced. Duplicates are removed. Data types are standardized. Business keys are resolved. Source system idiosyncrasies are smoothed out.

Finally, data is transformed into a presentation layer: data marts, dimensional models, and aggregated reporting tables. These structures are optimized for consumption by analysts, dashboards, and business users.

Now, rename those three layers. Call them bronze, silver, and gold. You have the medallion architecture.

So What Did Databricks Actually Change?

The logical concept is identical. The differences lie in the technical implementation and some philosophical shifts about how data should be treated at each layer. These differences are real, even if they do not justify calling the whole thing a new architecture.

Persistent raw data is a design choice. Traditional staging can also retain history, and medallion layers do not require permanent retention. Set retention from replay needs, cost, privacy and deletion obligations.

Medallion is a logical layering pattern, not a requirement that every layer use Delta. Traditional staging can be transactional and persistent too. Delta time travel is bounded by retained logs/data and VACUUM behavior; it does not permit querying any point in history indefinitely. Choose formats and controls for each layer.

Streaming and incremental processing. Traditional ETL was predominantly batch-oriented. You scheduled nightly or hourly load windows, processed everything in sequence, and the pipeline was idle between runs. The medallion architecture is designed around incremental and streaming-capable processing. Databricks features such as Auto Loader (for bronze ingestion) and Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables) enable each layer to update continuously and independently. To be clear: incremental and continuous loading is not a new concept. Teradata supported incremental loading via TPump long before Databricks existed. Snowflake offers Snowpipe for serverless continuous ingestion and scheduled Tasks for orchestration. What the medallion architecture changed was not the existence of these capabilities but their accessibility. The tight integration of Auto Loader and Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables), combined with the layered pattern, made continuous processing easier for teams that had previously relied on batch-only workflows.

Multi-consumer access. Traditional data warehouses assumed a single, tightly coupled system. You staged, transformed, and served data from the same engine. The medallion architecture assumes decoupled consumers. Data scientists might query Bronze directly to explore raw data. Analysts work with silver. Dashboards consume gold. Different compute engines can access any layer because the data is stored in open formats (Parquet/Delta) on shared cloud storage. This is less a feature of the medallion pattern itself and more a consequence of the lakehouse architecture it sits on.

The Marketing vs. The Reality

Let me be direct: the medallion architecture gained so much traction not because of technical innovation.

The big data movement of the 2010s produced a generation of data engineers who came from software engineering, not from data warehousing. Many of them had never built a traditional ETL pipeline with staging, integration, and presentation layers. When organizations moved to data lakes, they often abandoned layered architectures entirely. Everything was dumped into a single S3 bucket or HDFS directory with no structure, no quality enforcement, and no clear lineage. The industry coined a term for this: the data swamp.

The medallion architecture provided these teams with a framework to restore discipline to their data pipelines. It told them: separate your raw data from your cleaned data from your business data. Use distinct layers. Enforce quality progressively.

That was valuable advice. It was also advice that the data warehousing community had been giving for two decades.

What Databricks did well was packaging. “Bronze, silver, gold” is simple, visual, and memorable. It is easier to explain to a room of engineers than “staging, integration, and presentation layers following Kimball’s dimensional modeling methodology.” Good naming matters. But let us not confuse good marketing with invention.

What Teradata Professionals Should Take Away

If you are coming from a Teradata background, you already understand layered data processing intuitively. When you encounter the medallion architecture in Databricks, Snowflake, or any other cloud platform, do not let the new vocabulary intimidate you.

The principles you have applied for years still hold: land data quickly, clean it systematically, and serve it in structures optimized for the consumer.

What has changed is the economics. On Teradata, storage was expensive, so you minimized what you kept in staging. In the cloud, storage is cheap, so keeping raw data permanently becomes practical. On Teradata, compute was a fixed resource, so you carefully optimized batch windows. In the cloud, compute scales on demand, so incremental and streaming approaches become feasible.

The medallion architecture is not a new idea. It is your old idea, running on new infrastructure.

Roland Wenzlofsky is a data warehouse consultant with over 20 years of experience helping major organizations manage and optimize their analytical platforms. He is the author of “Teradata SQL Tuning” and writes regularly about data warehousing, Teradata, Databricks, and Snowflake.

Deciding what to change in your data platform?

Compare optimisation, modernisation and migration against your workload and business requirements.

Explore independent platform assessments

Practical data engineering insights, regularly

Join more than 3,500 data professionals: a regular newsletter with practical insights on data engineering, new data platforms (Snowflake, Databricks, Teradata) and AI.

Follow DWHPro in Google to see our articles more often in Search.

Leave a Comment

Get more practical data engineering advice

Join the DWHPro newsletter for practical data engineering tips, troubleshooting examples and data platform insights.

Protected by Security by CleanTalk and CleanTalk Anti-Spam