Data lineage impact analysis before a warehouse migration traces every path from the source systems through the warehouse to the reports, extracts and applications that consume the data. It tells you, before anything moves, what each object feeds, who depends on it and what can be retired. That answer sets the migration scope, the order of the migration waves and what reconciliation has to prove before cutover.
We at DWHPro have spent more than twenty years working inside Teradata systems and now lead migrations from Teradata to cloud platforms. In our experience, programmes that skip this step either migrate far more than they need or discover a forgotten consumer on the day of cutover.
Why lineage decides scope, sequence and risk
A Teradata warehouse that has run for fifteen or twenty years carries layers of history: tables nobody reads, reports that were replaced but never switched off, and consumers that the central team never knew about. Typical examples are spreadsheets with direct connections, statistical tools, nightly extracts to other systems and regulatory reports that run once a year.
Without lineage, the programme has two bad options. Migrate everything, and pay to move, test and run objects without value. Or migrate what the team believes is used, and risk breaking a business process after cutover. Lineage replaces belief with evidence.
Where lineage comes from
No single source gives the full picture. A complete analysis combines several, each covering a different part of the chain.
| Source | What it shows | What it misses |
|---|---|---|
| ETL tool repositories (for example Informatica, DataStage) | Designed data flows from source to target tables | Logic outside the tool, and anything changed by hand |
| Load and transformation scripts (BTEQ, TPT, stored procedures) | The actual code that writes the tables | Dependencies built at run time from parameters |
| Teradata query log (DBQL) with object logging | Which users and applications actually read and wrote which objects, and when | Only what was logged during the retained period; views are resolved to their base tables |
| BI and reporting tool metadata | Which reports and dashboards use which tables and views | Desktop tools and personal extracts |
| Interviews with business owners | Why a report matters and what happens if it is late or wrong | Precision; people forget rare processes |
The query log is the most underrated of these sources. It records real use rather than design, including users nobody listed in the documentation.
Dependency is not the same as importance
Lineage tells you what depends on what. It does not tell you whether anyone cares. We therefore combine two views for every object:
- Dependency: what feeds it and what it feeds.
- Usage: who read it, how often and how recently.
Objects with no downstream consumers and no reads over a full business cycle are candidates for retirement, but only after their owners have confirmed it. Year-end and audit processes are the classic trap: an object that looks dead in a three-month window may be essential every December.
What goes wrong
- The analysis stops at the warehouse boundary. The most painful breaks are downstream: files sent to partners, feeds into operational systems, models in other departments.
- Generated or dynamic code is ignored. Macros, procedures and scripts that build their logic from parameters do not appear in static parsing.
- Tool output is treated as complete. Automated lineage scanners are useful, but each sees only the sources it can read. Someone has to own the gaps.
- The query history is too short. A few weeks of logging misses quarterly and annual processes.
- Lineage is built once and then goes stale. The source system keeps changing during a migration that runs for months.
Turning lineage into migration decisions
Lineage is only valuable if it changes the plan. We use it for five decisions:
- Scope: what is migrated, what is retired, and what is rebuilt differently.
- Wave design: objects are grouped with their sources and consumers, so that each wave delivers complete business outcomes instead of half-connected tables.
- Reconciliation priority: the objects behind critical and regulatory reports get the strictest checks. We describe how we prove a migration in how to prove a Teradata migration before cutover.
- Consumer communication: every known consumer is told what changes for them and when.
- Cutover order: upstream objects move before the consumers that depend on them, and nothing is switched off while it still has readers.
In our Swiss retail migration from Teradata to Snowflake, workload analysis came first, before the target data models were designed and the critical ETL pipelines were rebuilt. The details are in our case studies.
Keep lineage alive on the target platform
The cloud platforms now capture lineage for work that runs inside them. Snowflake provides data lineage on Enterprise Edition and higher. Databricks Unity Catalog captures lineage automatically, down to column level, for queries run on Databricks. On Google Cloud, Knowledge Catalog (formerly Dataplex Universal Catalog) records lineage for BigQuery tables and views.
These features only see what runs on the platform. External consumers and upstream sources still need to be registered. Insist that lineage is a programme deliverable with an owner, kept current through cutover and handed over to operations afterwards.
What to insist on
- Lineage that reaches from source systems to the final consumers, not only inside the warehouse.
- Usage evidence from at least one full business cycle of query history.
- Owner sign-off for every object marked for retirement.
- A wave plan and a reconciliation plan that visibly follow from the lineage.
How DWHPro helps
We build the lineage and usage picture of your Teradata estate from its own metadata and query history, and turn it into a migration scope, a wave plan and a retirement list. This is part of our independent data platform assessment.
Common questions
Can a lineage tool do this automatically?
A tool does much of the collection, but not the judgement. Gaps between sources, dynamic code, downstream consumers and retirement decisions still need experienced people and business owners.
Do we need column-level lineage?
Not everywhere. Table-level lineage is enough for scope and wave planning. Column-level lineage pays off for critical and regulatory reports, where you must prove exactly where each figure comes from.
How much of a typical warehouse can be retired?
It varies too much to give a reliable figure. The analysis itself answers this question for your estate, which is one of the reasons to do it early.
Planning a Teradata-to-Snowflake migration?
Review the assessment, conversion, reconciliation and cutover work needed for a controlled transition.