DWHPro Insights

Hadoop and Teradata Data Warehousing: A Comparison and Integration Perspective

By Roland Wenzlofsky ·

Hadoop and Teradata Data Warehousing: A Comparison and Integration Perspective

This article records historical Hadoop, MapReduce and SQL-H commentary. It is not a statement of current Teradata integration availability; consult the documentation for the deployed products before designing a new integration.

The marketing hype surrounding Hadoop can often obscure its actual value and related topics.

Big data is a buzzword that has caused confusion and numerous incorrect statements. Some individuals claim that Hadoop and Map Reduce will replace the traditional Data Warehouse, but I have doubts about the accuracy of this prediction.

Teradata and Hadoop have similar technical features, functioning as parallel systems utilizing shared-nothing architecture.

Teradata and Hadoop have similar technical

Hadoop can support distributed storage and processing, but scalability is finite and workload dependent. It is not categorically superior to a database.

A bottleneck can arise at a particular stage in a Teradata Data Warehouse. While additional nodes can enhance Teradata’s scaling capabilities during database data loading, gathering, storing, and loading substantial quantities of data, otherwise known as big data, is still necessary. The Hadoop framework is a viable solution for addressing constraints originating from network bandwidth, ETL servers, and storage devices.

Teradata Query Performance Tuning: The DWHPro Guide book cover

Get the complete Teradata tuning book, free

Teradata Query Performance Tuning: The DWHPro Guide
By Roland Wenzlofsky & Artemios Vogiatzis

206 pages on statistics and demographics, access paths and indexes, query execution plans, and joins. Get the full PDF and practical Teradata updates from DWHPro.

Confirm your email to receive your download link. Privacy policy. Unsubscribe in any newsletter.

About the free book

Hadoop enhances the value of your Teradata Data Warehouse by gathering and processing vast amounts of data before its ingestion into the warehouse.

Hadoop enhances the value of your Teradata Data

HDFS is a cost-effective way to store data instead of leaving it unconsolidated online, storing it on tape, or deleting it after loading.

Hadoop and MapReduce are powerful parallel ETL servers that preprocess incoming data for advanced processing.

ETL platforms can support SQL, procedural code and other transformations. MapReduce offers a different distributed execution model, not unlimited processing capability.

In a standard ETL-Server environment, processing

After the initial excitement, I believe these measures will aid in integrating Hadoop into your data warehousing.

Leading database providers are incorporating Hadoop’s features into their relational database management systems (RDBMS) at the highest level. The long-term efficacy of this integrated approach is yet to be determined. Personally, I remain unconvinced.

The SQL-H discussion reflects a historical integration and its then-observed limitations. It does not establish the capabilities or availability of current Teradata integrations.

Some attempts to participate in the big data

Developers should explore alternative solutions similar to Hadoop in light of the current situation. It’s crucial to take into account the abuse of ETL tools.

If the ETL server is not meeting performance standards for a particular transformation, incorporating the mapping logic into pre-aggregation steps in the RDBMS may be necessary. While this method does disrupt the data lineage, it is currently considered advanced. Nonetheless, removing these implementations may present difficulties in the future.

Evaluate supported interfaces, tool compatibility, pushdown behavior and support lifecycle against current product documentation rather than relying on this historical SQL-H opinion.

Teradata’s SQL-H syntax is a problematic blind

I am optimistic about Hadoop and MapReduce’s potential, but I am apprehensive that some implementations may yield subpar results, necessitating replacement with a superior alternative. Unfortunately, this could entail significant client redesigns to exploit the capabilities fully.

Although big data has received significant attention, numerous firms appear hesitant to embrace these novel technologies. Additionally, I assert that database suppliers should exercise restraint. Despite its capacity to manage vast quantities of data, countless clients continue to encounter difficulties with the basics of data warehousing. I recollect when “active data warehousing” was recognized as a significant advancement.

Deciding what to change in your data platform?

Compare optimisation, modernisation and migration against your workload and business requirements.

Explore independent platform assessments

Teradata Query Performance Tuning: The DWHPro Guide book cover

Get the complete Teradata tuning book, free

Teradata Query Performance Tuning: The DWHPro Guide
By Roland Wenzlofsky & Artemios Vogiatzis

206 pages on statistics and demographics, access paths and indexes, query execution plans, and joins. Get the full PDF and practical Teradata updates from DWHPro.

Confirm your email to receive your download link. Privacy policy. Unsubscribe in any newsletter.

About the free book

Follow DWHPro in Google to see our articles more often in Search.

3 thoughts on “Hadoop and Teradata Data Warehousing: A Comparison and Integration Perspective”

  1. Hi. Yes, after 4 years the picture is still the same. My clients are big companies in the finance and telecommunication markets. All of them have been quite enthusiastic about big data. None of them implemented any useful application in all these years. Structured data stored in a traditional Data Warehouse is still the main source of information. The maximum they achieved was considering Hadoop as a kind of cheap mass data storage…very disappointing and eye-opening.

    Reply
  2. SO this Teradata and hadoop article was written in 2014. Its 2018 now !! Is your opinion still the same ?

    Even I work as Teradata Developer and trying to understand if I need to start learning Hadoop.

    Thanks for helping.

    Reply

Leave a Comment

Teradata Query Performance Tuning: The DWHPro Guide book cover

Get the complete Teradata tuning book, free

Teradata Query Performance Tuning: The DWHPro Guide
By Roland Wenzlofsky & Artemios Vogiatzis

206 pages on statistics and demographics, access paths and indexes, query execution plans, and joins. Get the full PDF and practical Teradata updates from DWHPro.

Confirm your email to receive your download link. Privacy policy. Unsubscribe in any newsletter.

About the free book

Protected by Security by CleanTalk and CleanTalk Anti-Spam