To delve deeper into Teradata architecture, it’s essential first to understand the fundamental structure of a computer, as this forms the foundation of a Teradata system.
Teradata Architecture – Why Does Everyone Copy It?
Teradata has been a pioneer in data warehousing and an exemplary model for subsequent database systems regarding architecture.
Teradata has stood the test of time thanks to its developers’ foresight, who initially incorporated many details into the system. These details have enabled Teradata to remain competitive to this day.
Examining contemporary database systems, such as Amazon’s Redshift (or Netezza), reveals several features initially implemented by Teradata.
Teradata was originally created with a focus on parallelism in even the most minute aspects, placing it among the leading relational database management systems (RDBMS) for data warehousing to this day.
Data is stored on mass storage devices and loaded into the CPU’s memory for processing.
It is crucial to comprehend that accessing the mass storage device is significantly slower than accessing the main memory. Additionally, accessing data already in one of the CPU caches is much faster than accessing the main memory.
Before processing data, the CPU requires it to be loaded into the primary memory.
The following describes the classic shared-nothing Teradata architecture. Physical storage, resource ownership and limits depend on the deployment and release.
Get the complete Teradata tuning book, free
Teradata Query Performance Tuning: The DWHPro Guide
By Roland Wenzlofsky & Artemios Vogiatzis
206 pages on statistics and demographics, access paths and indexes, query execution plans, and joins. Get the full PDF and practical Teradata updates from DWHPro.
Confirm your email to receive your download link. Privacy policy. Unsubscribe in any newsletter.
About the free book
Teradata Data Distribution
Teradata hashes primary-index values and uses the table map to assign rows to AMPs. Even distribution is a design goal, not a guarantee; repeated values and hash collisions can produce skew.
The Parsing Engine
The Parsing Engine (PE) is a crucial component of the Teradata architecture.
The Parsing Engine generates an execution plan for all necessary AMPs upon receiving a request (such as an SQL statement) to complete that request. Ideally, the plan is structured to allow all AMPs to start and finish tasks simultaneously. This guarantees the best possible utilization of the system in parallel.
The figure above illustrates the position of the BYNET between the AMPs and the parsing engine, which facilitates the exchange of data and instructions via a communication network. Further explanation on BYNET will be provided later in this article.
The main responsibilities of the Parsing Engine include:
- Logging on and Logging Off Sessions
- The parsing of requests (syntax check, checking authorizations)
- Preparation and optimization of the execution plan
- The Parsing Engines use statistics to build an optimized plan.
- Controlling the AMPs by Instructions
- Communication with the client software
- EBCDIC to ASCII conversion in both directions
- Transfers of the result of a request to the client tool
Teradata Systems have the capability to utilize multiple parsing engines.
The system can add more parsing engines as each one has a finite capacity to handle sessions.
Parsing-engine session capacity is release and configuration dependent; verify the applicable system limits rather than treating 120 as universal.
The Teradata AMP
AMPs are the primary agents in a Teradata System that execute instructions from the Parsing Engine, also known as the Execution Plan.
In the classic architecture, AMPs manage assigned logical memory and storage resources. Physical resource separation depends on the platform.
Logical AMP ownership does not imply exclusively attached physical storage in every deployment.
The primary responsibilities of an AMP include:
- Storing and retrieving rows
- Sorting of rows (for details, read How Teradata sorts the result set)
- Aggregation of rows
- Joining of tables (see also: The Essential Teradata Join Methods)
- Locking of tables and rows
- Output conversion ASCII to EBCDIC (if the client is a mainframe)
- Management of its assigned space
- Sending of rows to the Parsing Engine or other AMPs (via the BYNET)
- Accounting
- Recovery handling
- Filesystem management
AMP worker-task limits depend on release and configuration. A historical default of 80 is not a universal capacity limit.
The Teradata Node
Parsing engines and AMPs run on a node, typically a Linux machine with multiple physical CPUs.
Nodes have the capability to operate numerous AMPs, each with its own allocation of primary and virtual memory.
Classic systems allocate logical storage to AMPs. Intelligent Memory manages data placement in memory; it is not the mechanism assigning disks to AMPs. SSD, shared and cloud storage implementations require platform-specific documentation.
Massive Parallel Processing
A Teradata system comprises numerous nodes interconnected by BYNET.
The network within a node, known as BYNET, connects the AMPs to the parsing engine and to each other through software, in contrast to the physical network.
Deciding what to change in your data platform?
Compare optimisation, modernisation and migration against your workload and business requirements.
Get the complete Teradata tuning book, free
Teradata Query Performance Tuning: The DWHPro Guide
By Roland Wenzlofsky & Artemios Vogiatzis
206 pages on statistics and demographics, access paths and indexes, query execution plans, and joins. Get the full PDF and practical Teradata updates from DWHPro.
Confirm your email to receive your download link. Privacy policy. Unsubscribe in any newsletter.
About the free book
It is important to understand that accessing the main memory is many times slower than accessing the main memory. typo here
Thank you very much. I fixed it.
Thanks, I fixed it.
Great article, just thought you’d like to know there’s a couple of errors: “It is important to understand that accessing the main memory is many times slower than accessing the main memory.”, think that first ‘main memory’ should be ‘mass storage’.
Also the phrase “Each node can run hundreds of AMPs. Each AMP has its own portion of the main memory and its own portion of mass memory (called virtual disk).” is repeated both before and after the Teradata Node diagram.
These are minor things, thanks for writing these interesting articles. Keep up the good work.
@Jason
Thank you very much. I fixed it.