Cost-cutting, unfortunately, leads to the unpleasant situation that many clients save on the wrong side, starting Data Warehouse projects where they should finish them.
As a Teradata Performance Optimization Analyst, you will probably be confronted with scenarios with no maintained data model, unclear business specifications, and unyielding and incomplete mappings. You can consider yourself lucky for every piece of information you can squeeze out of your customer’s working environment.
In my experience, you are usually not in a position to be responsible for performance. This task is unfortunately often deemed as being an exclusively technical task.
However, although we can fix several performance problems by providing purely technical expertise, you will probably end up with many workarounds not correcting the root causes of your performance problems.
Despite such a purely technical approach that may be applied for a very long time, adding up the costs at the end of the day will leave you in shock! Unfortunately, this head-in-the-sand politics is a daily routine in many companies.
As a performance specialist, you must be a business analyst, data modeler, and developer. This is a critical insight: Performance Optimisation often evolves into fixing the broken data model.
It is your task to make good for the past failures in all these areas of expertise. Try to contact people involved or responsible for these roles in the past at this client.
Sadly, you will often be confronted with uncooperative behavior as people tend to defend their petty areas of expertise. You are questioning their work results from the past. You are on a very delicate mission. Strong management support would probably make your life easier.
Fixing a poorly designed data model is like time travel. You have to go back to the start of the project, get to know the original business requirements, and question how they have been transformed into the existing data model and why. You have to evolve a behavior of questioning all past decisions made.
Most times, it was for lack of budget, a wrong assignment of people to roles, lack of time, or simply missing experience that turned the project into a big mess and a failed state. Still, I think the most outstanding cause is the over-specialization of project members. Over-Specialisation leads to evaporation of responsibilities. Everybody is shifting problems back and forth, and lots of resources are wasted finding the responsible person until a problem is finally solved.
One approach I prefer, if I know that only a redesign of the data model can solve performance issues, is to create a small prototype, demonstrating improvements.
I would take an assessable subject area and redesign the chain. Using such a prototype as a communication tool can make the difference between getting the chance to fix the performance problem or getting an answer like “would be nice, but we don’t have the budget”—the more tangible your approach, the better.
As always, success has to be measurable. Often, reporting will be at the end of the chain, and making some reports performing better on your prototype would be a good starting point for showing your expertise in Teradata performance optimization.
I hope the main messages from this article are clear:
These days, being just a highly specialist developer is not enough. As a performance specialist, you must understand the data warehouse life cycle.