To illustrate the impact these mistakes tend to have for Data Warehouse projects, let me compare it to a team of medics in a hospital and imagine the project is you who is about to receive a costly and critical treatment. I am sure you do not want to hear staff exchanging conversation mentioned below before you doze off at the operating table.
“Does anyone know why we operate the patient? Well, let’s start anyway …”
Most of the Data Warehouse projects start with a lot of enthusiasm but fail at the end because they just did not meet the business objectives. Very well designed Data Warehouses will fail if the content doesn’t help to solve any business needs. Even if you see yourself as a technician first or are in a position without contact to users of the final product, never forget that whatever you produce, it is the customer that must be satisfied.
“Anyone feels like this treatment is successful so far? Can we find out? No ideas?“ (silence).
Even if you are aware of your business objectives, success is not guaranteed by this alone. Every clear business objective is measurable. If you can’t quantify, you will have a tough time commenting on your expenses, and you will show up on top of the blacklist as soon as the next restructuring takes place. The latter usually revolve in cycles of about 3 to 5 years in length.
“We need one nurse for every swab and one class-2 middle-belly-doctor for the first opening cut. Then, the inhalation support technicians, please stand to the right and the exhalation support professionals to the left. Is our scalpel holding team ready?”
Overspecialization became a huge problem in today’s Data Warehouse projects. Industries with a longer history are well aware of the adverse effects of the extreme division of labor (look up “Taylorism”). Think of the Kaizen idea that became famous in the 80s and 90s – , it seems to be the case that Data Warehousing is about to repeat history. Any environment where each project member has to contribute to a very narrow range of skills, lacks understanding of the big picture or is not expected to develop such, is predestined for failure. It is very fashionable these days to create ever more exact roles with the sole purpose of cutting costs – at least at the beginning of a project. From my experience, this never actually pays off. It just moves the costs from the beginning to the end of the project while they multiply along the way and it is the time when you have to hire real data warehouse experts to get back on track again. Be wary of so-called experts without real-world, hands-on experience.
While it seems tempting, from a cost perspective, to pick up the source data and provide it without any costly transformations to the customer – even developers with no or only a little knowledge about data warehousing can be employed to do this, once again this is the recipe for failure. I cannot overemphasize how important the implementation of a theoretically sound data model is. Don’t even think about leaving out surrogate keys, taking over the source system specific naming conventions or trying to solve the issues caused by the mentioned ODS approach at a next place, like in the data marts or reports.
“Why not leave out the surgeon and the anesthesiologist and ask the ambulance driver and the caretaker to help the nurse out a little?”
The usual approach nowadays seems to be to leave out important roles from data warehousing. You should never forget: Data Warehousing is a process, not a sequence of stand-alone projects. Famous positions being regularly cleared are:
Data Warehouse Architect, Data Modeler, Performance Specialists or any other kind of system-specific technical experts (Teradata experts).
The typical project team consists of some highly specialized database developers missing the big picture and a project manager without academic and hands-on experience in Data Warehousing, one shortage often accompanied by a lack of good relations with business departments or even fundamental project management skills. Without the support of a technical lead (Teradata expert) which is capable of taking over the roles of the architect and the general modeling of the solution, project delays and failure are most likely.
“Why should there be any complication? Just slice, rip and sew!”
We should never forget, that data warehousing is, at the end of the day, a very technical topic. The technical component si all too often considered a low-prestige subsidiary location.
Solid education of people involved is required at many levels, in-depth knowledge and fluency are needed in the fields of Teradata, ETL-Tools (Data Stage, Informatica, Ab Initio) and Reporting Tools (Cognos, Business Objects, Microstrategy).
Too often, the interaction of these different technical components is not getting enough attention and performance is almost always ignored completely at the beginning of the project. Often, omitted clarification of professional interaction and compatibility of various software can’t be handled anymore at a later time. One ends up in a minefield of workarounds upon workarounds and never-ending issues around new releases that soon start to lead a life of its own.
“What do you mean by no vital signs? It says “operation successful” on my slide!“
The first mile on the highway to disaster always looks perfect. It appears to be brand-new, there is no speed limit, all traffic lights are on green, and you seem to drive a racing car. This part of the journey ends after the first turn.
First, if you start a project with amounts, assumptions or milestones that are illusionary or grossly out of touch with the average experience of professionals, it does not matter if, but only when you will see yourself in unpleasant meetings, bending over backward to cautiously add reality to the plan.
Second, if the company shows a distaste for straightforwardness, clarity, transparency and decisiveness, not only will the real problems multiply, but, alongside them, the bureaucratic dissimulation tactics will take over more and more of the remaining resources.
Your best project members are usually the first to exit this race to the bottom for better opportunities.