Data Warehousing is a unique blend of Technologies, Processes (business rules), and People (knowledge workers) that integrate disparate sources of data and transform it into usable views of information. Decision Makers are then able to effectively discover, analyze and derive knowledge, culminating in cross-functional analytic views that enable Management to strategically move new products to market quicker.
Data Warehouse (DW) is a relational database designed for query and analysis rather than for transaction processing. It usually contains historical data derived from transaction data, but it can include data from other sources. It separates analysis workload from transaction workload and enables an organization to consolidate data from several sources.
Relational Database Management System (RDBMS) designed to:
- Store hundreds of Terabytes
- Model the enterprise
- Start small and grow
- An open system
- Available on UNIX MP-RAS & Windows NT, Windows 2000
- Compliant with industry ANSI standards
- Data evenly distributed
- Ensures top performance
- Enables linear growth by adding “nodes”
Analysis vs. Standard Reporting
- Ask complex questions
- Ask new questions
In addition to being a relational database, a Data Warehouse environment includes an extraction, transportation, transformation, and loading (ETL) solution, an online analytical processing (OLAP) engine, client analysis tools, and other applications that manage the process of gathering data and delivering it to business users.
The Data Warehouse track consists of:
- Data Warehouse Readiness Assessment
- Data Warehouse Strategy and Roadmap
- Data Warehouse Architecture (Link)
- Data Warehouse Implementation
- Data Mart Design and Implementation
- Establishing a Data Warehouse Center of Excellence (COE) (Link)
Data Warehouse Readiness Assessment: A one day corporate assessment seminar to prepare and educate clients in the design, architectures and key concepts to embark on a Data Warehouse project.
Data Warehouse Strategy and Roadmap: In-depth consulting services to help define the Data Warehouse strategy and roadmap to incrementally build the architecture. We build on our experience in developing Department and Enterprise Data Warehouse Frameworks for Pharmaceutical, Consumer Goods, Banking, Brokerage, Insurance and Information Services clients.
Data Warehouse Architecture: We help the client to choose the applicable Data Warehouse architecture depending upon the specifics of an organization’s situation.
- Three common architectures are:
- Basic Data Warehouse Architecture
- Data Warehouse Architecture with a Staging Area
- Data Warehouse Architecture with a Staging Area & Data Marts
Basic Data Warehouse Architecture
We use Basic Architecture for end users to directly access data derived from several source systems through the Data Warehouse. In Basic DW Architecture operational data needs to be cleaned and processed before putting it into the DW. This can be done programmatically. The metadata and raw data of a traditional OLTP system are present, as is an additional type of data, summary data. Summaries are very valuable in Data Warehouses because they pre-compute long operations in advance.
Data Warehouse Architecture with a Staging Area
We architect most Data Warehouses with a staging area, which simplifies building summaries and general DW management.
Data Warehouse Architecture with a Staging Area & Data Marts
We can customize the DW architecture for different groups within your organization by adding data marts, which are system designed for a particular line of business. The following illustration shows an example where purchasing, sales, and inventories are separated. In this example, a financial analyst might want to analyze historical data for purchases and sales.
Data Warehouse Implementation
After the appropriate architecture has been chosen, we size the implementation of the Data Warehouse project. A typical team consists of a project manager, architect, two ETL developers and some extract programmers. Pricing is based on the sizing, architecture, number sources to be integrated and the number of data marts and or portals to be built.
Data Mart Design & Implementation
A data mart is the access layer of the Data Warehouse environment that is used to get data out to the users, a subset of the Data Warehouse that is usually oriented to a specific business line or team. Tools used to build a data mart include Oracle RDBMS, SQL Server; using OLAP: Cognos, Business Objects, Qlik, Tableau, Essbase, ROLAP; MicroStrategies.
Data Warehouse Center of Excellence (COE)
We are experienced in establishing the Center of Excellence for Data Warehouse. The Creed by which we abide is: “Excellence is the Result of Caring more than others think is Wise, Risking more than others think is Safe, Dreaming more than others think is Practical, and Expecting more than others think is Possible.” ― Ronnie Oldham
COE EDW Strategic Principles
The COE Principles that we live by:
- Information has dimensions
- Information is an enterprise asset
- Information is available at all levels
- Information is distributed – pulled when needed
- Hierarchical implementation – stored at lowest atomic level and aggregated-up
- Information includes all data and data types
- Business Process Custodians are responsible for timeliness, availability and sharability of the data
- A strong metadata strategy is critical for a successful warehouse/mart implementations
- Need-to-know security on all levels of data is required