Data Warehousing – Strategic Tool

Today's global organizations need to implement a data warehousing solution to centrally store and make sense of customer, sales and analytics data housed in various disparate systems. With the wide spread adoption of computing within the enterprise, the rate at which information is being created has grown exponentially.

Bill Inmon defines a data warehouse as "A subject-oriented, integrated, time-variant and non-volatile collection of data in support of management's decision making process."

You can use a data warehouse to analyze a particular subject area. For example, sales can be a particular subject. It integrates data from multiple data sources and thus standardizes the structure and attributes this information to be analyzed in a common format. Business intelligence (BI) is the art of analyzing information in a data warehouse to identify trends and make intelligent decisions.

Data warehouses are typically used to store historical data. In most transactional systems it makes sense to keep only the current or the most relevant data due to database performance reasons. You should never alter the historical data to maintain its integrity.

Data Life Cycle Management

Data warehouse is an important asset for organizations to maintain efficiency, profitability and gain competitive advantage. Organizations collect data through many touch points - Online, Call Center, Sales Leads, Inventory Management. The data collected has different degrees of value and business relevance. As data is created, it has to be managed through a well-defined Data Life Cycle Process.

Pre-Data Warehouse

OLTP databases are where operational data is stored. OLTP databases can reside in transactional applications such as Enterprise Resource Management (ERP), Customer Relationship Management (CRM), Supply Chain and Point of Sale systems. These OLTP databases are designed for transaction speed and accuracy.

Metadata helps maintain the sanctity and accuracy of data entering into the data lifecycle process. It helps ensure that data is in the right format and relevant. The data cleansing process will go a lot quicker if you do a good job with the metadata to begin with.

Data Cleansing

Before data enters the data warehouse, the Extraction, Transformation and Loading (ETL) process ensures that the data passes the data quality threshold. ETLs are also responsible for running scheduled tasks that extract data from OLTPs.

Data Repositories

The data warehouse repository is the database that stores active data of business value for an organization. The data warehouse modeling design is optimized for data analysis. There are variants of data warehouses - data marts and ODS. Data marts are not physically any different from data warehouses. Think of a data mart as a smaller data warehouse that focuses on a particular department instead of the entire company.

Data warehouses act as a repository for historical data and not always efficient for providing up-to-date analysis. This is where Operational Data Stores (ODS) come in. ODS are used to hold recent data before migration to the Data Warehouse and thus you can tap into them to analyze information that is not in the OLTP but happened recently.

Front-End Analysis

The last and most critical pieces are the front-end applications that business users use to interact with data stored in the repositories. Data Mining is the discovery of useful patterns in data to help with prediction analysis and classification. It helps answer questions such as "What is the likelihood that a customer will migrate to a competitor?"

Online Analytical Processing (OLAP) is used to analyze historical data and slice the business information. Reporting tools are used to keep track of key performance indicators (KPI).

Data Visualization tools are used to display data from the data repository. Often data visualization is combined with Data Mining and OLAP tools. Data visualization can allow you to manipulate data to show relevancy and patterns.

Summary

A data warehouse implementation enables your team to have easy access to information required to identify trends and gain a better understanding about the environment that your business operates in.

Data warehouses also increase the consistency of the data and allow it to be checked over and over to determine how relevant it is. Because most data warehouses are integrated, you can pull data from many different areas of your business, for instance human resources, finance, IT and accounting.

Tags: Business Intelligence, Data Warehouse

Business Intelligence & Dashboards

Overview

Business Intelligence (BI) and Dashboard Applications help increase the competitive advantage of an organization by enabling it to make better decisions based on multi-dimensional analysis of information within and outside its four walls. Business Intelligence helps companies with detailed and timely information about the competitive environment and internal operations. Every employee in today’s organization should have a dashboard view into the key metrics impacting their work to enable them to make better decisions backed by data instead of being based on gut feeling.

Benefits of BI

A well thought-out and executed BI strategy provides a 360-degree view of data and metrics from different divisions and functional areas so teams are not looking at information silos.

• Helps generate ideas for new business initiatives


• Results in more targeted marketing campaigns


• Provides a better sense of customer needs and desires


• Creates a strong understanding of competition


• Positions an organization to deal with evolving market conditions in a proactive manner


• Highlights business areas that require improvement

Key Components of the BI Strategy

• WHO: Identify the people that will lead the BI initiative from IT and different business teams.


• WHAT: Clearly outline the role of BI in the organization. What are you trying to achieve with the BI program? Do you want to build company-wide metrics or Key Performance Indicators (KPIs) that can be used to measure and drive company-wide strategy? Do you want to build decision-making engines to help identify patterns and trends?


• HOW: Identify the tools and processes that will need to be in place to implement the BI strategy to help achieve the objectives.

Process

The BI process starts with the extraction and storage of data from multiple internal and external data sources and storing it in a data warehouse. This involves extracting, transforming and loading data into the repository and then using tools to manage the retrieve metadata. The BI tools then enable the generation of ad-hoc custom reports that help with historical analysis and forecasting of business trends. There are several advanced BI tools in the market today that can crunch through large volumes of data and help you make sense of it. The ability to extract, integrate, analyze and interpret business information in a timely manner makes BI a vital capability for any organization.

• Data Sourcing: In any organization, source data typically resides in spreadsheets, multiple homegrown databases, and third party systems. The first step in the BI process is to identify all relevant data sources and then aggregate it in a standard template.


• Data Analysis: Business Intelligence is about synthesizing useful knowledge from collections of data. It is about understanding current trends, integrating and summarizing disparate information, validating models of understanding, and predicting future trends. This process of data analysis is also called data mining or knowledge discovery. Typical analysis tools use probability theory, statistical methods, operations research and artificial intelligence techniques.


• Decision Support: BI helps you uncover important data, such as products that are not performing well, market demand for different services, and poor staff performance, so that you can take preventative steps. It helps you analyze and make better business decisions.

Business Intelligence Trends

• Information Overload: More data translates to a greater need to manage it and make it actionable. Organizations are recognizing they don't have the information they need to manage the business. The data is there, but it's trapped in different silos and its accuracy can't be trusted. How information is entered can vary widely from how it needs to be used to make organizational decisions and, all too often, definitions vary from silo to silo. For example, finance and marketing could define gross margin differently, which influences how and what numbers are reported.


• Fewer BI Vendor Choices: Large ERP/CRM vendors are integrating business intelligence capabilities into their products by acquiring independent BI vendors. This will result in fewer choices new dependencies.


• Operational Business Intelligence: Traditionally, BI is not a top-of-mind investment for companies, but rather an afterthought once the major application decisions were made. However, companies now are making BI tools and dashboards available at all levels of the corporation to enable employees to make smarter decisions.


• Unstructured Data: E-mail, memos, voicemail messages and other sources of unstructured data are rich sources of information, and companies are responding by looking for ways to blend structured and unstructured data for better decision-making. For example, retailers could add comments and complaints from e-mail and call centers into a BI application to enhance their market segmentation analysis.


• Data Visualization: The next generation of business intelligence applications like JMP (SAS), Spotfire (Tibco), Tableau, Thinkmap and others provide visualization for complex data.


• BI moves to the cloud: The increasing scalability, performance, flexibility, and availability demands on the enterprise BI infrastructure will increase the adoption of the cloud to host and deliver these services.


• Most companies will fail at BI initiatives: Most organizations will not have the information, processes and tools their managers need in order to make informed, responsive decisions. This is due to lack of proper investment information infrastructure and dashboard tools.

Summary

Business Intelligence helps predict and anticipate market changes, minimize exposure to risk, reduce operational cost and boost productivity. The continuous monitoring of the key performance indicators in a Business Dashboard allows organizations to change direction in a timely manner.

Tags: Business Intelligence, Dashboards, Data Warehouse, Reports