Introduction to Data Warehousing and Business Intelligence
History of Data Warehouse and Business Intelligence:
To really understand business intelligence (BI) and data warehouses (DW), it is necessary to look at the evolution of business and technology.
In the 1970s and 1980s, computer hardware was expensive and computer processing power was limited. A medium-sized business typically operated a handful of large mainframe-based application systems that were designed for operational, data entry purposes. Information needs were addressed through paper reports. Report programs, however, were expensive to write and generally inflexible. A computer programmer was required to write the report programs. Fortunately, salaries for programmers were relatively low during this period.
In the 1980s, relational databases became the rage. Data was stored in tables with rows and columns, not unlike Excel Spreadsheets of today. Although relational databases were much more intuitive for end users, complex logic was often needed to join multiple tables to obtain the information that was needed. Although it was possible for end users to write simple reports, the queries were often inefficient and had to be run after normal business hours, in order not to impact online transactions.
In the late 1980s, many businesses migrated from mainframe computers to client servers. Business people were assigned a personal computer. Office applications such as MicroSoft World, Excel and Access became popular. The personal computer empowered end users and allowed them to develop their own applications and present data in a manner that was meaningful to them, such as in grid or graph format. Excel spreadsheets could easily be tweaked as business needs changed, without the assistance from the IT department. Unfortunately, corporate data remained centralized and was generally not directly accessible to end users.
The need for improved business intelligence and data warehousing accelerated in the 1990s. During this period, huge technological changes occurred and competition increased as a result of free trade agreements, globalization, computerization and networking.
In the early 1990, the Internet took the world by storm. Companies rushed to develop eBusiness and eCommerce applications with hopes of reducing their staffing needs and providing 24 hour service to customers. The volume of application systems mushroomed during this period as a parallel set of Internet applications was deployed. Back-end 'bridges' were built to try to integrate the 'self service' application systems with the legacy 'full service' applications. Unfortunately, integration was often messy and corporate data remained fragmented or inconsistent.
As the demand for programmers increased and salaries climbed, businesses looked for alternatives to custom built application systems. In hopes of reducing costs and remaining competitive, companies purchased software packages from third parties. These packages were designed for generic business requirements and often did not integrate well with the existing legacy systems.
By the end of the millennium, businesses discovered that the number of application systems and databases had multiplied, that their systems were poorly integrated and that their data was inconsistent across the systems. More importantly, businesses discovered that they had lots of fragmented data, but not the integrated information that was required for critical decision making in a rapidly changing, competitive, global economy.
Companies began building Data Warehouses to consolidate data from disparate databases and to better support their strategic and tactical decision making needs.
|
