Enterprise Data Management and its Tight Coupling with Data Warehousing and Business Intelligence

Originally published November 20, 2008

With longer time to market and “foggier” ROI from data warehousing and business intelligence related investments, companies are increasingly analyzing why that is the case. Whether it is the bad quality of data, “project-wise” governance, integration woes or simply “multiple versions of the semi-truth,” organizations are realizing the importance of placing higher investments into a wise enterprise data management (EDM) strategy to reduce delivery times and reap better returns.

Taking a leaf from Wikipedia, EDM refers to the ability of an organization to precisely define, easily integrate and effectively retrieve data for both internal applications (such as business intelligence, enterprise resource planning and customer relationship management systems) and external communication (for compliance and regulatory reasons). EDM is focused on the creation of accurate, consistent and transparent data content. It emphasizes data precision, granularity and meaning, and is concerned with how the content is integrated into business applications as well as how it is passed along from one business process to another. The goal of enterprise data management is trust and confidence in data assets.

By this definition, there are at least six intrinsic data components that, in my opinion, should contribute toward a good EDM strategy (see Figure 1). These are:

  • Data architecture

  • Data quality management (DQM)

  • Metadata management

  • Master data management (MDM)

  • Data security

  • Data governance

Let’s examine each one of these in the data warehousing and business intelligence (BI) context.

Data Architecture

In a BI environment, it is extremely important to map, visualize and, ultimately, model the structure and flow of data as it flows from the source systems to the various reporting systems. The business requirements, along with the intrinsic relationships that exist among various entities and attributes, assist in defining the conceptual, logical and physical models typically found in a data warehousing and business intelligence environment.

Data architecture describes how data is processed via ETL (extract, transform and load), stored in its staged journey and, finally, used for decision making. Thus, key activities involved in data architecture – such as data modeling, data administration, data visualization and data flow analysis – enhance the ability of an organization to precisely define and easily integrate enterprise data.

Data Quality Management

A big part of successful BI implementations is the fact that the end users (including management) have a high degree of trust and confidence in the reported data. That degree of trust and confidence is directly proportional to the high level of emphasis placed on data quality as data flows within and outside the organization.
 
Eventually, users care about two main things in their reports, namely:

  1. Can it be generated in decent time and get the desired results?

  2. Is the data consistently accurate for the different slicing and dicing scenarios that the report offers?

Hence, without good data quality management, one cannot have trust and confidence in the data, which is a prerequisite to glean actionable business intelligence.

Metadata Management

As Bill Inmon mentions in his article about success factors for data warehousing, metadata is the glue that holds the data warehouse together. In a typical data warehouse environment, information about the meaning (semantics) and structure (syntax, constraints and relationships) of data is captured in the metadata for various stages of a data flow, including source systems, ETL/ELT tools, data warehouse (operational data store, multidimensional database) data models, reporting systems and business subject areas. Managing this metadata (irrespective of its central, distributed or federated topology) becomes imperative to allow easy integration of various data warehouse components, as well as to provide the necessary data lineage for regulatory and compliance purposes (external communication).

Master Data Management

Master data is defined as the data that has been cleansed, rationalized and integrated into an enterprise-wide “system of record” for core business entities, such as Customers, Products, Suppliers, Employees, Accounts, Locations, Branches, Factories and Stores. By the very nature of multiple source systems in a data warehouse environment, data coming from various sources could differ in semantics and usage. Without a consistent definition of these important business entities, an organization can face downstream problems in areas such as:

  • Customer Satisfaction – the sales department is unaware of the marketing view of customer needs

  • Decision Support – incorrect decisions resulting from bad quality data

  • Operational Efficiency – people, processes and disparate systems have to be in place for mapping one definition or view to another

  • Regulatory and Legal Compliance – inconsistent and inaccurate data leading to fines, delays in reporting and brand tarnishing

There have been many customer implementations where I have seen operational data stores (e.g., financial, supply chain, CRM, vendor management) being populated from source streams, but with slight variation in their semantics (e.g., customer definitions) and suffering from classical database update issues. The resulting bottom-line impact from a direct marketing campaign where multiple mailers have to be sent to the same person living at the same address can result in huge cost increases. The need for master data management (MDM) is acutely felt when organizations embark on an enterprise data warehouse strategy as they mature from silos of departmental data marts.

Thus, MDM contributes toward creating accurate, consistent and transparent enterprise-wide data content and becomes a source for clean and consistent data for the data warehouse.

Data Security

Providing the right access to appropriate objects and systems for the right people at the right time lies at the core of data security.

The Société Générale fraud case in early 2008 is a good example of why protecting the confidentiality, integrity and availability of data is of paramount importance to organizations, especially in the Internet age. The data warehouse environment acts as a central repository for all of the key business data, which is then fed into various downstream systems. The role of data security is to prevent unauthorized access and facilitate effective retrieval of this data by both internal applications as well as external-facing communications.

Data Governance

Data governance implies assigning (people) responsibilities and ownership around data and processes to ensure effective data management, with the use of appropriate technology. It is the “glue” that holds the entire EDM strategy together.

In the context of the enterprise, many organizations resort to establishing a formal business intelligence competency center (BICC), creating roles such as the chief data officer (CDO) to provide the seriousness and espouse the culture that treats enterprise data as a “strategic asset.” This organizational body is also responsible for making project priority decisions, providing skill-set training and cross-functional grooming, and providing opportunities within the enterprise as a whole for selecting tools and technology, as well as for evangelizing the ROI benefits of various BI initiatives across business and IT.

Successful BI implementations have realized the importance of data governance and have made the right investments in being able to precisely define who the producers, administrators and, possibly, consumers are for its enterprise data.

One grassroots movement worth mentioning in the EDM space is EDMCouncil.org, which is a nonprofit trade association focused on managing and leveraging enterprise data as a strategic asset to enable institutions to increase efficiency, minimize risk and create competitive advantage.

As you can see, investing in a sound EDM strategy built on the six components outlined at the beginning of this article can pay rich dividends toward successful data warehousing and business intelligence investments. As a matter of fact, I would even go beyond that and say that not only data warehousing and business intelligence, but other data initiatives (as outlined in the oval in Figure 1) can benefit from a good EDM strategy (as shown along the edges of the hexagon in Figure 1). Some of the sub-components of the EDM strategy are listed inside the hexagon.

alt

Figure 1: Six-Component EDM Strategy and its Beneficiaries

Hence, as you sow (into EDM), so you reap (better ROI in data warehousing and business intelligence and beyond.

As always, I do encourage you to visit some of other contributors/channels on BeyeNETWORK.com. I look forward to your inputs through comments and feedback to enrich our experience and to make my BeyeNETWORK expert channel a truly interactive forum.

'Till then, so long, and as Bartles & Jaymes would say, “... and thank you for your support.”

SOURCE: Enterprise Data Management and its Tight Coupling with Data Warehousing and Business Intelligence

  • Ajay BhargavaAjay Bhargava
    Ajay has more than 20 years of industry, research, mentoring and teaching experience in areas relating to databases, data warehousing (DW), business intelligence (BI) and data mining. His career started with working on database product development with companies such as Novell, Pervasive Software and BMC Software. He has been Novell’s chief database architect to standards organizations such as SQL Access Group (SAG), ODBC and IDAPI, and has been doing strategy consulting and implementation in areas of BI and DW since 1998 to various customers in retail, banking and financial services, telecom and insurance. As adjunct faculty at the University of Texas (UT) in Austin, he has taught a database design course for 4 years. Ajay is currently heading Enterprise Data Management (EDM) Services at TCS and teaches an undergraduate database management systems (DBMS) course at the College of Engineering in Pune. He can be reached at ajaybhargava00@yahoo.com.

Recent articles by Ajay Bhargava



 

Comments

Want to post a comment? Login or become a member today!

Posted January 25, 2009 by Anonymous

Nice one.The article explains the very basics of EDM.THope the data architecture covers the Data integration part. Hence going by Kimballs approach do the EDM built out from existing Data marts across  departmental silos?

Is this comment inappropriate? Click here to flag this comment.