From Business Intelligence to Enterprise IT Architecture, Part 2

Originally published May 5, 2010

Part 1 of this series described how current business trends demand greatly improved integrity in the information and processes that lie at the core of every enterprise today. Because of the ever-widening scope and increasing speed of business, this need for integrity extends to issues of consistency across multiple data stores and types, timeliness of information, reliability of processes, support for social networking and, indeed, to every aspect of IT systems. The proposed solution was to look back to the original driving principles of data warehousing and understand how they apply in the current situation and where they need to be enhanced. This article provides the initial view of a logical architecture, BI2, based on these considerations.

Sense and Respond and the (Non-) Evolution of Business

In a previous series of articles entitled “BI is dead, long live HEB”, I introduced the powerful concept of sense and respond, first defined a 1992 Management Review article by Stephan H. Haeckel as an approach to managing an adaptive business in a continuously changing world.1 Even three years ago, when I wrote these articles, it was clear that business was in need of (and technology was moving toward) an integration of all aspects of business activity. As seen in Figure 1, repeated from that series, I termed a business with such a level of integration as “highly evolved,” because such a business would be able to plan for and react to faster and unpredictable change in its external environment. Such behavior requires advanced sensors of external events, a closed-loop feedback system for monitoring the outcomes of action taken, as well as more sophisticated organizational structures for decision making and more highly networked social interactions than found in a traditional business.



Figure 1:
The Sense and Respond Loop and the Highly Evolved Business

In the intervening years, we’ve seen an increasing drive in business for speed of response and a demand for ever more timely and accurate sensing. However, there has been limited recognition of how this drive for speed demands extensive integration of information and deeply linked processes. Once a business has reached a certain size or level of complexity, advances in speed can no longer be achieved unless these integration issues are solved. This level of complexity has been reached in many businesses, and the combination of rapid action-taking combined with a lack of integration (of both information and processes) can lead to a rapid spiral into dysfunctional behavior and faulty decision making. Such effects have been seen dramatically, for example, in the recent near collapse of the financial industry.

In the interim, I’ve stopped using the term highly evolved business because, in my opinion, business structures and approaches have not yet really reached the required level of sophistication. But, technology has certainly advanced, allowing us to begin to conceive of the architectural structures that are required to allow such business evolution to occur.

People, Process and Information

The sense and respond model described above provides the foundation for this new architecture by identifying the three levels at which integration must take place – people, process and information. As shown in Figure 2, these are the three layers of the Business Integrated Insight (BI2) architecture. This article introduces the structure and the layers; later articles will delve deeper into each layer.



Figure 2:
The Three Layers of the Business Integrated Insight (BI2) architecture

Information: The Core of Business

Let’s begin with the information layer, as this is the most familiar layer to data warehousing and business intelligence (BI) professionals. This layer, called the Business Information Resource (BIR), contains all the information needed by the enterprise to conduct its business. And when I say “all,” I do mean all! This includes, of course, the information traditionally stored in a data warehouse and all the data marts, as well as the operational data used to run the business. It also covers soft information2 such as emails, documents and even images and other such content. It stretches from information stored in servers to that on users’ PCs, as well as the cloud. And, in addition, it includes all information from external partners, regulatory bodies and even the Internet used by the business.


The extent of this information is broader than most architectures would dare to include. Some might argue that it is foolhardy to include such a breadth of information. Whilst agreeing with the premise that the scope of information covered presents significant challenges in organization, definition, quality management, maintenance and other aspects, I contend that there is little choice but to address this scope. Information from each and every one of these areas contributes today to the fundamental and basic tasks of running and managing a business. If excluded from consideration in the architecture, any part of this information set can contribute to a level of disintegration in the information asset of the business. The extent of the contribution varies considerably depending on the information involved. The level and type of disruption relates to such traditional concepts as operational vs. informational data, data vs. content, internal vs. external or personal vs. centralized information. But we need a deeper and more formal analysis of how information is created, structured, used and managed.

Consider the impact, for example, that erroneous, personal spreadsheet information can have on the calculation of published business results. Or how messages on Twitter can contribute to knowledge of customer satisfaction with a product. The role and relevance of all information is modeled in the BIR, as will be described in Part 3 of this series. For now, we simply recognize that the BIR structure is more complex than the layers above it – shown by its three-dimensional depiction in contrast to the other two and the number of information classes represented by the cells in the layer. The bottom line is that any information that can add to or subtract from the overall integration of business information should be within scope of the architecture, and the impact on such integration of any set of information must be well defined and understood.

Process: Running a Business with Panache

The history of running a business has seen a steady increase in the level of automation and consolidation of the activities needed to achieve the goals of the enterprise. Early automation of the manufacturing process, for example, moved from artisan creation of individual products to an assembly line where each action was optimized and linked together in a defined and managed process. This approach integrated processes that were traditionally separate within the business; and, today, just-in-time manufacturing moves process integration beyond the walls of the business.

Within IT, we have seen operational systems develop from individual, stand-alone applications to highly integrated systems such as SAP, creating integrated processes across vast swathes of the business. Service oriented architecture (SOA) has further driven both wider and more flexible integration by creating processes consisting of well-bounded and defined services linked by user-defined workflows. This development has two very significant implications.

First, activities previously beyond the normal operational processes of the business become available for integration within existing processes. One place where this clearly appears is in operational BI. Here, queries or analyses of specific short-term trends are embedded within operational process flows as services in the process. Users are thus encouraged to transition seamlessly between two very different ways of working as and when needed. Operational concepts of process and response time extend into the informational world. Similar overlaps appear between operational/informational and collaborative function. The result is that process orientation extends beyond operational applications to become pervasive across all types of computing activity.

Second, SOA enables users to define and change workflows. While more in theory than practice in current implementations, this functionality is becoming more pervasive and is a key direction for SOA to enable adaptive and flexible workflows unimpeded by IT bottlenecks. Closely related to this, but coming from Web 2.0, is the creation of mashups by business users. As a result, processes such as “application” design and ETL that were formerly the preserve of IT can potentially be performed by business users. The process layers must therefore cover both business and IT processes equally.

These considerations imply that the process layer in the BI2 architecture, called the Business Function Assembly (BFA) has a considerably wider scope than might be traditionally assumed. At its heart, the BFA is a process definition, operation and management environment, as can be seen in SOA, business process management and similar tools. But with a scope extending into informational and collaborative activities, Web / Enterprise 2.0 function also plays a strong role.

People: Acting Together or Falling Apart

The top layer in the architecture, the Personal Action Domain (PAD) describes how users behave and interact with the two underlying layers of process and information. Of the three, this layer is the most loosely defined as yet. User motivation, intention and behavior are more the domain of psychology than IT. However, a model of how people act to manage, run and innovate in the business sphere is vital to the definition of workable processes that users actually use and the creation and maintenance of usable information.

In the future, the PAD layer must include the model and rules that describe behavior as well as some form of engine that uses this model to bridge from user intent to the processes required to deliver the intended outcomes. This model and engine must reflect the highly adaptive nature of human behavior in order to enable the highest possible level of innovation in the business. Today, however, this layer consists only of rather stand-alone components of user interface functionality. This layer, together with the BFA layer above, will be described in more detail later in this series.

Conclusion

So far, we have but skimmed the surface of the Business Integrated Insight architecture. However, the drivers are clear: enabling flexible and adaptive behavior within a highly integrated information and process environment. The goal is to enable business users to react rapidly and safely to the most subtle changes in the external environment and to clearly relate real-world outcomes to specific decisions made and actions taken. Companies now exhibiting early developmental aspects of business integrated insight stand to gain business advantage in a rapidly changing marketplace.

End Notes:
  1. See “Adaptive Enterprise: Creating and Leading Sense-and-Respond Organizations,” Harvard Business School Press, 1999 and www.senseandrespond.com for further details.
  2. See Part 1 of this series for a definition of soft and hard information.
 

  • Barry DevlinBarry Devlin
    Dr. Barry Devlin is among the foremost authorities in the world on business insight and data warehousing. He was responsible for the definition of IBM's data warehouse architecture in the mid '80s and authored the first paper on the topic in the IBM Systems Journal in 1988. He is a widely respected consultant and lecturer on this and related topics, and author of the comprehensive book Data Warehouse: From Architecture to Implementation.

    Barry's interest today covers the wider field of a fully integrated business, covering informational, operational and collaborative environments and, in particular, how to present the end user with an holistic experience of the business through IT. These aims, and a growing conviction that the original data warehouse architecture struggles to meet modern business needs for near real-time business intelligence (BI) and support for big data, drove Barry’s latest book, Business unIntelligence: Insight and Innovation Beyond Analytics, now available in print and eBook editions.

    Barry has worked in the IT industry for more than 30 years, mainly as a Distinguished Engineer for IBM in Dublin, Ireland. He is now founder and principal of 9sight Consulting, specializing in the human, organizational and IT implications and design of deep business insight solutions.

    Editor's Note: Find more articles and resources in Barry's BeyeNETWORK Expert Channel and blog. Be sure to visit today!

Recent articles by Barry Devlin



 

Comments

Want to post a comment? Login or become a member today!

Be the first to comment!