Recently in Business Integrated Insight Category

I've previously written about IBM Watson, its success in "Jeopardy!" and some of the future applications that its developers envisaged for it.  IBM has moved the technology towards the mainstream in a number of presentations at the Information on Demand (IOD) Conference in Las Vegas last week.  While Watson works well beyond the normal bounds of BI, analyzing and reasoning in soft (unstructured) information, the underlying computer hardware is very much the same (albeit faster and bigger) as we have used since the beginnings of the computer era.

But, I was intrigued by an announcement that IBM made in August last that I came across a few weeks ago:
"18 Aug 2011: Today, IBM researchers unveiled a new generation of experimental computer chips designed to emulate the brain's abilities for perception, action and cognition. The technology could yield many orders of magnitude less power consumption and space than used in today's computers.

Last week I blogged on the Future of Business Intelligence and today as I'm making final preparations for two events dealing with this topic, I'd like to add a few further thoughts that struck me and deserve some emphasis.  (The events are in Brussels tomorrow, 15 June and in Rome on 22-24 June.  Between them they bring together an outstanding line-up of international speakers, including Rick van der Laans and Jan Henderyckx in the first event and Colin White, Claudia Imhoff, Cindi Howson and James Taylor at the second.  Not to mention yours truly at both!)

To start, two trends in addition to last week's five:

(6)    Operational vs. informational distinctions are disappearing.  In the past, decision support was seen as a relatively relaxed process, typically based on a stable, point-in-time view of the business.  Today, many decisions need to be made based on near real-time data.  This process is known as operational BI and breaks down the boundaries between operational and informational systems, a distinction on which the original BI architecture is based.

(7)    Service oriented architecture (SOA) is reinventing the application landscape.  Although SOA has been promoted since the early 2000s, its uptake has been relatively slow.  However, most application developers recognize that it is the only concept that has the potential to address the business need for more rapid and flexible application development.  The impact on BI is significant.  ETL techniques will need to radically change as applications become plug-and-play.  On the positive side, SOA offers the best chance in twenty years to finally address the metadata problem in BI--simply because SOA can only work if complete, live metadata exist in the the operational environment.

It is my firm belief that these seven trends, when taken together, spell the end of our current way of thinking about and implementing Business Intelligence.  I'm not suggesting that BI is at the end of its useful life.  Far from it.  In business terms BI is growing dramatically in popularity and value delivered to business users, its influence has extended across the entire spectrum of business activities--from the board room all the way down to the factory floor--and exciting and profitable new uses are being constantly invented.  However, this success and ever-broadening remit has significant consequences.  

First, it becomes clear that the boundary assumptions and resulting structures that framed BI twenty years ago have become totally obsolete.  For example, what forward-looking business manager today would be satisfied with traditional BI month-end data when making daily operational decisions?  Or be willing to operate with only internally sourced data?  The rules have changed.  The required timeliness and breadth of information today are dramatically different than seen two decades ago when the BI architecture was emerging.

Second, BI today encroaches into areas traditionally covered by other IT disciplines.  Operational BI and collaborative BI are obvious examples of the blurring of boundaries that were once clearly demarcated in both technological and organizational terms.

My personal conclusion, about which I've written extensively elsewhere, is that we need a new architecture for BI that encompasses the entire gamut of IT support for business.  Essentially, we need to move our thinking up a level from separate operational, collaborative and BI architectures to a comprehensive, joined-up Enterprise IT Architecture that I call Business Integrated Insight (BI to the power of two).

Of course, I'm not alone in thinking about the future of BI in this time of dramatic change.  So, come along to the conferences in Brussels or Rome to hear what other leaders in the field think.

If there's one thing BI folks love more than playing with data, it's classifying it.  And one of the endless debates has been about the data, information, and knowledge taxonomy.  How we could simplify our lives if there was only one all-encompassing term!  Or maybe not...

I was teaching a Business Integrated Insight (BI²) seminar in Helsinki a couple of weeks ago and a TDWI member and Development Director of Visual Management Ltd, Vesa Tiirikainen, shared with me that in Finnish, there is only one word that covers data, information and knowledge and that word is "tieto".  So I asked myself: what if there was only one word in English?  Would our perception of IT change?  What would it mean for topics like Data Governance?

First, let's take a look at the traditional meanings of the English words.  From the Oxford English Dictionary, we have:
Data: (1) facts and statistics collected together for reference or analysis, and (2) the quantities, characters and symbols on which operations are performed by a computer
Information: facts provided or learned about something or someone
Knowledge: facts, information and skills acquired by a person through experience or education; the theoretical or practical understanding of a subject

The definitions are remarkably similar.  All are about facts, and interestingly, not about opinions, beliefs, etc. which seems a huge omission to me.  Data is a little more about how facts are manipulated.  Knowledge has more of a people orientation.  But, in general, not too helpful!

So allow me to propose perhaps more meaningful definitions.  Information, to me, is the starting point.  Information describes real-world objects in words, numbers, pictures and similar artifacts of communication in a way that is suitable for human cognition, use and processing.  It is inherently vague, context-dependent and open to interpretation.  I call it soft information.  Data is extracted from information by a process (modeling) of formalizing meaning and separating structure and meaning (metadata) from values.  This I call hard information.  Finally, knowledge is information that is understood and internalized by people so that it can be put to practical and innovative use.

On that basis, could we manage with a single word to cover all categories?  As BI practitioners, I suspect not.  Data governance and information management also demand such clarity, as I'm sure Mike Ferguson will touch upon in Rome on May 9-11.  Understanding these categories has been vital to me as I've created the BI² architecture, because they require fundamentally different processing methods, storage approaches, and so on.

But users?  From the business user's viewpoint, I believe we should use a single word - and let's choose "information".  In the business view, the underlying structure and metadata separation of data is actually unimportant.  And, increasingly, computers can store, manipulate and make available soft and hard information in a seamless fashion.  Knowledge, too, is being folded into the computer world through social media and networking approaches where the human interaction around a set of information becomes a key component of how information is used.

In the real world, the Finns probably have it right - one word should be sufficient.  But for BI and IT folks, we just need to make life more complex... but for good reason!

Following on from a previous discussion on the need for integrated information in modern business, and the focus on a consolidated process in my last post, I'd like to complete the picture here with a look at the role of people in the Business Intelligence of the future.

Traditional BI typically sees people in two roles.  First, they are the largely passive and individual receivers of information via reports or even dashboards.  Once that information is delivered, the classic BI tool bows out; even though the real value of the process comes only when the decision is made and the required action initiated.  Furthermore, the traditional BI process fails to link the action taken to results that can be measured in the real world, overlooking the vital concept of sense and respond described in my last post.  These two glaring omissions, among others, in the current BI world lead directly to the relatively poor proven return on investment in many BI projects.

Second, BI treats people as largely disconnected providers of "information requirements" to the BI development process, often leading to failed or, at best, disappointing BI solutions for real user needs.  Agile BI approaches do address this problem to some extent.  However, the real issue is that the process of innovative decision making is largely iterative with final information needs often differing radically from initial ideas on what the requirements may be.  The path from such innovative analyses to ongoing production reporting using the discovered metrics is also poorly understood and seldom delivered by most current BI tools.

The good news is that many of these issues are central to the techniques and tools of Web 2.0, social networking and related developments.  However, simply adding a chat facility or document sharing to an existing BI tool is insufficient.  To truly deliver BI for the People, we require a significant rethinking of the framework in which BI is developed and delivered.  This framework includes (1) a well-managed and bounded social environment where users can safely experiment with information and collaborate on their analyses, (2) support for peer review and promotion to production of analyses of wider or longer-term value, and (3) an adaptive, closed-loop environment where changes in the real-world can be directly linked to actions taken and thus to the value of the analyses performed.

Today's users are of a different generation to those BI has previously supported.  Gereration Y (born 1980-2000, or thereabouts, depending on which social scientist you follow) is the first generation to have grown up with pervasive electronic connectivity and collaboration in their personal lives.  They bring these expectations, as well as some very different social norms, into the business world, and are now beginning to assume positions of decision-making responsibility in their organizations.  They are set to demand radical changes in the way we make and support decisions in business.

I'll be discussing these issues at three seminars I'm presenting on the transformation of BI into Enterprise IT Integration in Europe: a half day each in Copenhagen and Helsinki (4 and 5 April) and a full two-day deep dive in Rome (11-12 April).  

In my last post, I mentioned the Business Information Resource (BIR) as the single, logically integrated store of all information--from soft to hard, internal and external, enterprise and personal--used by the business. The BIR is the foundation of the Business Integrated Insight architecture I propose as a new way of addressing the needs of modern decision makers; needs that extend far beyond the data typically stored in a traditional data warehouse environment. However, enterprise IT integration does not stop at the data, of course. It extends to the processes of the business, as well as to the people who comprise the enterprise as a whole.

As a result, the BI² architecture comprises of three components, arranged as layers from the top down: people, process and information. Each layer builds upon the one below it and is, in turn, influenced by the needs of the layer above. And just as the information layer challenges the current assumptions about how information is organized and used today, the other two layers raise similar challenges. So, let's think a little about the process layer, which I call the Business Function Assembly (BFA).

While BI departments and practitioners invariably focus on the information of the enterprise, my experience is that business people pay little attention to information in itself. They focus almost entirely on the activities they need to fulfill their responsibilities and the sequence and linkage between these tasks. In particular, today, one of their most pressing concerns is how to be highly adaptive to unanticipated changes in the market. I use Stephan Haeckel's Sense and Respond concepts to think about this need and have come to two game-changing conclusions about how we need to deal with process going forward.

First, applying sense and respond to the traditional division of IT into operational, informational and collaborative environments makes it clear that these divisions must be dismantled. Every activity, when considered from an adaptive enterprise viewpoint is described in exactly the same terms, has the same steps and has the same interactions with both people and information. The tripartite operational / informational / collaborative division is an old way of delivering IT solutions when computing was much less powerful and business much slower and less complex. Modern business needs and, especially, their integration can only be satisfied if we deal explicitly with their commonality.

Second, delivering highly flexible business processes requires that business people have direct input to and control over the processes they use. Up to now, IT has a separate and distinct set of activities (often called application development processes) that were designed to build IT systems support for business activities and processes. This division of work is defunct! All processes are really business processes and increasingly business people can and want to do them themselves. In BI, we've seen this first in spreadsheets, where users develop their own BI reports and analyses. On the internet, people increasingly "mash-up" their own applications; this approach is also becoming prevalent in the business arena.

The bottom line is that business process implementation and automation is undergoing radical change and IT departments and, in particular, BI practitioners need to urgently address how this change affects their systems and roles and what they must do about it. I'll be discussing these issues at three seminars I'm presenting on the transformation of BI into Enterprise IT Integration in Europe: a half day each in Copenhagen and Helsinki (4 and 5 April) and a full two-day deep dive in Rome (11-12 April).