In the event, Hearst's paper is somewhat futuristic.  Nonetheless, he makes some fascinating observations about how users' interaction with computers is undergoing significant change.  The first trend is a move towards voice interaction, driven in particular by smart phones.  There have been dramatic improvements in voice recognition over the past few years, and the ability of software to recognize and interpret simple commands allows users to speak more naturally.  This is driving a move away from single keyword or key phrase searches, even when typed into Google, for example.  However, more work is needed to enable reliable identification of context and the sequential inquiry approach often favored by users.  Natural language-like query has been a topic of ongoing interest in BI, and NeutrinoBI's product supports this in its query interface.  

Hearst also notes the growing importance of social search, in collaboration, crowdsourcing and basic social communication.  This is a topic I'm currently developing in a new White Paper with Lyzasoft and to which I'll return in a future post.

The final point that Hearst makes is on the emergence of video as a means of communication, replacing text-based approaches among some younger users of the Web.  This poses some significant challenges for both search and analytics in the future.  In recent years, text analytics has become an important tool for sentiment analysis and so on.  "Video analytics" is still in its infancy.

Returning to NeutrinoBI, let's take a look at the exciting concept of freeform search.  Freeform search, in one sentence, enables Google-like search of highly-structured information as is found in data warehouses and data marts.  The secret of moving from keyword search of content to freeform search lies in understanding and representing the structure of relational data in a way that supports intelligent parsing of free text searches.  This structure originates in data modeling and is carried into relational database design and associated metadata--which columns occur in which tables, identification of primary and secondary keys and foreign-key relationships between tables.  Structuring, whether into normalized or multidimensional schemata, also constrains allowable values in certain columns and relationships between them.

The result is a conceptual hierarchical structure that is hardwired in the database or application in the multidimensional approach.  For example, geographical location is often expressed in the form of regions, which are comprised of countries, which are further comprised of states / provinces, which break down to counties, each of which contains a known and limited set of values.  In the case of freeform search, a process known as hierarchical value decomposition is used to automatically analyze database structures and create the indexes and metadata--the information context, unique and specific to the structure and content of the underlying data--needed to extract meaning from the key words entered by the user at search time.  The result is a system that can be easily used by business people in their own terminology and can be constructed rapidly and efficiently by IT.  Further details can be found in my White Paper "Freedom from Facets: Discovering the data you really need".

On the Web, a search interface is already the norm.  As Hearst describes, it is evolving into something closer to the way humans interact.  What we see on the Web usually transfers into the enterprise environment.  The function that  NeutrinoBI is already delivering shows how these developments can be applied to BI and, in the process, is beginning to provide business users with a powerful, yet simple, way to get the information they need.

The napkin picture led me to take a look at the characteristics of data in the light of the rapid, ongoing change in the volumes, varieties and velocity we're seeing in the context of Big Data.  A survey of data-centric sources of information revealed almost thirty data characteristics considered interesting by different experts.  Such a list is too cumbersome to use and I narrowed it down based on two criteria.  First was the practical usefulness of the characteristic: how does the trait help IT make decisions on how to store, manage and use such data?  What can users expect of this data based on its traits?  Second, can the trait actually be measured?

The outcome was seven fundamental traits of data structure, composition and use that enable IT professionals to examine existing and new data sources and respond to the opportunities and challenges posed by new business demands and novel technological advances.  These traits can help answer fundamental questions about how and where data should be stored and how it should be protected.  And they suggest how it can be securely made available to business users in a timely manner.

So what is the "Data Equalizer"?  It's a tool that graphically portrays the overall tone and character of a dataset, IT professionals can quickly evaluate the data management needs of a specific set of data.  More generally, it clarifies how technologies such as relational databases and Hadoop, for example, can be positioned relative to one another and how the data warehouse is likely to evolve as the central integrating hub in a heterogeneous, distributed and expanding data environment.

Understanding the fundamental characteristics of data today is becoming an essential first step in defining a data architecture and building an appropriate data store.  The emerging architecture for data is almost certainly heterogeneous and distributed.  There is simply too large a volume and too wide a variety to insist that it all must be copied into a single format or store.  The long-standing default decision--a relational database--may not always be appropriate for every application or decision-support need in the face of these surging data volumes and growing variety of data sources.  The challenge for the evolving data warehouse will be to ensure that we retain a core set of information to ensure homogeneous and integrated business usage.  For this core business information, the relational model will remain central and likely mandatory; it is the only approach that has the theoretical and practical schema needed to link such core data to other stores.

"Seven Faces of Data - Rethinking data's basic characteristics" - new White Paper by Dr. Barry Devlin (sponsored by Teradata)