So what? I feel that this is important because nCluster is available as a virtualized on-demand service. Without buying, installing, staffing and maintaining a data center, you can utilize cutting-edge database technology for sophisticated data analytics. If your BI group is the DIY kind, you can push the cutting edge even further. I hope that Aster will be innovative in marketing their offer through low-cost trial programs to introduce in-Database MapReduce (MR) technology to IT professionals.

An equally important item is buried on the second page. Aster also announced nPath, which is an SQL/MR extension that enables discovery of relationships between rows in a data set. For you SQL folks out there, remember writing SQL statements to compare one row with another, using UNIONs or subqueries or other ugly stuff. Now think of the SQL for time-series analysis. Now think of doing that in a (relatively) simple SQL statement that is efficient and scalable. I am studying this blog by Steve Wooledge for the technical details.

As a side note, Curt Monash has been tracking the MR influence for some time. I commend his DBMS2 blog, especially this posting on why MR matters to data warehousing. 

As a second side note, I discovered an interesting MapReduce counter-argument. Michael Stonebraker argued with Curt Monash that standard SQL-92 with User-Defined Functions (UDF) can do everything claimed by MR and hence MR is not needed. Does the MapReduce emperor wear any clothes?  Tasso Argyros, CTO and Founder of Aster, blogged in August about how MR takes UDFs to the next level. To say that there is controversy in our fair database community over MR is an understatement, and it started over a year ago. I recommend that BI/DW folks take note and understand the issues.

Impressive demo! This product takes slicing and dicing of massive amounts of data to a new level. It has nice features for surfacing the formats or templates of data values, like street addresses, zip codes, etc. It also can search phonetically and probabilistically, so you can find 'loose' matches on a search phrase. And so on...

I wonder, however, about who would buy this product... A data modeling or data quality specialist would find this tool very useful. It certainly fits the requirements for an exploratory data warehouse. But, duplicating the terabytes from an enterprise data warehouse into this engine would not be feasible. If you target smaller companies as the platform for their primary data warehouse, will these companies have the sophistication to realize the business value inherent in this tool?

Take a look. What is your opinion?

I ran across another great example of explaining the complex simply. Take a look at this 3-minute video on Social Media. It is about making ice cream in a small town. However, it explains a board range of Web 2.0 dynamics at the same time. Enjoy!

The first session of the Gold Consulting briefings was by Ambuj Goyal, GM of IBM Information Mgt Software, the blog of which is here. Through the week, I was struggling with this theme. How much substance does IA really contain? How will IA influence IBM’s strategies and practices?

Part of the answer lies in the corner of the exhibit hall. Did you see the IA exhibit? I would not, except that a colleague - Piyush Malik, an IBM GBS Consulting Executive serving as the Global Delivery Leader for IOD and Business Intelligence solutions - dragged me over to the booth. Having been involved with the design and staffing, Piyush gave me a personal in-depth tour, guiding me through the IA Experience that shows how this approach can transform information into a strategic asset.

The focus was on the activities of the Henson family - typical family with a teenage daughter Lily who is about to go off to college in Switzerland. The family is confronted with arranging bank financing and insurance before Lily's trip and purchasing the right in-style clothes for college. Unfortunately on the way to the airport, they were in a car accident. Lily is hospitalized, and her luggage is damaged. The family is urgently confronted with medical issues and insurance issues. Luckily, integrated medical systems exchange Lily's medical records between US and Swiss hospitals, and insurance data was instantly transferred to quickly repair the damages. The story ends happily. Lily graduates and becomes manager at IBM. :)

At each stage, the story describes the difficulties and satisfaction that the Hensons experience at the digital hands of complex information systems. Behaviors of today’s customers are complex and continuously evolving. Through approaches like IA, companies can identify the proper services for customers like the Henson family, integrate the relevant data across their data silos, and properly design their business processes to leverage that information. In particular, the linkages into the pie slice of IBM methodologies and products were explained. Remember that

The pitch. . . If your company is lacking in understanding customers, then shape up! Let IBM assist in developing an IA tailored to your unique requirements!

My take on this. . . It was a bit corny but definitely refreshing to see/understand specific ways that typical families are dependent on complex information systems. For me, it gave new meaning to CRM. In other words. . . Help me, just don't segment me!

The point is to understand all the ways that the Hensons are (could have been) served by intelligent systems that integrates data into a single version of business reality, enabling innovative business processes. Hmmmm Now I am sounding a bit corny!

[Blog stream from IBM IOD/Gold October 2008 is here]

Robert LeBlanc, GM of Software Sales, opens on “What’s Your Information Agenda?” We need to optimized enterprise performance. Whether you are CEO, CFO, LOB, CIO all have the responsibility to drive innovation. He Global enterprise CEO expects substantial change (83%) and change in their business model (98%). Technical talent will come from new emerging countries. Corporate social responsibility has a positive impact on society, by 68% of the CEOs.

Sidenote: A quote that stuck me was “We are surprised by the impact that just a few consumers can have with their blog entries.” by Ron Hurlbut, CEO of Attune Corp. Hmmmm Is anyone reading this stuff? If so, please post a comment!

Robert introduced Tom Davenport, Professor of IT & Management at Babson College, and his research studies on the interaction of information and decision making. His title is Linking Information and Decisions: The Unfinished Agenda. Tom noted that we have made fantastic progress in storing and managing data, but progress over 20 years on decision support has been lacking.

Tom (who likes pies) sliced the information pie: decisions, control communication, processes. Heading toward a planetary realignment via: application agenda, automating processes, creating data, key unit is the transaction. Types of decisions researched: supply chains, etc. The results from these studies have found: wide range of decision perspectives, factors of structure level versus human-contribution level, difficulties of achieving One Version of Truth.

Tom suggested a pyramid figure with top-down levels for: Automated Decision – Structured Human – Loosely Coupled. [I am not buying the framework; too much is all mixed up] From bottom-up, proper requires are: info infrastructure to linkage to DP/behaviors to tight process/system integration. From bottom-up, technologies relevant: DW/analysis tools to scorecards/dashboard to workflow/rules/scoring. IBM is developing or acquiring all these technologies. Four-step decision sequence (which I missed because of a text message). Striving toward a Single Version of the Truth (SVT) is very expensive to do; hence, we need to deal with multiple versions of the truth.

Next was a panel with Noel Garry, Irish Life & Permanent, and Larry Williams, JM Family Enterprises . Good sketches of business challenges and how their specific information infrastructures were being remolded to support better decisions in specific processes. Putting together all the information is a prerequisite to decision support.

Robert wrapped up the session. What a great era! Why IBM? Answer offered: Global IT leader, technology innovation, proven solutions, industry focus.

My take on this. . . I was expecting a deeper exploration into the linkage of information to decisions. In particular, we need better thinking about allocating human judgment (involvement) to business processes. Tom touches on this. I would asset that ALL automated business process MUST have include some degree of human judgment. Anyone agree?

[Blog stream from IBM IOD/Gold October 2008 is here]

The market for speech analytics is currently driven by application for: government intelligence, media analytics, and contact (call) centers. David concentrated on the contact centers, where the goals were: quality assurance monitoring, customer satisfaction, increasing sales, marketing effectiveness.

We had an interesting discussion of the quality (error rates) of speech mining. David shared a variety of speech sample to motivate the difficulties with accent, noisy backgrounds and the like. The technology for speech to text is getting closer to the quality of human transcription, which is in the 90% range. However, many commercial applications need only 50% accuracy, which allows processing at 5x real-time speed. In other words, 5 hours of call center recording can be processed in one hour.

My take on this. . . It is amazing the progress that speech translation technology has had in the last three years. We are only beginning to imagine the business potential of this technology.

[Blog stream from IBM IOD/Gold October 2008 is here]

Scott painted the overview of sensor technologies and their growth in global systems. Foci of using sensors are: Identification, Location, Condition, and Traceability. The flip side is doing the process innovation that properly reacts to sensor data. Current applications of real world are: out of stock, inventory mgt, asset visibility, work in process, dynamic asset mgt. and the like.

We need to create business processes that are aware of what is happening in the real world. Several customer examples were then discussed, such as Airbus applying RFID to transport and warehouse logistics for asset tracking and supply chain optimization. They are tracking huge airplane assembly components.

Daniel explained that sensor systems have adopted a standard called EPCIS (electronic product code information services). IBM products in the sensor area involve: WebSphere ESB, WebSphere Premises Server (collecting sensor data), bundled with WS Business Server, and InfoSphere Traceability Server. There is tight integration with Cognos with scorecards, dashboards, reports, and alerts.

I asked about the significance of the RFID tags on all of our conference badges. He estimated that it cost the conference organizer about 20 cents of which 5 cents is the processor.

My take on this. . . The explosion of sensor data will remold the DW landscape over the coming years. Operational BI and business process monitoring will become a deeper reflection of business reality.

A key point was that the second generation of sensors (such as, RFID and 2D barcodes) allows unique identification of each item. Hence, future DW analytics will be able to drill down from [women clothing] to [dresses] to [New England Fall Leaves Style, Color light brown, Size 6] to [Skirt #4W38Y295T012] hanging in my wife’s closet. Now, what is the business potential for this generation of sensor-enabled analytics? And, what are the ethical considerations?

[Blog stream from IBM IOD/Gold October 2008 is here]

Future data warehousing technologies will evolved around analytics enabled by: Map-Reduce, Data-in-Motion, High-Dimensional, and Relational.

Power and cooling is the fastest increasing cost component in data centers, as shown in this chart. IBM Balanced Warehouse is designed with Green in mind through EnergyScale technology to dynamically managed resources.

Solid State Disks (SSD) is coming, bringing 5x performance improvements, 75% reduction in rack space, and 89% power reduction. Cost of SSD is the primary issue but is steadily declining. Subsequent discussion of performance studies with advanced storage technologies was under NDA.

OLAP assets from Cognos and other acquired companies are rolling into three areas:

- Cognos PowerCube as Transformer for general BI
- Cognos TM1 as Turbo Integrator for optimized write-back and high volatitity
- InfoSphere Warehouse Cubing Services as Design Studio for large datasets with large dimensions

[Blog stream from IBM IOD/Gold October 2008 is here]

Big three screen flash video with very hard bass-driven sounds and lasers darting around. What is the point, other than to wake us up? Blue robots dancing (Phoenix Fire). Is this the IBM sales team? Star wars! I could lose my hearing after this! What is the message? I think that this is considered entertainment. Oh, now a rock band (LA All Stars), louder than the dance routine. Now regretting this VIP seat in front! FREEDOM! . . .from what? Martin Short, a seasoned IT professional - NOT! Hey, he knows how to spell IBM and IOD. Does IOD really mean Injured on Duty? He is funny. This intro flash was entertaining but so far, the first 22 minutes for 7,000 attendees is content free.

Ambuj Goyal, GM of Info Mgt Software, started with IOD is Information-Obsession-Dedication. Locking in the value of information. In the past, the answer to all business problems has been “you need a data warehouse”. Ovum stated, "IBM has deliberately and doggedly constructed an unparalleled portfolio of software." It is about: consistent client success, accelerated value delivery, and industry-based value.

Steve Mills, Senior VP and group executive SWG, noted an interesting fact: If you printed out all the information in the world, it would weigh 6 tons for every person in the world, about an elephant for each person. What is the name of this decade of IT? It is a era of business optimization. It is all about Information Agenda (IA). Steve mused about whether, if financial institutions have had effective analytics, they might have avoided this financial crisis. We are moving from business automation (5% CGR) to business optimization (11%). Investing in diverse portfolio of software tools and systems. Things that were not economic are now quite economic. Your key technologies are: time series/cluster/pattern-mining, linear programming/scheduling, natural language test analysis, entity analytics, social analytics, and streaming analytics.

Martin Short interlude. . . He did the three amigos routine, for which three random IT folks were dragged onto the stage. Ouch!

John Iwata, SVP of Marketing/Communications, outline the perception of senior executives. 80% anticipate turbulent change, but 39% feel unprepared. The world is increasingly more instrumented, interconnected, and therefore intelligent. Really? By 2010 there will be one billion transistors per human, for no cost. In two years, there will be 30 billion RFID tags. Businesses in the world are operating in a dumb way. Message: IBM is enabling businesses throughout the world to operate smarter.

Panel of three IT executives: Shay Reid of AmerisourceBergen, Peter Keukert of Commerce Bank, Karmal Bherwani of NYC. Snapshots of innovative technology innovations in pharma, city government, and banking. Final comment from Peter about the global financial crisis was concise and challenging. We need to have the courage to ask the right questions, to have the right information, and to take the right action.

Martin Short back again. . . A hot BBQ of political candidates. At least, he dishes out bards evenly.

Ambuj circled back and summarized the key points of the three panelists. [Good style!] The HHS-Connect system is a single view of NYC services from a citizen perspective. They started with regulatory compliance, the investment was extended to citizen services. In a large global bank, technology can get into trouble if each project is evaluated and funded on the value of individual projects. You can not leverage the synergism. Not enable for reuse. Avoiding the rip-and-replace strategy.

Video from several IBM executives giving sketches of issues and solutions: Rob Ashe, Ron Ercanbrack, Jim Welch, Arvind Krishna. Technologies were viewed individually. But we are shifted to a holistic view. New announcements on InfoSphere tools, Dognos 8, InfoSphere streams, enterprise content Manager.

American Express receives the customer innovation award. enChoice receives the CTO Innovation Award. Sky Solution receives the Best of IOD Showcase.

Yet another Martin Short interlude. . . We are too marvelous for words! Finis

Afternote #1: RFID is working in masse. I wonder how data mining will be used by IBM on this data. There is an opt-out in the fine print of conference materials.

Afternote #2: A nice surprise was a 66-page pamphlet passed out as we exited: IOD For Dummies by Judith Hurwitz and colleagues at Hurwitz & Associates. It is a tangible take-away. It is too bad that its content was not emphasized in the keynote.

My take on this. . . Business-oriented message to a technical audience, delivered with low signal-to-noise ratio. I am bias having heard the message previously in the Gold sessions.

The basic problem is that this audience is mainly people who are piecemeal specialists in technology hearing a message calling for them to be holistic generalists in business. They may be sympathetic to the message but totally unable to embrace it. Unfortunately, the keynote may be perceived by them as two hours of entertainment delaying the start of the good technical stuff.

[Blog stream from IBM IOD/Gold October 2008 is here]

This week I am getting a glimpse inside the new social mechanisms driving global technology evolution. All of this is scary, especially for the old-line control-oriented IT managers. I think that, for IBM and other leading IT vendors, we are moving from a strategy of stick-to-the-wall to a full food fight! In the video, is IBM like Robin Williams?

Yesterday I heard a talk on the Mashup Center by Guylaine Cantin. My one criticism is that there are several other mashup products (from Lotus and WebSphere) from IBM. Further, if you add up all the various country initiatives, professional services engagements, and research projects, there are probably 20 mashup efforts within IBM. How is a poor customer to know which mashup direction to go? You certainly do not want to get smash with mashups in your enterprise architecture! (sorry, had to write that!)

Late in the afternoon I heard about the Data Champion program and the XML Challenge. That challenge is either the smartest or dumbest program. Now, I am leaning toward smart. What a way (at a low low cost) to marshal 47,000 young professionals in India to learn XML. . .the IBM way!

Smart is what smart does. An innovative and viable company in our turbulent global economy has to discard lots of old paradigms. Like Robin Williams, maybe IBM is finding its inner Peter Pan.

[Blog stream from IBM IOD/Gold October 2008 is here]