2008 was a challenging year for nearly everyone, especially those caught up in the economic downturn. My teams and I still helped over 25 companies improve their business and their use of information and thousands improve their skills and contributions. My goal for 2009 is to double the impact.

Finally, I must say Thank You, Thank You, Thank You! Without clients, friends, business partners and people who read and engage like YOU, there would be no practice. I am eternally grateful for your support and referrals. Happy New Year.

I’m using their hardware and paying for it as I use it. This is like a virtual private server and also similar to a SaaS offering. The major difference, from a user perspective, is the pay-by-use model as opposed to harnessing a fixed amount per month. With cloud, you also get access to a wide variety of hardware without having gone through a major evaluation process or having made a major commitment.

We are already seeing some Information Management vendor ports to cloud, including Vertica and Panorama Software. If this takes off, as for information management, what will it mean? Can somebody say “playground”?

1. Technology will enter into a much tighter cycle of adoption in the marketplace, similar to how songs are now instantly and volume rated on iTunes and the artist can almost calculate their lifetime earnings from the first weekend of release.
2. Enterprise data warehouses will take another blow in Fortune-level accounts as the difficulty of independent data mart deployment drops like the stock market. This may ultimately be good for business analytics if those organizations operate under healthy standards and have master data in place for bringing into their cloud operations.
3. SMBs will follow with quick deployments of their own. However, for SMBs, this may actually mean the enterprise data warehouse, long seen as out of reach.
4. More of these methods of deployment that are alternative to the onsite hosting method will emerge and will integrate their offerings in the cloud. Specifically, I see SaaS offerings taking advantage of the cloud.

Cloud computing shows some real promise for making analytics transparent to and integrated with organizational behavior. In what I expect to be a “flight to value” year of 2009, it is difficult to say if organizations will spend much time in evaluation or much adopting of what little there is available now in the cloud, but in 2010 and beyond, we may begin to see some of the promise of cloud computing realized in information management.

Technorati tags: cloud, Business Intelligence, cloud computing, Information Management,EC2

It is really difficult to tell the depth of the deleveraging and decoupling that the world economy will go through. The economy is wound up pretty tight and must let out the built up pressure. Questions remain about the approach and the timing, but there is no avoiding that pain has, and will, occur.

One point I made is that financial companies were motivated to get mortgages out the door and that they sold their toxicity. This was true, but why were they motivated as such? Some point to the Community Reinvestment Act of 1977, which required institutions to loan to those less qualified. 1994’s Riegle-Neal act compounded the CRA’s effect by rewarding banks with high CRA scores to bank across state lines. And then more ability to compound behavior was possible in 1999 with Gramm-Leach-Bliley, which allowed banks to combine investment and commercial operations.

There was also incentive to take undue risks with the dilution of executive accountability once the firms went public and the executives became more minority interests in the entities. This started with Salomon Brothers, important in Citi’s heritage, going public in 1981. While I’m at it, the rating agencies’ presentation of their business intelligence left some things to be desired. And over 100% home equity loans, combined with a real estate downturn, tossed more toxicity on the fire.

Another point is that the mortgages were put into complex packaging, which business intelligence did not keep up with. So, in context of business intelligence, did the financial companies know what they were buying? I think business intelligence has some room to grow in terms of that, as pointed out in the article. A better question may be did they care? In some respects they did, but in other respects business intelligence was relegated to secondary consideration given that the institutions were not incented purely by profitability and good business. As I said “full visibility into exposure and liquidity is going to be a must.” Visibility and rewarding only good business are part of the “executive sponsorship” I mention that is required.

I had an MBA professor who went through some of the early lineage above with his students and predicted a dire outcome. I took his notes (early 1990’s) and extrapolated the more recent events for this entry. Many probably could have seen this coming, but when times were going well, nobody wants to stop the music. Executive sponsorship and business intelligence will be critical to mend the markets as painlessly as possible.

What are your thoughts?

Technorati tags: data, Business Intelligence, financial crisis, Information Management,Community Reinvestment Act, Gramm Leach Bliley

So, to make them easier to find I thought I'd put them here. These also comprise a fair overview of business intelligence related topics.

White Papers:

Data Mart Consolidation: Repenting for Sins of the Past
Modernizing and Advancing Information Management Across the Enterprise
Overall Approach to Data Quality ROI

Articles (to find them, google "McKnight" plus the name of the article):

Justifying and Implementing Master Data Management
BI Business Value: Does it Come from the Program or the Projects?
Supporting KPIs with the Data Warehouse
The Integration of Data Warehouse Data with Campaign Management

Technorati tags: data, Business Intelligence, Data Warehouse, Information Management

Observations from the field will continue in the blog.

Technorati tags: DAMA, Syndicated Data

Anyway, this session was by Pat Wilson at Disney, from just down the street. She’s on the data warehouse team. The premise was that the approach to gathering requirements for a data project is different from other projects. Like many companies using analytics, Disney makes hotel offers based on your answers to their questions with the offers that are most likely to succeed.

She went through some BI basics and made a good point about not collecting data for which you were unaware of the eventual usage and to avoid the “kitchen sink” requirement which goes like “just give us everything, we’ll use what we need.” Also avoid the “mind reader” approach which goes like “just give us what we want, we’ll know what to do with it.” Also avoid the “trial and error” approach which goes like “we’ll know it’s right when we see it.”

I think this presentation could correlate each scenario above to a Disney character. May I suggest, respectively, Goofy, Evil Queen, and Sleepy.

The (great) overall point is get DEEPER with your requirements for these projects and don’ t accept the shallow “requirements” the business tries to get away with. It’s a recipe for failure. She recommended an approach that follows along with this model:

I definitely agree with everything said here. It tracked pretty closely to my "ROI and Focus" blog in terms of why the data warehouse team exists and my article on Requirements Gathering (search Google for “McKnight Data Warehouse Requirements Analysis”).

Technorati tags: Business rules, Business Rules Forum, Business Requirements

What does business DNA look like? Process is heart and soul of how things get done. Then, decisions go into those processes. Similar to how we’ve evolved, you can change DNA without becoming a new business.

Process describes the how of the core activities of the enterprise. A decision determines the what of enterprise activity and it’s typically automatable or human, but not a mixture.

Automated decisions are ubiquitous (i.e., commissions, cross-selling, fraud.) The talk then focused on how to automate a decision intelligently. Sometimes, it’s a decision table, sometimes a rule flow (flowchart.) How to turn analysis into rules: model the landscape, understand the business goal, and formulate and formalize the solution. The last step is where your business rule management system (BRMS like ILOG, Fair Isaac) comes in. A value-added step at that point is to add simulation on historical data, perhaps in your data warehouse. Then, analyze the simulated outcome.

Then, Steve talked about numerically characterizing history to evaluate those outcomes and predict the best solution to take. It’s about predictability and likelihood. For example, in a group of transactions, how many will be fraudulent or how many will take-up the cross-sell offer?

Then, Steve talked about planning and scheduling with BRMS and how it can create a (for example) optimized nurse schedule for a hospital and deal with the inevitable last-minute decisions that must occur based on last-minute no-shows. This is an example of creating the adaptive enterprise – one that adapts to business changes. However, to get there, we need to break down the hedgerows between business departments, specifically IT and business groups.

Finally, 80% of a business’ problems are about being better at what you do. Business rules can help. 20% of the problems are about being something different than what you are. Both this “adaptation” and “creation” (potentially “destruction”) are necessary.

Technorati tags: Business rules, Business Rules Forum

I pointed out that the important thing about these committees is that all of the necessary information management functions for the organization are done. These committees normally comprise data governance, data stewardship, program governance and a business intelligence competency center (or center of excellence.) I do not wish to overdo committees at my clients, but want to be sure all of the required functions for success are being done.

The benefits Carolina cited for their Analytical MDM implementation were:
1. Data management is aligned with the company strategy
2. Operational systems (by product) supports reporting and compliance
3. The hub allows the single customer master to be shared to all product systems
4. Early data issues detection
5. They know their complete exposure to clients, whereas before it was piecemeal and incomplete
6. Reconciliation of transformed data to GL metrics
7. Managers consuming information and providing constant feedback for improvements
8. A unified customer view... for all its other benefits

I generalized from many MDM implementations and presented "Top 10 Mistakes Companies Make in Forming Data Governance." They are (in no particular order):
1. Not Translating IT Investments into Business Objectives
2. Thinking of it as a Technical Function
3. Scope Creep
4. A Revolving Door of Membership and Participation
5. No Decision Maker
6. Failure to Create a Charter
7. Turning Governance into the Blame Game
8. Lack of Customization to the Culture
9. Thinking of it as "just meetings"
10. Hyperfocus on a tactical issue

Technorati tags: Master Data Management, Business Intelligence, MDM Summit

1. Knowing BI & MDM go hand-in-hand
2. Focusing on MDM when combining 3 large organizations to formRR Donnelley
3. They didn't pick the technology first, but grew into it
4. Somebody there had the wisdom to declare early that MDM must be minimally invasive to the source systems, and it was something RR Donnelley followed
5. They used D&B DUNS number for identifying (B2B) customers
6. They built in capability for (what I call) master data query
7. Data governance and stewardship

They use the Registry model for MDM.

The "ROI" from the effort was in sales reporting, reduced manual work in reviewing customer names, and knowing their exposure to companies who were/are potentially going under in the challenging economy.

The last best practice was to use outside implementation services. I know of one that can help there.

Technorati tags: Master Data Management, merger, Business Intelligence, MDM Summit

Since 85% of data is unstructured data, we are making decisions on 15% of data. Search is the backbone of access, but it’s only one way we find information. We also categorize, cluster, and link (if you like a, you’ll like b). She made the case that most searches are small (i.e., few predicates) and query revision is the norm. I think she means queries issued by people, not queries in production, which have tended to get large over time in development and can tend to be quite large (and frequently run.) However, the point that we want our data back fast, whether we’re IT people or business people, and without writing long queries is taken. I.e., select gross-profit, not select . Taking this a step further, I am continually pushing in my consulting for deriving more data in the back end and making it easy on the end user. Easy=they use it, Hard=they don’t.

Back to Julie, she said search is not the answer, but language may be. However, we need to get ambiguity out of language – and there’s plenty of that. I.e., “light brown dog”, “pressing a suit”, “the chicken is ready to eat”, “he saw her duck”. Then there’s similes, metaphors and idioms in language. We need text mining and NLP (natural language processing not neuro-linguistic programming) is the most effective. We decomposed some sentences like the tools do.

Adding unstructured makes it not the “single version of the truth”, it’s the “single version of the whole truth.”

Julie went into “voice of the customer” – mining call logs, for example to determine “unhappy” sentiment was due to the fact that beneficiaries are not getting the support they needed. Regarding VOC, what if call records were not structured? Search (i.e., for the word “billing”) would not suffice for analysis. You really must structure the data to see trends and norms.
She showed a dashboard from Claraview of some analysis of hotel sentiment from tripadvisor.com, replete with categorization of sentiment and drill-through to the detailed feedback. i.e, “This was one of the worst hotels I have stayed at. The room heater made deafening noises all night long. The staff at the hotel was all extremely rude and condescending.”

Example used were from products Attensity and Clarabridge.

In closing, there was some discussion of mining abbreviations. One language that will be a real challenge to mine is the one we heard Tuesday night at Circque de Solei’s “Ka” show – Grammelot (yet another new word tangentially credited to Partners.)

A couple of classic Teradata case studies and a panel of grumpy old men (their title) - where we were asked not to journal it - later and it was time to leave Partners. As I left Partners, what I’ll remember beyond the conference is the high quality of the Teradata team – both Teradata employees and the community of professionals. That is certainly one of Teradata’s biggest assets.

Technorati tags: Teradata, Teradata Partners, Unstructured Data

1. Any weaknesses you have in your BI will only get worse when you speed it up
2. Don’t make (or think you’ll make) big changes to operational processes – that’s done a bit at a time

Technorati tags: Teradata, Teradata Partners

Use cases presented were in insurance (customer density) and communications (find customers near stores or in a marketing area.)

Teradata 14 will add the Raster data type for imagery and geocoding.

Overall, I think this is a nice addition to Teradata’s already strong feature/function set. It will be very important to some and ignored by others.

Technorati tags: Teradata, Teradata Partners, spatial,data warehouse

I spoke with Deb Hoefer of Teradata, who is spearheading a training program for Teradata business users. For example, one course is Teradata SQL for Business Users. I definitely agree they need training and we spent some time talking about how they can customize a program – something obviously needed. I think this is a great idea as user communities are clamoring for empowerment! This topic actually has been a top three requirement in the strategies that I have been conducting lately for clients.

I also spoke with Ray Wilson. We talked about Teradata’s programs for anti-money laundering and meeting the regulations for uncovering the concealment of sources of income.

Finally, I had a podcast with Alok Pareek, the CTO at Golden Gate. We talked about their technology (they did some co-development with Teradata), but primarily we talked about dual-active. It was a good quick journey into the value of dual-active, some of them in ways not so obvious, as in environments where there can be zero down time or risk for an ERP upgrade. By using dual-active, some companies are running multiple versions of their ERP against the SAME data, with only limited pressure on the end users to get on the more recent version. I mention at the end that upgrades are one of the most challenging things IT does and better ways than the high-risk weekend war rooms are needed.

Enjoy all the Partners podcasts.

Technorati tags: Teradata, Teradata Partners, business intelligence,data warehouse

Some of the highlights:

IT doesn’t need to communicate to the business, they need to work with the business. Nicely put!

By 2012, users will interact with BI analytics as an element of greater than 85% of every business application. This BI will need the DW.

Half of BI and IM initiatives will fail. How will they fail if they’re already successful? There will be different measurements going forward. IT doesn’t listen and ignores the need for change.

There is a lack of vision for BI. Creating strategy is not fun so it doesn’t get done. I note that's what I do and I think I think it’s fun.

DW/BI is the #1 technical priority for CIOs and IT for 4 years straight now with a 9.2% growth estimate in 2008.

Every student graduating from college today has knowledge of IT and ability to get their own data. This will lead to Excel and Access proliferation and data quality problems if we’re not careful. It will also lead to a rise in SaaS and DW in the cloud.

The DW is mission critical!

Linux revenue growth from 2006 to 2007 was 61.8% and Unix was negative. He stated Linux will be able to handle any workload.

DW appliances are getting popular (several available now from Teradata) because nobody wants to lose their job due to a misconfiguration of a machine and there is one vendor to call with problems.

Technorati tags: Teradata, Teradata Partners, business intelligence,data warehouse

Good luck in France Lance and great job Partners for bringing us Lance.

Technorati tags: Teradata, Teradata Partners, Lance Armstrong