These worlds I'm talking about are the post-operationally-minded information management professionals and the operational-systems-minded professionals.

On the one hand, MDM does comprise quite a bit of traditional information management - the data modeling, the data quality programs, the data profiling, the data integration - all familiar to builders of data warehousing. 

On the other hand, MDM is operational and will interact in real-time, not batch, with other operational systems.  MDM will utilize governed workflows to bring together its data and will need to be keenly aware of operational issues and operational data needs.  Changing the data so suit analytics is not always best.

While the perspectives are not mutually exclusive, the skill gap to what is needed can be astounding if teams are formed, and therefore opinions of MDM formed, exclusively from one camp or the other.  Make sure your MDM program achieves the right balance of skills and focus.  Take inventory.  This can be done through consulting and education, as well as conscious effort towards balance and achievement in both operational and post-operational issues.

• Enterprise Data Model using relatively acceptable best-practices and standards
• Views for the Business Model metadata
• ETL framework containing mappings/routines to load and refresh the data warehouse
• 100s of reports and dashboard content
• ETL routines have built-in Change Data Capture logic
• Open Interfaces to  other data sources
• Industry standard Metadata for Common Business Functions (i.e. AP, AR, GL, Sales Forecasting, Pipeline, etc.)
• Faster Time to some value
• Small team to deploy initially
• New features available via upgrades

It IS the fastest route to SOME value.  Is it the right value for you?  Is the time savings worth the extra cost?  Will it pay off in the long run?  It's not something to get into lightly.  Unfortunately, packaged data warehousing tends to come with many misconceptions that make it seem more attractive.  These are the most common:

1.      It's a total plug-and-play

2. Data modeling is not important
3. The client does not have to staff data warehouse expertise
4. The business does not have to be involved
5. Data quality is not important
6. Your individual enhancements will merge seamlessly with vendor enhancements in their releases
7. The vendor will continue to upgrade its solution in real-time with the corresponding source system
8. There are no other costs, i.e., Hardware costs, Employee costs, Consulting costs

Be sure to take a complete and multi-faceted view of this important decision.

I'll do my best, but one thing that struck me as interesting as I reviewed was that the future of data management will have many opportunities as well as confusion in it.  Unlike about 5 years ago, the enterprise data warehouse was not only hitting stride, it was considered a "holy grail" and an end-game.  Many data management resources went into building data warehouses (with single platforms, strictly row-based structures, on standard hardware and in-house.)

This is not so true any more.  We've re-entered chaos.  A columnar orientation is working its way into enterprise architectures and popular database offerings (Microsoft, Oracle).  You need to know how it works and when to use it.  Appliances and on-demand services are pushing function out of the enterprise.  You need to know how to manage the changing roles and responsibilities.  With departmental budgets buying software-as-a-service business intelligence, low-end appliances, and open source BI tools, shops need to have integration capabilities now more than ever.  And, something I've been saying for a while, master data management is leading the charge of business intelligence back into the operational world.

The economies and performance abilities of data management are changing and the benefits need to go to your bottom line.  Most unhelpful is the terminology wars surrounding all of this.

And finally what to start doing about the MapReduce approach, and syndicated data, and the data cloud.... 

Sorry, no answers here, but sometimes it's helpful to identify the questions.

Like many of SQL Server's R2 releases, it builds on its corresponding R1.  SQL Server 2008 been commercially available since mid 2008.  From a data warehousing perspective, SQL Server has long been a choice for data marts, regardless of the data warehouse platform.  It has also been a data warehouse platform for the midmarket and occasionally a Fortune 100 company.  Some of the scalability concerns that have limited SQL Server's reach may be being answered in R2 with the added support for 256 processors.  This is quite a move up from 64 processors.  Also improving scalability will be the acquired DataAllegro technology, rebranded Parallel Data Warehouse, Microsoft's data warehouse appliance.

What I am most interested in and excited about is Master Data Services, Microsoft's entry into the crowding master data management market.  Microsoft is the first to make it a part of the DBMS package.  They are clearly targeting the Microsoft shops that are having master data issues.  I've had the CTP for some time (this is a worked over form of the product from Stratature, which Microsoft acquired in 2007) and have been able to exercise it and even see it implemented at a client.  While its capabilities are limited compared to its more mature competition, it has a lot of potential. Microsoft will be putting strong development effort behind Master Data Services.  Even today, it can certainly, with some effort, play the technical role in a MDM program.

And then there's Gemini or PowerPivot for Excel. Gemini is the new generation of Analysis Services.  Those who chagrin at the notion that Microsoft Excel is the #1 business intelligence tool (it is) will have a lot more concern now.  Bye-bye ProClarity interface.  We must embrace Excel.  I'm increasingly crafting procedures for IT's role with Excel and this need will only increase as Gemini will cause an even more fluid spreadsheet environment in shops.  Data security strategies are imperative. 

Gemini will also be a collaborative environment.  Everyone in a workgroup can cooperate in managing Excel.  Excel is certainly already mission critical and R2 will create even more possibilities to depend on Excel.  The Gemini server is SharePoint, another element gaining traction in the SQL Server family.

I am also completely impressed with the addition of the columnar, in-memory storage option for this downloaded data, called VeriPaq.  Some data just belongs in columnar, though most would not want to put all their data in this structure.  It's great for data where a lot of columnar functions will be done to it, as well as generally for those queries that don't return a lot of columns.  It's also great for compressing data.  I have a seminar on columnar and expect to be helping more clients effectively tier their data to this format now that SQL Server will be utilizing the option.

So, are you ready?  Are you on SQL Server 2008?  Are you ready to upgrade to Office 2010 to take advantage of Gemini?  SQL Server 2008 R2 will be one of the big BI stories of 2010.

Remember the first Inmon definition of data warehousing? "a subject-oriented, integrated, non-volatile, time-variant collection of data, organized to support management needs."  It's a solid definition that has stood the test of time.  I know it's for data warehousing, not business intelligence, but you can see the mentality of the management user there and no mention of the rest of the organization.  Of course, BI has all kinds of users - people and systems - now.  And organizations are better off for it.

However, the so-called Pervasive BI movement goes beyond any of this.  It's "BI for everyone."  Unlike my thoughts on self-service BI, I am less optimistic about pervasive BI in the short term.  I'm all for smarter organizations, but not many workforces are structured such that everyone, or even most, employees have impressive key business decisions to make that incorporate the need for interaction with data well beyond their immediate environment.  I can think of a few exceptions - financial research firms, pharmaceutical research, etc. 

Probably the key thing to find palatable ground here is to define user.  Everyone in an organization can be the beneficiary of BI in the organization.  However, the tiered nature of the user community will remain true for the next decade.  Some will:

1.       Mine the data

2.       Interact with the data

3.       Consume and interact with reports

4.       Consume reports for decision making

5.       Make tactical decisions for others based on information seen in a report

6.       Make small adjustments to their team's workdays based on information seen in a report

7.       Make tactical decisions based on information seen in a report

8.       Make small adjustments to their workday based on information seen in a report

At some point in this hierarchy, the need for actual BI tools stop.  So, if you have a project that will entail a large number of new BI users - and they're going to productively benefit the business through the knowledge they gain from being a user - that is great.  If you are displaying corporate KPIs throughout the organization, I can see that.  If the culture that arises from this sharing, and possibly decomposing the KPIs to multiple levels, is empowering everyone to do their best job, that too is great.  But that's not BI for everyone.

Let your organization benefit from BI.  However, a project to blanket the business with BI tools in an untargeted fashion because it is thought that [pervasive BI is good, it means everyone needs BI and BI means tools] is not the best use of resources.

As time passes, many of these reports will lose their value to the requestor.  They won't break off the request, however, because it's not their time that's invested in building the report and the report MAY still have some value anyway.  Whether the decision maker moved on without it or not, she will have to sort through some voluminous data to get to some value, and that's also time consuming and value-reducing.

So it goes in many environments that have not made the leap to empowering the end user.  Yet, it seems every shop sees the value in self-service BI and minimizing or eliminating the IT bottleneck, yet few have made it happen.  So should we accept this reality and down-license the business intelligence software we're buying and forsake user empowerment?  No way.

Self-service BI takes work.  It takes focused energy until an organization makes some breakthroughs in process and the culture change necessary.  This is important enough that if most users are dependent on IT, this is probably one of the most important strategic activities to work on for the BI program, and the business as a whole.  So why the delay?

Change is hard.  Even small investments in changing habits are hard to do.  Despite comments to the contrary, using IT and treating BI as "just reports" is the devil many know.  Here are some tips (directed at IT) for getting past this point.  Remember it is IT that bears the brunt of the failure of user empowerment.

1.       Promote the data that is available.  Sure, BI is built now based on known requirements.  However,  there are still those unknown value propositions for the data that can be unearthed quickly and provide more value than the requirements for which BI was built for.  Users are often reluctant to articulate requirements if the turnaround is multi-months, but if the data is already accessible, well-performing and clean, usage ideas will take form.  More usage, and getting  will mean more user profiles.

2.       Promote the various methods of accessing data.  Early in a BI cycle, I will profile the users to the style of access that is most suitable for them and passive report receiving is not the default.  Dashboards are a step up from reporting and ad-hoc, interactive environments  are beyond that.  Furthermore, it may be possible to automatically enact the business change that is desired through operational business intelligence.   Put it all on the table with the users.

3.       Invest in training the users to interact with the data.  Real training - classrooms, materials, breaks, snacks, etc.  Half technical-general, half company/data model-specific.  The first few trainings will be learning experiences for everyone involved.  There may be serious feedback or some serious deskside component to the training until it is learned what works to make the user feel empowered.  And, while the training is serious, it can't be LONG, boring or irrelevant.

4.       Invest early extra energy in a few key users.  Invest in them the notion of the iterative nature of BI and other things they need to know and get them sharing the good word throughout their peers.  Self-service BI is iterative like everything else.  Start small.

5.       Don't, for a second, believe it's an all-or-nothing proposition.  This is where many shops go wrong.  They think they can flip a switch and users will do it all.  There should be a healthy split of responsibilities between IT and the users for data access.  There is still a data access role in IT in self-service BI environments.

I think of MDM as a term to address to a number of real business problems.  It has to do with data quality, sharing data across systems, an enterprise data model, and governance processes and workflows as well as sometimes being a direct data layer to a data-intensive application like CRM.  It's not a product although some products bearing the label provide these benefits.

So, why do MDM?  If and when it alleviates real business pain, solves business problems and enables new ROI-producing functionality.  MDM happens to be a discipline sitting squarely in the sweet spot of modern competitive advantages.

In addition to the price drop, the upper limit has been expanded to, depending on who you speak with, 700 terabytes or 1 petabyte.  Either way, it's a big leap and a huge amount of storage that's now possible with Netezza.

Making this all possible are some forklifts and tweaks to the underlying technology.  First and foremost is the switch from the Hitachi drives with 2- or 4-way HP/Intel host CPUs to Intel-based IBM blade servers.  Netezza is taking advantage of the faster chips, bigger disks and better interconnects that have come to market in recent years.  It has also introduced a cache, which will improve the access performance of commonly accessed tables and sections of tables.

The field programmable gate array (FPGA) remains very important in the architecture.  However, the disk controlling function has been removed from the FPGA in favor of an actual disk controller. 

I wrote a description of Netezza technology some time ago that may be worth refreshing on regarding the FPGA:

"The architecture is a shared nothing but there is a major twist.  The I/O module is placed adjacent to the CPU. The disk is directly attached to the SPU processing module.  More importantly, logic is added to the CPU with a Field Programmable Gate Array (FPGA)  that performs record selection and projection, processes usually reserved for relatively much later in a query cycle for other systems.  The FPGA and CPU are physically connected to the disk drive.  This is the real key to Netezza query performance success - filtering at the disk level.  This logic, combined with the physical proximity, creates an environment that will move data the least distance to satisfy a query.  The SMP host will perform final aggregation and any merge sort required.   Enough logic is currently in the FPGA to make a real difference in the performance of most queries."

The Intel adaption, as well as going all-Linux, makes other software more compatible with Netezza.  Obviously one of Netezza's aims is to bring over some of those other DBMS applications - appliance and non-appliance. 

The lowered price point is actually quite important in this rapidly commoditizing field.  And data size is actually a good barometer for measuring price since once you get into the terabytes with an enterprise data warehouse, the workload tends to mix in some similar ways across enterprises.  For those high-data, but specific-use workloads, Netezza will have a high capacity model available soon.  As well, Netezza intends to deliver entry level and a "memory intensive" models.  This strategy is not dissimilar to Teradata's appliance line, already available and at around Netezza's new price points.

This is a very good signal from Netezza - that it is still investing and intends to pursue price/performance for its customers.  At a time when major players like Teradata, with a longer pedigree and half the Global 2000 as customers, have entered the appliance market, and with Microsoft's looming Madison, something was necessary from Netezza.  The question is will Netezza be able to make up for the price drop with significantly more volume in this space they essentially pioneered?

      1.   The last time I checked, the NBA All-Star teams were stocked with players from 20 or so teams.  Kobe Bryant, Dwight Howard, LeBron James, Steve Nash and Dwayne Wade all play for different teams.  If a consultancy puts forward its team as the all-league all-star team, with no deficiencies whatsoever, that is a red flag.  All teams have them.  Both sides should understand this and strive for a best fit, given the realities that talent gets spread around naturally.

2.       However, consulting teams need a winning formula.  Do they know what it is?  Will that work in your environment?  For the Lakers, it was Kobe and a solid supporting cast.  For the Magic, it was Howard, Lewis, Turkoglu and a solid rotation.  Other teams put all shooters on the floor or play defense first.   

3.       I did not notice an NBA team, in an effort to save money, put the cheapest, most inexperienced player they could find on the court this season.  Heck, there are people who would pay for the glory of paying.  No, I think every team tried their best to win as many games as possible.  If your consulting team consists of 3 solid players that you are presented with, with the rest to be named later, make sure they are not filling it out with the cheapest players they can find.  Of course, that is misguided on their part as well, but sometimes you need to save the consultancies from doing the wrong thing for both of you.

4.       Scores and game clocks are not kept in the referee's head.  He does not suddenly blow the whistle and say "game over, Suns win 104-99, goodbye."  The time and the score are kept on large scoreboards for all to see throughout the game.  Do you have a scoreboard?  Does your consultancy?  It is important to know how much progress is being made throughout the game.

5.       Beyond the starting 5, NBA benches are filled with world-class athletes, many of whom get as much or more playing time as starters.  What is your consultancy's bench?  I'm not referring, necessarily, to their employees not on billing, but just what is their contingency plan in case of injury, sudden and unexpected poor performance or if a player were to leave in the middle of the game?  Is the consultancy plugged into the culture of the discipline they are engaged in?  Do they have a warm network?  Do they scout?

6.       NBA teams come to expect certain things from the places they play - things like fans, referees, locker rooms, food, transportation, hoops, lights, a marked court and basketballs to play with.  What is your consulting team expecting from you?  Software?  Hardware?  Requirements?  Access to certain individuals?  Physical space?  The ability to network their laptops?  It would be a drag to see the game try to start without a basketball or to have the lights go out in the 3^rd quarter.  Clear up expectations ahead of time with your consultancy.

7.       When the Pistons show up to the American Airlines Arena in Miami, they expect the Heat to come out of the dressing room to play against.  Imagine their surprise should the Warriors come out!  Or they have to play against 6 players on the court.  Now, they have game-planned for one team (5 players at a time) and get to play an entirely different team.  This bit of surprise will not help the Pistons be successful that night.  Is there information the consultancy is not asking for that they should be in order to know what they are up against?

8.       Sure, playing basketball is fun.  However, it's also work.  Players dive after loose balls, flying into the stands if necessary, and are expected to go all out with little consequence to their body.  They need to be skilled at avoiding injury, but cannot play overly concerned with it.  There are many moments in a consulting project where it's less fun and more work.  Are you hiring a consultancy that is prepared for the potential hard work ahead?

9.       NBA teams shoot about 80 field goals per game, hitting less than half.  Actually, only a handful of players in the league hit over 50 percent of their field goals.  However, you can't score or win if you don't shoot.  The Harlem Globetrotters are entertaining when they go into their circle and keep passing the ball, but you don't see that in a real game.  Is your consultancy willing to shoot, and are you willing to let them, even though half of the shots aren't going in, or is the consultancy interested in making entertaining passes, perhaps back to you? 

10.   Finally, experience counts.  At the NBA draft last week, I was alarmed when the announcers said that some of the second round picks would not even make the NBA.  Only 60 players are drafted each year, all with eye-popping highlights from college and European leagues, and some won't make it?!  That's how tough it is.  Is your consultancy circumventing this rule and passing along the inexperienced to you?

What I can say is I received a sneak peak at PADB (ParAccel Analytic database) V2.0 as well as a customer review.  Their new deal was sold by EMC, and this is, in my opinion, perhaps a forbearer of a new outlet for ParAccel as well as where EMC might participate in the large data appliance market.  EMC is a technology partner to ParAccel.

I have otherwise talked about these columnar appliances and when to use them.

PADB Version 2 innovations include SAN-Based optimizations which will find ParAccel claiming to be the fastest data processing machine in the world.

ParAccel uses a leader node, similar to Netezza, but different from Veritca and others, which initiates, coordinates, and collects results for the distributed queries and parallel load operations supported by the system.   The SAN utilization in Version 2 may further differentiate ParAccel and Vertica.  ParAccel is a one of the recent market entrants with the potential to offer "game changing".  

Page 120 sparked something that I thought I'd blog about.  It says "if you tell the sponsor that this initiative will help improve the company's marketing program effectiveness by making sure that we have correct customer information, and that all departments are sharing this information for their corporate-wide initiatives, you are likely to get some interest (and funding!)."

While I agree with the sentiment of aligning initiatives with the business goals, there is another part of the story I wanted to interject.  The problem in gaining interest may not be in the alignment.  Sponsors may agree that doing the project (in this case, data modeling) well will achieve some important business goals.  However, the problem may be that they do not believe that the people bringing the idea are well equipped to EXECUTE the project appropriately to achieve the desired results.

The point is to not only align projects, but also to convey the capability to deliver on the promise. 

Abstract:

Oracle Corporation (Oracle) is undoubtedly one of the market leaders in business intelligence technology.  With tools and products for just about every business function and problem, Oracle Business Intelligence (OBI) offers something for your entire enterprise.

But with so many options available, what's a customer to do?  How does one make sense of Oracle Business Intelligence?

Leveraging that experience, William has put together a highly informative webinar for this spring on The Hottest Issues in OBI!

This complimentary, 60-minute webinar is scheduled for Thursday, May 14, 2009 from 1pm to 2pm Central.  During the session, William will a range of OBI-related topics.  Those include:

• Oracle's BI and EPM product evolution
• Oracle BI applications:  What do you really get?
• OBIEE+, OBI applications and Oracle Enterprise Performance Management (EPM):
  What are the differences and which is right for your business?

While this adds some more data to the syndicated marketplace - for you to utilize - it also adds some more chaos to that world.  Syndicators are quickly integrating their new assets with their existing business and consumer profiles.

Consider newspapers.  The market is moving away from print content and to digital content.  Newspapers had the position of community pillar and the the storekeeper of its data.  For many years, there was little else to turn to for advertisement.  Now, there are the eBays, amazons, CraigsLists, googles, etc., and many are running with community sections now so they have global efficiencies with localization.  They are game-changers.  Newspapers may fall further behind.  One challenge has been the newspapers structure of separating the digital and print teams.  A place to begin bringing these together is at the information.

Some time ago, they also announced Kilimanjaro, the next version of SQL Server, and Gemini, the next version of SQL Server Analysis Services.  I didn't rush out and blog this though because the release is not expected until 2010.  Still, it's exciting and worth thinking about now.  And who knows, maybe these will emerge sooner than expected.

I thought SQL Server 2005 was BI focused.  Now, Microsoft says Kilimanjaro will really be BI focused.  And they could be right.  With the addition of optional column-oriented storage capabilities and in-memory storage, they are addressing 2 very obvious ways the market has recently addressed performance.  Both options were built from scratch by Microsoft.  I look forward to seeing how the optimizer determines when to go columnar vs. not.  To-date, solutions have addressed it with disparate systems and obvious suboptimal performance of some of the queries within each system. 

I'm a fan of SSAS as long as the MOLAP option is not overused, but that's another story.  Stronger connectivity is promised between Gemini and Excel and that may really get the casual user's attention.  Gemini will be publishing data to SharePoint Server as well.  SharePoint is becoming a centerpiece of Microsoft BI activity.  With the publishing options,Gemini will have the Microsoft enterprise covered.

What will be the next game changer?