Blog: Dan E. Linstedt Subscribe to this blog's RSS feed!

Dan Linstedt

Bill Inmon has given me this wonderful opportunity to blog on his behalf. I like to cover everything from DW2.0 to integration to data modeling, including ETL/ELT, SOA, Master Data Management, Unstructured Data, DW and BI. Currently I am working on ways to create dynamic data warehouses, push-button architectures, and automated generation of common data models. You can find me at Denver University where I participate on an academic advisory board for Masters Students in I.T. I can't wait to hear from you in the comments of my blog entries. Thank-you, and all the best; Dan Linstedt http://www.COBICC.com, danL@danLinstedt.com

About the author >

Cofounder of Genesee Academy, RapidACE, and BetterDataModel.com, Daniel Linstedt is an internationally known expert in data warehousing, business intelligence, analytics, very large data warehousing (VLDW), OLTP and performance and tuning. He has been the lead technical architect on enterprise-wide data warehouse projects and refinements for many Fortune 500 companies. Linstedt is an instructor of The Data Warehousing Institute and a featured speaker at industry events. He is a Certified DW2.0 Architect. He has worked with companies including: IBM, Informatica, Ipedo, X-Aware, Netezza, Microsoft, Oracle, Silver Creek Systems, and Teradata.  He is trained in SEI / CMMi Level 5, and is the inventor of The Matrix Methodology, and the Data Vault Data modeling architecture. He has built expert training courses, and trained hundreds of industry professionals, and is the voice of Bill Inmons' Blog on http://www.b-eye-network.com/blogs/linstedt/.

EII has been getting a lot of buzz lately, especially with the purchase of Meta Matrix by Red Hat. I want to turn your attention (instead) to where EII needs to go as an industry. These are my opinions, and I welcome you’re constructive comments. EII (enterprise Information Integration) is a pull technology - grabbing data on-demand when needed from all kinds of sources, and building a single integrated view of the current world of "transactional data." So what's left?

In the future as we progress towards heterogeneous appliances, we will need EII more and more, especially with it's persistence of data in a virtual world. But what we are missing today are a few feeds on metadata (both business and technical), infrastructure and management of multiple web-services domains (both inside and outside the company walls), and the ability to track changes to data models - be it web service structure changes or physical data model changes in source systems.

EII will become more and more important as a back-office integration system and "glue" providing the framework needed to run the back-office more efficiently. I would expect that the EII tool of the future will pick up and integrate the appliances, along with managing the network of appliances in the plug and play scope. The more we can virtualize the information on a transactional level (and integrate it on the fly) the better we can manage all the back-office systems.

Furthermore, I expect the GUI of the EII tools to be focused more on the front-end users, bringing the integration management out of the back-office and more into the business user world. I believe that by focusing the EII GUI on plug & play nature it will provide additional power to business rule engines, workflow engines, processing engines, metrics engines, and of course metadata engines.

The EII GUI will reach the front office, and be simplified (as it should be), the the advanced interface will still be available for the IT staff, however business users should be able to switch context within their portals and not know or care that they are using EII for data exploration. Plugging EII directly into source data systems and pumping the data into MS-OLAP cubes (MDB), or Excel will push utilization forward.

Metadata collection systems are being built and focused on, particularly over the past year by all kinds of vendors including Meta Integration, ASG Systems, CA and so on. However, the interfaces used to collect and manage (not to mention link together) the metadata leaves a bit to be desired. EII is a perfect fit for integrating all kinds of metadata in a visual format, and providing a repeatable metadata integration and management front-end. By leveraging EII's ability to connect to all kinds of sources, and by visualizing the metadata stores we can easily combine the metadata into a common data model and write the metadata back.

Not only should EII be providing visualization of Metadata, but it should also plug in to the Reporting Tools out there, and provide the metadata feed on the fly with all the security and accessibility that the reporting tools offer. Management of the metadata MUST be created into a GUI somewhere, and it should be leveraged with EII's ability to not only "allow alteration" but provide write-back of the metadata to a common repository.

Summary:
EII of the future will have a much more robust GUI. There will be two different components to the GUI - a business user interface, and a technical interface. The EII GUI will become a BI tool in it's own right, and should plug and play with business rules engines, business processing workflow engines, web-services, metadata engines, and front-end applications like Excel and Microsoft OLAP cubes. The write-back capabilities should be leveraged to manage change, and assist with producing a common data model for the data to reside in (which will eventually be an exploration warehouse of sorts).

Do you have any thoughts?

Thanks,
Dan Linstedt
Check out a Masters of Science in Business Intelligence at: http://www.COBICC.org


Posted May 10, 2007 5:55 AM
Permalink | 2 Comments |

2 Comments

Dan,

Interesting thoughts on where EII needs to go. Let me give you my thoughts as a vendor.

I generally agree that EII servers need to continue to improve their ability to track changes in underlying data and metadata. While EII is not alone in having this problem, it tends to be top of mind for potential buyers, so it needs to be addressed. You will see some new things o n this in our next major release later this year.

As for the UI being usable by "end users," I'm of two minds on this. On the one hand, the job of a good EII server is to be transparent, working underneath tools like Crystal and BusinessObjects. So having an "end user UI" would be somewhat counter to this. On the other hand, you are not the first to ask for this, and it is very tempting to try to empower users - so long as you can give admins and DBAs comfort that the whole system won't come crashing down. Maybe there's some happy medium. I don't have the answer yet, and would love some feedback.

Generally speaking, I think servers like Ipedo are mature in their capture and handling of internal metadata, transactionality, and Web Services handling. We just need to make all of it more obvious and easy to use. As one of the two remaining pureplay EII vendors, I can tell you this is the direction we are moving in.

Tim
Co-Founder, Ipedo

Hi Tim,

Thank-you for your contribution. I think having "two minds" is the right way to address this. I believe that transparency is of utmost importance to the back-office, but I also believe there are holes in the BI (front-end) solutions today.

I think having two different interfaces on top of EII would bring it into the lime-light. In other words, one front-end for administrators and sophisticated users. One front-end to manage end-user metadata and ontologies.

Again, I think exposing parts of the architecture through a smart client interface (web-based, thin client) is an excellent way of tying the two pieces together (all the SLA's that we must meet technologically, and the business users wishes to manage their own metadata and definitions).

Cheers,
Dan L
Full Disclosure: I am a part of the Ipedo Technical Advisory Board

Leave a comment

    
Search this blog
Categories ›
Archives ›
Recent Entries ›