But very few (if any) actually execute on the vision that I am laying out here. This is a very short entry, but basically re-iterates some of the points of Dynamic Data Warehousing that I believe to be necessary before it (software/appliance/database) can be labled as being something like this.
In my definition of dynamic data warehousing the software around the database is an artificially intelligent engine. The database contains metadata about the structure, about the usage of the structure, and versions of all this metadata (producing a structural and usage life-cycle).
In other words, the AI engine is fed or kick started with an ONTOLOGY. The ontology of terms defines the basic data model that is executed underneath. The Ontology is driven by business terms, business definitions, functions, and descriptions (in accordance with OWL ontology). Secondarily, the AI engine is fed many different data points including usage of the ontology:
* SQL Queries
* Loading Code
* Scripting Code
* Application Code
* Web Service Code
And all of the table references/usages/join criteria components within the code.
Dynamic Data Warehousing RESPONDS to changes BY ITSELF. It responds to USAGE controls (ad-hoc queries, repeated queries, and so on). It responds to LOADING controls (changes to structures, appearance of new attributes/fields, changes to loading code, volume and width). It responds to length of processes (metrics driven), and responds to USER DRIVEN ONTOLOGY CHANGES (based on business requirements).
At the end of the day, the AI engine grades changes, and figures out by itself, how to a) TUNE the structure b) ADAPT or CHANGE the structure, including indexing, c) Add new elements to the structure, d) retire old elements from the structure, d) OPTIMIZE the modeling paradigm for today's business execution cycles, e) manage and propogate structural changes TO the loading code, TO the queries, and TO the ontologies.
Vendors may claim that they have "Dynamic Data Warehouses" all they want, but until they have automatic detection of structural changes, and automatic propogation of those changes - and these automated systems are associated to/with business ontologies, I will not agree that they infact have a "Dynamic Data Warehouse".
This is just my opinion. I believe that in order to become more fluid, and more appropriate to the business, and closer to business change, these are the kinds of systems that will evolve in the next 5 to 7 years.
Cheers for now,
Feel free to contact me directly: DanL@GeneseeAcademy.com, we teach custom Informatica courses, DW2.0 and Unstructured Data Courses, Zachman Framework courses, and Data Vault Data Modeling courses.
Posted October 29, 2008 11:31 PM
Permalink | No Comments |