Fun Programs to keep your computer running smooth

Thu, 18 Dec 2008 15:44:17 -0700

This has nothing to do "really" with BI, but on the other hand, we nearly all use Windows at some point in our career, and over the years I have latched on to some vital programs that I really like to use to keep my machine running efficiently. They are mostly fairly cheap, and work really really well.

]]>

The need for: IT Agility & Data Warehousing/BI

Thu, 18 Dec 2008 03:01:34 -0700

In this day and age everyone is cutting costs, every customer and corporate client is looking for ways and means to become lean and efficient. I've heard a lot about disparate "enterprise data junkyards" recently, especially when it comes to stove-piped solutions involving star schemas as an EDW causing problems with IT agility. As a result of problems with IT agility in the area of EDW/BI processing, business users continue to build spread-marts (access databases, along side of complicated excel spreadsheets). In this entry we will explore this phenomenon, and discuss what executives and business users can do about this growing issue.

Dimensionitis - Federated Stars as an EDW

Tue, 18 Nov 2008 03:54:38 -0700

If you have a large number of star schemas, or a large federated star schema as an enterprise data warehouse, then you might or might not have this issue. This is one of the issues affecting business today. In this entry we will explore the issue called Dimensionitis from a business perspective, in other words: how much does it cost to maintain, what happens when... and so on.

People believe Surrogate keys are better than Business Keys

Sun, 02 Nov 2008 22:07:05 -0700

Well, it's happened again. IT is trying desperately to eliminate the value of the EDW from the business (at least this is what I see). Business is responding by demanding the creation of Master Data systems. There seems to be an age-old argument in the market space about the use of, definition of, and condition of: Business Keys. IT appears to be telling people to use surrogate keys and to ignore the business keys entirely. In this entry we will explore this single notion, and see what some folks have to say about it (me included!) Mind you, this is a bit of a rant; they seem to know how to "get my goat" as they say...

Many vendors Claim dynamic data warehousing...

Wed, 29 Oct 2008 23:31:08 -0700

But very few (if any) actually execute on the vision that I am laying out here. This is a very short entry, but basically re-iterates some of the points of Dynamic Data Warehousing that I believe to be necessary before it (software/appliance/database) can be labled as being something like this.

VLDB/VLDW Expected Issues

Thu, 25 Sep 2008 12:15:52 -0700

VLDB (very large databases) and VLDW (very large data warehousing) are two different terms in the industry that evoke a lot of stir. The terms have been changed, altered, re-defined, and brought back to the table many times by many people. Their are many problems associated with implementing "big systems" and not very many solutions (although vendors are trying). There are some major business questions around the data sets and the application of such large data sets.

In this entry I will explore the business questions, and the technical challenges faced by big systems. I will attempt to hold my opinion, and see what the responses are - what do you think are issues faced by your business?

Part 9: Secrets of the Masters - Tracking & Governance

Wed, 24 Sep 2008 20:51:38 -0700

In part 7 of this series I mentioned that I would share how to number deliverables of the project to assist in monitoring progress, and managing metrics (KPA's and KPI's of the project). In this entry I provide some very simplistic starting blocks on how this is done within a project methodology, and hopefully - there's light at the end of the tunnel where we can begin to see the impact on the risk, estimations of hours, cost measurements/forecasts, and actuals for delivery. This entry is all about Project Management and deliverables - how to tie them together.

Dynamic Data Warehousing Stepping Forward

Sun, 21 Sep 2008 21:52:48 -0700

I've blogged about this topic for many years now, my first mention of it was in my www.TDAN.com articles regarding the Data Vault Modeling architecture. However, that said, I've been blogging on everything from autonomic data models, to dynamic data warehousing, but in my research, I've come to realize I've left out some very critical components. I've lately been experimenting with building a self-adapting structured data warehouse. There are many moving pieces and not all the experiments are finished, so I cannot write (yet) about any of the findings. But here, I'll expose some more of the under-belly as it were that is necessary to make DDW a reality (in my labs anyhow)....

VLDB: Column Based versus Row Based

Sun, 21 Sep 2008 14:56:49 -0700

Column based databases/appliances are making headway in the VLDB/VLDW world. There is no doubt that there are benefits to this approach, but there are also drawbacks. In this entry I explore some of the articles, links, facts and figures - as related to my personal experience. Then I compare what different authors are saying against Row-Based MPP technologies to see what the differences and similarities are. This by no means is a complete research paper, but just a peek into what the future may hold for RDBMS vendors and the new Column based data stores. Of course, Solid state disk, and RAM/Flash based data sets will change things again shortly. I'll also touch on the impacts to Data Modeling and what it may mean going forward.

Self-adapting Data Models

Wed, 27 Aug 2008 05:54:29 -0700

In my last entry in this category, I described automorphic data models and how the Data Vault modeling components is one of the architectures/data models that will support dynamic adaptation of structure. In this entry I will discuss a little bit about the research I'm currently involved in, and how I am working towards a prototype of making this technology work.

If you're not interested in the Data Vault model, or you don't care about "Dynamic Data Warehousing" then this entry is not for you.

Part 8: Secrets of the Masters - Business Requirements

Tue, 26 Aug 2008 05:32:21 -0700

Here is another installment of the secrets of the masters. Quite frequently customers and IT alike complain about how difficult it is to gather business requirements. They discuss the pain of having to "get together" for a day, or for a week-long process to write down and document business processes, and ultimately their needs and desires for a new BI/EDW system. Any good analyst worth their salt has battle-scars from negotiating these treacherous grounds.

We've all walked in to an environment with a blank white-board and asked: business, please give me your requirements, only to be confronted with: "What can you provide to us?"

IT Agility and Repsonsiveness to the Business

Sat, 23 Aug 2008 05:53:03 -0700

It's not often in our industry you get a chance to read about successes. Too much press is given to negative types of issues. This entry is about successful implementations.

Would you like your IT team to build "Data marts in about an Hour?", How about full EDW's with AS-IS star schemas in 2 weeks (regardless of size of source systems, or number of systems to integrate)? Would you as a business user like to hear that your new requirements for reporting can be met within a 2 day turnaround? How about your IT team becoming a profit center for the stake-holder rather than a cost center?

Sound too good to be true? It's NOT! Honestly, this is the first time in a long time that I'm excited again to be in IT. I'm working with several customers in which we have made these things a reality. This entry is about how we did it, and how you can do it too.

True Temporal Based RDBMS engines

Fri, 18 Jul 2008 05:22:06 -0700

When I teach, I frequently discuss temporal based data sets - after all, that's a big piece of what data warehousing and BI is about - Data Over Time. But when examining the database engines ability to "retrieve" specific data sets as a snapshot in time, it seems there is a problem. There appears to be no "consistent" manner in which to retrieve these layers for use by the business. We are left to create physical dimensions and physical fact tables - aggregate our data up to higher levels (to shrink the amount of data) so that joins can execute cleanly and efficiently across information. So why then, after all these years haven't vendors properly implemented the ANSI-SQL-92 standard of "PERIOD"?

Database Vendors, are you listening? There is a serious revenue gain to be had by implementing these feature sets...

Part 7: Secrets of the Masters, Templates for Projects

Wed, 16 Jul 2008 15:15:30 -0700

Any time we get back to secrets, we seem to fall right back to the category of standards, standardization, measurement and enablement. The old saying is: "if you can't measure it, you can't monitor it, and if you can't monitor it - you don't know when it's broke, or you can't optimize it/fix it." Something like this anyhow.

The common feedback from the general project implementation community is usually: "Why do I need to standardize? Why should I document? Won't it take more time to follow standards than to build rapidly?"

Design: Subject Oriented Versus Function Oriented

Mon, 30 Jun 2008 05:57:31 -0700

For a long time Dr Ralph Kimball has spoken about subject oriented design. Many have made a living off of producing subject based data marts. One of the problems this has lead to is a series of loosely coupled stove-piped answer sets that are then "discussed" in the light of an enterprise data warehouse. I've been teaching, talking and writing about (over the last 10 years at least) a notion called Functionally Oriented Design. In this entry I will briefly introduce my notions of these concepts.

Blog: Dan E. Linstedt