We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

Blog: Claudia Imhoff Subscribe to this blog's RSS feed!

Claudia Imhoff

Welcome to my blog.

This is another means for me to communicate, educate and participate within the Business Intelligence industry. It is a perfect forum for airing opinions, thoughts, vendor and client updates, problems and questions. To maximize the blog's value, it must be a participative venue. This means I will look forward to hearing from you often, since your input is vital to the blog's success. All I ask is that you treat me, the blog, and everyone who uses it with respect.

So...check it out every week to see what is new and exciting in our ever changing BI world.

About the author >

A thought leader, visionary, and practitioner, Claudia Imhoff, Ph.D., is an internationally recognized expert on analytics, business intelligence, and the architectures to support these initiatives. Dr. Imhoff has co-authored five books on these subjects and writes articles (totaling more than 150) for technical and business magazines.

She is also the Founder of the Boulder BI Brain Trust, a consortium of independent analysts and consultants (www.BBBT.us). You can follow them on Twitter at #BBBT

Editor's Note:
More articles and resources are available in Claudia's BeyeNETWORK Expert Channel. Be sure to visit today!

How much do you trust what Gartner, IDC, Forrester, and the rest of the IT research firms say? This is the topic of an upcoming (February 6) Information Week article as announced in a blog written by Larry Greenemeier. The magazine interviewed dozens of IT vendors, end users, and executives at the top analyst firms to answer the credibility question. The article promises to be a pistol to say the least.

According to Larry, the analyst industry has "tapped into an almost desperate need that businesses have to stay on top of emerging technology and a community of IT vendors even more desperate to make the sale". Not the most complimentary comments but you have to admit -- it is suspect when an analyst publishes a purportedly independent report on the state of the vendors in our industry, but the report is funded by the very vendors being reported upon.

Certainly we all have become enamored with the Magic Quadrant and the future predictions from these companies. We can't wait to see who's in and where they are positioned. But do the analysts actually work with the products? Is there testing done? How do they evaluate "vision" versus "ability to execute". These seem rather subjective to me (IMHO). In an age of data transparency and validity (a la Sarbanes-Oxley), perhaps these firms should publish all the criteria they use in these evaluations.

On the other hand, there can be no question that these firms do have significant influence in our industry. The fact that we are all reticent to question them speaks volumes. What vendor would consider criticizing these companies in public? What customer questions the research findings?

So -- I wait for the article with great anticipation. What will Information Week have to say about these companies? Will it be an expose or a white wash?

Meanwhile, what do you have to say? If you are more comfortable being anonymous, feel free to use a pseudonym for your name in the comment area. I would love to hear your opinions.

Yours in BI Success,


Posted February 1, 2006 4:32 PM
Permalink | 4 Comments |


There seems to be a difference between IT analyst evaluations and consumer or automobile evaluations. I use Choice in Australia for evaluations of common consumer items and they give a good description of the testing process and a list of ranking criteria and scores in each category. The evaluations from IT analysts are not so transparent.

This may be the analyst firm trying to protect what they see as IP. If they reveal the ranking engine then won't other analyst firms copy it?

There is also ambiguity over what makes software good. If I evaluate a toaster I want to know how well it toasts bread and whether crumpets fit. It is not so clear cut with enterprise software.

An example is the often cited proprietory repository which I see pop up in BI and DI evaluations all the time. One vendor may support open repository platforms such as Oracle/SQL Server etc so an analyst may see this as a plus. Another vendor may choose to support only a proprietory repository because it provides faster software and is easier to install. So how do you score this? As you point out it gets subjective.

Vendors need to keep talking to analyst firms to explain their vision and customers need to take the reports with a grain of salt. If they are spending a lot of money on a project they should do their own evaluations.

Excellent comments, Vincent. Your last two sentences summarize what vendors and consumers should do nicely. Thanks for the input. But crumpets? In a toaster? :-)

You can put them under the grill but they burn easily. A wide mouth toaster is the way to go. The really clever ones have sides that press in to fit the width of item to be toasted!

Hey, check out Information Week, 6 Feb: "IT Research Firms Need to do a better job of educating customers about policies."

Leave a comment


Search this blog
Categories ›
Archives ›
Recent Entries ›