Hey - I was quoted in a recent article about finding value in "dirty data." Apparently the author, Hannah Smalltree, read my blog entry back in August on "Dirty Data and Embedded Knowledge," and decided to follow up on the concept with others in the DQ field, including Ted Friedman from Gartner, and Ramesh Menon from Identity Systems. Let me know what you think!

As a data quality practitioner who always preaches the value of determining return on investment, I relish the opportunity when a simple data quality problem has significant impact. Well, last week, my dream came true. As reported in December 9th's Washington Post, erroneously placed sell orders will end up costing a financial services firm approximately $225 million dollars (yes, I said million).

Apparently, the company "mistakenly sold 610,000 shares of J-Com Co. at 1 yen (less than 1 cent) per share, instead of fulfilling a client's request to sell just one share at 610,000 yen ($5,080)." The problem occurred because of an erroneous data entry mistake that was not caught at the time the order was placed.

The impacts of this mistake bubbled through the Japanese market, sending the Nikkei average down 1.95% (wow).

Actually, the Tokyo Stock Exchange is admitting some responsibility as well. This will probably result in some government directives to improve their business processes to enable the determination of suspect transactions such as this one.

Business Intelligence tools help individuals gain insight through the presentation and visualization of information. That insight, though, is often achieved as a result of a process of reviewing reports, drilling into and through different performance metrics, and some good ol' fashioned noodling. How do you capture the process you went through to reach some conclusion so that you can recreate the thought process for others?

In a recent networking meeting, I was asked whether I thought blogs and/or wikis would be good platforms for capturing those thought processes. The idea appeals to me - anyone out there have any experience with doing this?

According to a news item I read the other day, according to a study done by the Pew Internet and American Life organization, the top 3 web activities are:

3) Reading news,
2) Using a search engine, and
1) E-mail

The interesting note is that searching has moved up into the number 2 spot, meaning that aside from sending and reading emails (which is standard operating procedure for most people these days), the thing that people do most on the web is using some kind of search engine to look for something. Of course, one might assume that the search is a prelude to some other action, but this fact establishes that the key to web activity is the search engine.

The ability to provide search capability really epitomizes master data management - it means being able to:
- identify entities
- maintain entity metadata in a browsable repository
- provide fast access to the seek through the metadata

The fact that search engines now provide some kind of spelling suggestions when your searches don't have significant results demonstrates how fundamental data quality, metadata, and record linkage techniques are being integrated into the "search business."