But if you do not know the index or a catalog to this wonderful library and you cannot correlate the emails and the library of documents, there is certainly a large gap that needs addressing. Consider the business value of the time spent searching these documents to answer any business question that came from your data analysis of the data warehouse?

This is where the answer to why integrate the unstructured data and measure its business value lies in. If you can link all the data layers in the data warehouse ecosystem from operational to analytical to all the unstructured content, there is even more business value.

The day is not far off when your business users will request integration of unstructured content into your data warehouse, till then start preparing to understand this space and what you need to do to accomplish this task.

Amazon.com’s web-based database that will work closely with its S3 online storage service and EC2 online application hosting service. S3 is shorthand for Simple Storage Service, while EC2 stands for Elastic Compute Cloud. The SimpleDB dadatabases aimed at application developers looking for a less expensive, easier-to-use alternative to a full relational database for running real-time queries against structured data.
According to Amazon.com, SimpleDB provides the core functionality of a database - real-time lookup and simple querying of structured data, without the operational complexity. The database service will automatically index data and provide performance tuning, allowing Web developers to work without the aid of a full-time database administrator.
Though its core business is an online retailer, Amazon.com's move into cloud-based computing is being welcomed by users, especially those involved in Web 2.0 computing. CoConsideringhat cloud computing is an area where Microsoft, Sun, IBM and HP also want to dominate, the are several reasons for cheer here
• The cost of the company's services is one of the benefits that have been cited. For instance, S3 lets users store 1GB of data for just 15 cents per month.
• Storing data in structured form in SimpleDB, instead of the raw storage that S3 offers, will cost $1.50 per gigabyte monthly, according to Amazon.com.
• Database requests will cost 14 cents for each machine hour of processing time used, based on the hourly capacity of a Xeon processor.
• The cost of transferring data into SimpleDB will be 10 cents per gigabyte, while outward data transfers will start at 18 cents per gigabyte for the first 10TB of information transferred in a given month.
SimpleDB is designed to store relatively small amounts of information and then to provide fast access to the data. To keep storage costs down, it recommends that users store large objects and files in S3, and put pointers and metadata associated with the files in SimpleDB.
Apart from Amazon.com, Trackvia Inc, Google and Intuit are also offering services in this area.

While this is a new technology area, it does open doors for the next generation of scalability and performance for data practitioners to think about.

Data management from a lifecycle perspective requires input from all data consumers and their needs. It involves data governance and organization alignment to be champions of the process.

Once you have organizational alignment on the data value and its lifecycle, then you have to implement a program to manage that data and its associated metadata. This phase will require more involvement from the IS and IT teams to make it a successful program.

An article on this subject in greater detail will be published in my channel later this month.