Blog: Krish Krishnan

« January 2008 | Main | March 2008 »

February 24, 2008

A closer look at UnStructured data

We have been hearing and reading about the content of unstructured data and the potential it may unleash when integrated into the data warehouse. What is the operational value of this content and how to measure the value of such an integration effort.

Well if you are a healthcare or a pharmaceutical company, think of the vast amounts of data in your companies notes, emails, documents, research findings etc. Well if you rely on a document management system for all your document needs except the email portion, you have a good library.

But if you do not know the index or a catalog to this wonderful library and you cannot correlate the emails and the library of documents, there is certainly a large gap that needs addressing. Consider the business value of the time spent searching these documents to answer any business question that came from your data analysis of the data warehouse?

This is where the answer to why integrate the unstructured data and measure its business value lies in. If you can link all the data layers in the data warehouse ecosystem from operational to analytical to all the unstructured content, there is even more business value.

The day is not far off when your business users will request integration of unstructured content into your data warehouse, till then start preparing to understand this space and what you need to do to accomplish this task.

Share:

February 18, 2008

A future possibility - Data Warehouse on a cloud

In the past few months there have been a number of success stories on the Amazon.com cloud computing services for web developers called AWS - Amazon Web Services which include storage database and computing platforms. At a first pass this may seem like a reinvention of the ASP hosting model in the dotcom days. The key difference here is the simple service model and the infinite scalability offered by Amazon.com.

Amazon.com’s web-based database that will work closely with its S3 online storage service and EC2 online application hosting service. S3 is shorthand for Simple Storage Service, while EC2 stands for Elastic Compute Cloud. The SimpleDB dadatabases aimed at application developers looking for a less expensive, easier-to-use alternative to a full relational database for running real-time queries against structured data.
According to Amazon.com, SimpleDB provides the core functionality of a database - real-time lookup and simple querying of structured data, without the operational complexity. The database service will automatically index data and provide performance tuning, allowing Web developers to work without the aid of a full-time database administrator.
Though its core business is an online retailer, Amazon.com's move into cloud-based computing is being welcomed by users, especially those involved in Web 2.0 computing. CoConsideringhat cloud computing is an area where Microsoft, Sun, IBM and HP also want to dominate, the are several reasons for cheer here
• The cost of the company's services is one of the benefits that have been cited. For instance, S3 lets users store 1GB of data for just 15 cents per month.
• Storing data in structured form in SimpleDB, instead of the raw storage that S3 offers, will cost $1.50 per gigabyte monthly, according to Amazon.com.
• Database requests will cost 14 cents for each machine hour of processing time used, based on the hourly capacity of a Xeon processor.
• The cost of transferring data into SimpleDB will be 10 cents per gigabyte, while outward data transfers will start at 18 cents per gigabyte for the first 10TB of information transferred in a given month.
SimpleDB is designed to store relatively small amounts of information and then to provide fast access to the data. To keep storage costs down, it recommends that users store large objects and files in S3, and put pointers and metadata associated with the files in SimpleDB.
Apart from Amazon.com, Trackvia Inc, Google and Intuit are also offering services in this area.

While this is a new technology area, it does open doors for the next generation of scalability and performance for data practitioners to think about.

Share:

February 7, 2008

Lifecycle of your data

We have been seeing the latest surge in discussing the lifecycle of data and how to manage the same. Bill Inmon discussed it years ago and more recently has authored the DW2.0 methodology, one part of which is data lifecycle management. We have database vendors talking about data and different types of data and how they are proposing solutions to manage the same.

Well all of this is great, but how do we measure the value of data in your business and thus determine its lifecycle and manage the same? data warehouse and ODS data have different requirements for lifecycle from transactional systems.

Data management from a lifecycle perspective requires input from all data consumers and their needs. It involves data governance and organization alignment to be champions of the process.

Once you have organizational alignment on the data value and its lifecycle, then you have to implement a program to manage that data and its associated metadata. This phase will require more involvement from the IS and IT teams to make it a successful program.

An article on this subject in greater detail will be published in my channel later this month.

Share:

February 1, 2008

Microsoft to buy Yahoo !

Breaking news on Feb 1st 2008, Microsoft is offering to buy Yahoo. Well this one will be very interesting for Microsoft. It will give the company the ability to compete in many areas where it wanted to. We will have to wait and see where this goes, but certainly a Microsoft - ATT partnership from this buyout means VOIP calling from Windows desktops instead of SKYPE is a real possibility.

From a data perspective too there are endless opportunities that will evolve. Another offering of the future is an integrated appliance that will be your TV+VOIP Phone+Desktop+Search Engine+....

Till all shakes out..........................

Share: