We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

Blog: Richard Hackathorn Subscribe to this blog's RSS feed!

Richard Hackathorn

Welcome to my blog stream. I am focusing on the business value of low latency data, real-time business intelligence (BI), data warehouse (DW) appliances, use of virtual world technology, ethics of business intelligence and globalization of business intelligence. However, my blog entries may range widely depending on current industry events and personal life changes. So, readers beware!

Please comment on my blogs and share your opinions with the BI/DW community.

About the author >

Dr. Richard Hackathorn is founder and president of Bolder Technology, Inc. He has more than thirty years of experience in the information technology industry as a well-known industry analyst, technology innovator and international educator. He has pioneered many innovations in database management, decision support, client-server computing, database connectivity, associative link analysis, data warehousing, and web farming. Focus areas are: business value of timely data, real-time business intelligence (BI), data warehouse appliances, ethics of business intelligence and globalization of BI.

Richard has published numerous articles in trade and academic publications, presented regularly at leading industry conferences and conducted professional seminars in eighteen countries. He writes regularly for the BeyeNETWORK.com and has a channel for his blog, articles and research studies. He is a member of the IBM Gold Consultants since its inception, the Boulder BI Brain Trust and the Independent Analyst Platform.

Dr. Hackathorn has written three professional texts, entitled Enterprise Database Connectivity, Using the Data Warehouse (with William H. Inmon), and Web Farming for the Data Warehouse.

Editor's Note: More articles and resources are available in Richard's BeyeNETWORK Expert Channel. Be sure to visit today!

In an outstanding research article, Jure Leskovec of Carnegie-Mellon and Eric Horvitz of Microsoft Research analyzed 30 billion IM conversations among 240 million people during the month of June 2006. This resulted in a network graph with 180 million nodes (a person who sent a message during that month) and 1.3 billion undirected edges (two persons exchanged messages) - by far the largest data set for social networking. In fact, the authors described their investigation as being 'planetary-scale'!

A key result was the confirmation of Stanley Milgram's popularized hypothesis that any two persons in the world are, on the average, separated by six degrees. That is, only four other persons are needed to form a path between those two persons. Leskovec and Horvitz found that the average degree of separation was 6.6 with a median of 6. The longest paths had lengths of under 30, which is still amazing. In contrast, if one person wanted to find another person among a group of 180 million, you would expect to search through half of that group. Right? Correct if you were searching randomly for that person.

This is called the Small World phenomenon where the interconnections of a social network are much more strong than one would expect based on random networks. In other words, we as social beings have a strong affinity for selectively connecting to others, even using Microsoft Messager.

The above result about six degrees was only one of dozens. The results of the research article go on and on! The reader is forewarned that the article is slow reading, but only 10 pages.

I wrote an article entitled The Link is the Thing in August of 2003, in which I applied the Small World phenomenon to enterprise data warehouses. We may have small worlds lurking in our EDW! I encourage you to apply network analysis techniques to EDW as I suggested. I obviously would be very interested in what you find.

Posted August 5, 2008 1:17 PM
Permalink | No Comments |

Leave a comment