Originally published July 21, 2010
The explosion of instant content produced by online news sites, blogs, and social media has huge implications for the business and has become top of mind for information executives. When harnessed in real-time, predictions of future market trends and superior business decisions are inevitable.
What’s changed is information officers now have a whole new breed of data to think about thanks to the real-time Web. The real-time Web is actually much broader than microblogs such as Twitter – it’s instantly available news, search, and information from within the enterprise (e.g., collaboration tools such as wikis) and also legacy applications. The mind-staggering speed at which this data accumulates is what makes it so difficult to harness and make sense of. To quantify this in physical terms, a May 2010 EMC-sponsored IDC study found that the amount of digital information that will be created in 2010 alone will be 1.2 zettabytes, which equals 75 billion fully loaded 16 GB Apple iPads!
Making matters more complicated is that much of this data is completely unstructured, meaning the common tools enterprises use for business intelligence (BI) initiatives and related applications are rendered useless. As such, the existing challenge integrating this data into enterprise information systems is even more profound. And with real-time Web data only increasing, the winners in this new environment will be those companies that can figure out how to capitalize on the new real-time information streams available today, and make them relevant, meaningful and actionable to their organizations.
Given these real-time Web data access and delivery challenges faced by literally every industry, there’s recently been a lot of talk about Web data services as a possible fix. Wikipedia defines Web data services as “service oriented architecture (SOA) applied to data sourced from the World Wide Web.” In plain English, these services are basically designed to speed the finding, cleaning, and feeding of Web data and content into business applications. Industry watchers are validating that organizations increasingly need relevant, real-time data from public websites, such as BBC.com, or social networking sites such as Twitter, LinkedIn or Facebook, as well as from private, firewall-protected websites and Web applications to make better business decisions, faster. Web data services effectively deliver a modern and massive scale data integration method for the real-time enterprise.
A good first step in harnessing this treasure trove of Web data is for organizations to break down the different types of information available and then select what is most relevant to their particular goals and needs. What follows is a short list of information categories and how companies can leverage these to predict future business events and improve their competitive position: