Blog: Wayne Eckerson Welcome to Wayne's World, my blog that illuminates the latest thinking about how to deliver insights from business data and celebrates out-of-the-box thinkers and doers in the business intelligence (BI), performance management and data warehousing (DW) fields. Tune in here if you want to keep abreast of the latest trends, techniques, and technologies in this dynamic industry. Copyright 2014 Tue, 22 Jul 2014 06:52:49 -0700 Need Hadoop-based Analytics in the Cloud? Try Qubole Qubole is a new breed of analytic software company that runs on Hadoop in the public cloud. The ideal customer is one that already runs a lot of applications in the cloud and wants to accelerate the time it takes to make big data available to business analysts and data scientists. Qubole's big competitor is Amazon Web Services and its Elastic MapReduce (EMR) offering, which provides the Hadoop platform as a cloud-based service. Unlike EMR, Qubole was designed from scratch to support queries and analytics. Also, the service is geared t business analysts and data scientists, not Java and other developers, so it is easier to use. Tue, 22 Jul 2014 06:52:49 -0700 ScaleOut Software Sits at the Nexus of Operational and Analytical Computing ScaleOut Software provides an in-memory data grid that primarily provides fast reads/writes for high-speed operational applications, such as e-commerce, reservation systems, credit card processing, equity trading, smart grids, and cable network streaming. There are a lot of competitors in the space, such as Oracle Coherence, but ScaleOut and Apache Spark differentiates itself by supporting analytics on live, operational data, among other things. Sat, 19 Jul 2014 12:45:36 -0700 BI Survey 14 is the Largest Survey of BI Tool Usage in the World There is no better source for the trends in the use of business intelligence (BI) tools than the BI Survey conducted annually by the German-based research house, BARC. The BI Survey, now in its 14th year, is the world's largest independent survey of BI users, with more than 2,500 survey takers around the globe. This summer, I will help BARC's research staff evaluate the results of the BI Survey, which closes in a few weeks (late June.) The BI Survey tracks customer attitudes towards more than two dozen BI products. The published report discusses key purchasing and usage patterns, including total cost of ownership, market and customer penetration, query performance, support quality, business benefits, and technical challenges, among other things. Mon, 09 Jun 2014 09:05:50 -0700 The Keys to Scorecard Success A balanced scorecard is a powerful tool for aligning an organization. It displays the metrics that represent the key drivers of long-term performance. In many ways, it's a visual representation of an organization's strategy, tailored to every department and individual. Unfortunately, most organizations are operational in nature, not strategic. They focus on day-to-day tasks required to ship products on time and keep customers happy. While most organizations want to take a long-term view of the business, most are too busy fighting fires to focus on the big picture. And their corporate culture and funding processes undermine scorecard initiatives before the first metrics are even published. To ensure the success of a balanced scorecard, organizations need to excel at managing change, or rather, getting an organization (and the individuals that comprise it) to change habits for addressing and solving problems. Rather than address the symptoms of issues, a scorecard requires organizations to identify the core drivers of change that lead to new levels of performance. Tue, 13 May 2014 12:24:40 -0700 Hadoop Will Not Kill the Data Warehouse Hadoop advocates know they've struck gold. They've got new technology that promises to transform the way organizations capture, access, analyze, and act on information. (See Big Data Part II: "Hadoop 2 Changes Everything.") Market watchers estimate the potential revenue from big data software and systems to be in the tens of billions of dollars. So, it's not surprising that Hadoop advocates are eager to discard the old to make way for the new. But in their haste, some Hadoop advocates have plied a lot of misinformation about so-called "traditional" systems, especially the data warehouse. They seem to think that by bashing the data warehouse, they'll accelerate the pace at which people adopt Hadoop and the "data lake". (See "Big Data Part I: Beware of the Alligators in the Data Lake"). This is a counterproductive strategy for a couple of reasons. Thu, 10 Apr 2014 14:41:00 -0700 To Explore Hadoop, Find a Hunk Say you have a ton of data in Hadoop and you want to explore it. But you don't want to move it into another system. (After all, it's big data so why move it?) But you don't want to go through the hassle and expense of creating table schemas in Hadoop to support fast queries. (After all, this is not supposed to be a data warehouse.) So what do you do?? You Hunk it. That is, you search it using Splunk software that creates virtual indexes in Hadoop. With Hunk, you don't have to move the data out of Hadoop and into an outboard analytical engine (including Splunk Enterprise). And you don't need to create table schemas in advance or at run time to guide (and limit) queries along predefined pathways. With Hunk, you point and go. It's search for Hadoop, but more scalable and manageable than open source search engines, such as SOLR, according to Splunk officials. Mon, 17 Mar 2014 19:22:36 -0700 Big Data Part I: Beware of the Alligators in the Data Lake As silver bullets go, the "data lake" is a good one. Pitched by big data advocates, the data lake promises to speed the delivery of information and insights to the business community without the hassles imposed by IT-centric data warehousing processes. It almost seems too good to be true; and it is. With a data lake, you simply dump all your data, both structured and unstructured, into the lake (i.e. Hadoop) and then let business people "distill" their own parochial views within it using whatever technology they feel are best suited to the task (i.e. SQL or NoSQL, disk-based or in-memory databases, MPP or SMP.) And you create enterprise views by compiling and aggregating data from multiple local views. The mantra of the data lake is think global, act local. Not bad! Wed, 12 Mar 2014 16:49:04 -0700 RedRock BI Rocks the Cloud The cloud eliminates the need to buy, install, and manage hardware and software, significantly reducing the cost of implementing BI solutions while speeding delivery times. One new company hoping to cash in on the movement to run BI in the cloud is RedRock BI, which offers a complete BI stack in the cloud starting at $2,500 a month for up to 2TB of data. The service runs on Amazon EC2, leverages Amazon RedShift, and comes with a single-premise cloud upload utility, 120 hours of Syncsort's ETL service, a five-user license to the Yellowfin BI tools, and five hours of RedRock BI support. Tue, 04 Mar 2014 17:21:44 -0700 SiSense Prism: The New Kid on the Visual Discovery Block Data visualization vendor Tableau Software is the darling of the BI industry these days, giving daily doses of heartburn to established BI vendors. Yet, Tableau is being chased by newer vendors with innovative technologies that offer the promise of even faster, better, and cheaper BI for business users and analysts. One such vendor is SiSense, an Israeli firm now based in New York City, which launched in 2010 after six years of stealth development. Like Tableau, SiSense Prism is a Windows desktop tool that users can download and install from the internet. But unlike Tableau, SiSense was designed from scratch with a scalable, memory-optimized columnar database that can comfortably handle terabytes of data and dozens of concurrent queries. Thu, 20 Feb 2014 11:49:53 -0700 Rollup and Rollout: Actian Eyes Big Data Every database vendor of note has shipped a big data platform trying to capitalize on the current market opportunity and buzz surrounding big data. This includes industry heavyweights, such as IBM, Oracle, SAP, Teradata, Hewlett-Packard and Microsoft, but also many less-known vendors who are hoping to carve out a profitable niche in the emerging market that combines Hadoop with traditional database management systems and data integration and analytics software. Tue, 11 Feb 2014 10:47:25 -0700 Newcomer Decisyon Delivers Next-Generation BI In 2013, you wouldn't think the market (or your organization) could possibly make room for another business intelligence (BI) vendor, but think again. I just took a look at Decisyon, Inc., whose flagship product, Decisyon 360, offers an integrated BI and planning tool with a modern architecture (e.g. all Java and HTML5) and integrated functionality built from the ground up for collaboration, transactions, the cloud and mobile. What more could you ask for? Mon, 10 Feb 2014 08:25:21 -0700 The Battle for the Future of Hadoop: Cloudera versus Hortonworks Big data begs a big question: does Hadoop replace your enterprise data warehouse or augment it? The two leading vendors of Hadoop distributions offer very different answers. For Cloudera, the first vendor to commercial a Hadoop distribution, the answer is an unequivocal yes. Last November, Cloudera finally exposed its true sentiments by introducing the Enterprise Data Hub in which Hadoop replaces the data warehouse as the center of an organization's data management strategy. In contrast, Hortonworks takes a hybrid approach, partnering with leading commercial database vendors, such as Microsoft, Teradata, Red Hat, HP, CSC, NetApp, and SAP, to create a data environment that blends the best of Hadoop and commercial software supporting existing data warehouses, among other things. In short, Cloudera offers revolution, Hortonworks evolution. Thu, 06 Feb 2014 17:24:26 -0700 Integrated Visual Discovery: MicroStrategy Analytics Desktop Visual discovery tools have surged in popularity because they are quick to deploy, easy to use, and don't require upfront IT involvement or a sizable capital investment. Upstart BI vendors, such as Tableau, QlikTech and Tibco Spotfire, have led the visual discovery charge, but enterprise BI vendors are now jumping into the game, offering comparable tools that have the added benefit of integrating with their enterprise BI platforms. MicroStrategy was one of their first enterprise BI vendors to offer visual discovery capabilities two years ago. But unlike pureplay offerings, MicroStrategy Visual Insight was not a standalone, desktop product that customers could buy and download via the internet; they had to first purchase and install MicroStrategy's Enterprise suite, not an inexpensive or speedy proposition. Or they could use MicroStrategy Express, a cloud-based front-end to MicroStrategy Enterprise. Sun, 02 Feb 2014 12:22:23 -0700 Time to Watch Datawatch While a number of established BI vendors are showing signs of duress due to the market shift from reporting to analytics and the onslaught of sexy, new visual discovery vendors, one old-time BI vendor that is bucking the trend is Datawatch, a public company whose stock price has skyrocketed from 11 to 33 in the past nine months. Founded in 1985, Datawatch for many years has toiled out of the limelight selling decidedly unsexy "report mining" tools, which enable customers to extract and integrate data from print spools, ASCII files, EDI feeds, XBRL and PDF documents and just about any other formatted document or report as well as relational sources and output the results into BI tools, Excel, HTML, and other interactive environments. Mon, 27 Jan 2014 16:10:06 -0700 Adding Value to Big Data: Infochimps Raises the Bar Big data , by itself, is useless. Think about it. Data is simply a stream of 1s and 0s. It's not until you analyze data or use it to support or automate a business function that you gain any value. To date, the big data movement has focused on technology and components and how to make them easy to use. The big software and systems providers--IBM, Oracle, SAP, EMC, Hewlett Packard, Microsoft, and Teradata--have all recognized the changing tide and raced to acquire, assemble, and integrate various big data technology and components. But the next wave of competition is delivering real business value with these newly architected big data platforms. Wed, 22 Jan 2014 13:33:20 -0700