Blog: Wayne Eckerson Welcome to Wayne's World, my blog that illuminates the latest thinking about how to deliver insights from business data and celebrates out-of-the-box thinkers and doers in the business intelligence (BI), performance management and data warehousing (DW) fields. Tune in here if you want to keep abreast of the latest trends, techniques, and technologies in this dynamic industry. Copyright 2020 Sun, 26 Oct 2014 08:35:45 -0700 From Data Dumps to Data Wrangling In the good old days of data warehousing, the information technology (IT) department was king and business analysts were... well... locked out. If business analysts needed more or different data than was in the data warehouse, a data mart, or a report, they had to beg and plead IT to create a data set for them. Often, they had to wait days, weeks, or months for a data dump, and in the meantime they scrambled to find crumbs of relevant data and piece it together using Microsoft Excel or Access. Not surprisingly, research shows that business analysts spend at least a third of their working hours finding, cleaning, merging, and aggregating data--a huge time and cost sink that most organizations don't recognize. Sun, 26 Oct 2014 08:35:45 -0700 One Stop Shopping for Microsoft BI Customers Buying business intelligence (BI) from Microsoft is a little like buying your child a mail-order bike for Christmas. When you open the box late Christmas eve, you see umpteen parts, nuts, bolts and various accessories that you quickly realize will take all night to assemble, if you are lucky. Although Microsoft offers some very good BI capabilities, they are built into a number of different products (Excel, SQL Server, Sharepoint, etc.) and do not come as an integrated bundle. First, you need to purchase the right versions of each Microsoft product and then assign several developers to wire the BI capabilities of the products together to ensure a seamless user experience. So, although Microsoft doesn't charge for BI, it takes a lot of time and money to get a Microsoft BI implementation up and running, especially in an enterprise environment. Wed, 01 Oct 2014 16:53:15 -0700 Dundas Does BI Dundas Data Visuailzation, Inc. is about to complete its transition from a provider of charting components to a full-stack business intelligence (BI) vendor. Founded in 1992, the Toronto-based company became one of the leading providers of charting and programming controls. Seeing an opportunity to move "up the stack", Dundas sold most of its components to Microsoft in 2007 and focused on building a dashboard product, which it shipped in 2010. Capitalizing on its visualization heritage, Dundas quickly built a following for its Dundas Dashboards product. It now has almost 1,000 customers in 53 countries, including many Fortune 500 companies. Recognizing that many customers were using its dashboard product for other BI functions, Dundas recently decided to build a BI suite, called Dundas BI, that it plans to ship by end of year. The suite will support ad hoc reporting, analysis, dashboarding, scorecarding, visual discovery, and a data mashup utility along with an in-memory database. Tue, 30 Sep 2014 11:25:11 -0700 SQLstream Bets on the Internet of Things Stream processing has idled on the backwaters of the analytic market for years. But with the advent of Hadoop and new open source streaming tools, such as Storm, Spark, and Kafka, many companies are taking a closer look. And many stream processing tools are finally finding a home with the Internet of Things, in which consumer and commercial devices--from smartphones and household appliances to automobiles, utility meters, and medical equipment--emit millions of events per second and require specialized analytical systems to process them in real time. Stream processing platforms, like SQLstream, provide both the horsepower and smarts to filter, aggregate, group, compare, and analyze large volumes of data in flight as well as visualize the results in real time. Telecommunications companies use SQLstream to monitor network performance, track service usage, and detect fraud in real time; oil and gas producers use it to monitor operations of drilling rigs, digital wells, and intelligent oil fields;; and transportation companies use it to monitor traffic congestion, among many other things. Tue, 16 Sep 2014 11:22:03 -0700 Think Entrinsik for Operational BI: It May Be All You Need Sometimes the best things come in small packages. If you want to create reports and dashboards directly from source data, then you might want to consider Entrinsik, a small business intelligence (BI) vendor with a venerable pedigree and highly satisfied customers. Founded in 1984, Entrinsik got pulled into BI in 2002 when it built a Web reporting tool for customers using MultiValue databases, such as UniVerse and UniData, which store attributes as strings in the same table as the entity. This denormalized structure makes it difficult to generate reports quickly or easily. But Entrinsik cracked the MultiValue database nut and quickly became the go-to-player for MultiValue reporting tools. Since then, Entrinsik has expanded into the SQL market. Its Informer product queries both SQL and MultiValue databases. Mon, 01 Sep 2014 12:05:50 -0700 Cyberscience: When Your Data Warehouse Does Not Cut It In the 1980s, before the dawn of data warehousing, companies ran reports and queries directly against operational systems. But this analytical activity undermined the performance of core systems and created a rallying cry for a dedicated reporting and analysis system, known as the data warehouse. But what if you could have your operational cake and analytics, too? That's the fundamental question that Cyberscience asked back in 1977 when it was founded. And while the industry chose data warehousing as the architectural solution to manage reporting and analytical workloads, Cyberscience went against the grain. It kept optimizing the federated query technology of its Cyberquery product to run against operational applications and databases without degrading performance of either. Mon, 01 Sep 2014 11:51:26 -0700 Logi Analytics Reinvents Visual DIscovery For many years, Logi Analytics carved out a profitable niche selling an easy-to-use report and dashboard development platform to information technology (IT) professionals. But its recent foray into analytics--with Logi Vision, a visual discovery tool that first shipped in January--targets business analysts. Staying true to its commitment to ease of use, Logi Vision brings a fresh new perspective to analysis by making heavy use of heuristics. The tool makes it easy for business analysts to shape, analyze, and visualize data without IT assistance. For instance, the tool automatically identifies data types, concatenates related fields into a single data object (e.g. city, region, and country into location), rearranges columns based on relevance ratings, and suggests visualizations based on the contours of the data, among other things. Tue, 26 Aug 2014 10:00:16 -0700 MetaScale Offers an OnRamp to Hadoop If you're a big data trailblazer, one way to monetize your risk is to offer your services to those following in your footsteps. This is exactly what Sears Holding Corporation has done. In 2012, Sears Holdings formed MetaScale to offer big data consulting and managed services to large companies in any industry, although it has deep expertise in retail. MetaScale runs a Big Data Center of Excellence with big data experts who help companies install, implement, and manage Hadoop, NoSQL, and other big data tools and systems. Unlike big data vendors that also offer consulting and managed services, MetaScale is vendor neutral. It can piece together components from multiple vendors and Apache projects into an optimal environment based on a client's unique requirements. It also helps customers conduct proofs of concept, customize a solution to their needs, and manage production environments on a 7x24 basis, either at the customer's site or MetaScale's own hosting center. Sat, 23 Aug 2014 06:24:49 -0700 Dimensional Insight: The Best Kept Secret in BI One of the best, least known business intelligence (BI) vendors is Dimensional Insight. Founded in 1989, the company has done what few BI vendors have achieved in the past 25 years: maintain a growing base of staunchly satisfied customers. Without much fanfare, the Boston-based company has amassed 2600 customers in more than 30 countries, generating about $20 million a year in revenue. According to most independent customer satisfaction surveys, such as those from the Business Application Research Center (BARC) in Germany and Dresner Advisory Service, Dimensional Insights sits at the top of the pack in terms of overall customer ratings. Mon, 18 Aug 2014 15:50:25 -0700 SnapLogic is Ready for the Future Data integration is bread-and-butter technology. Never glamorous or sexy, it holds the key to opening data to business users. Until recently, most data was tabular, stored in relational databases, and pumped in batch from one system to another. Today, data is multi-structured and increasingly managed by cloud applications. Data integration platforms that don't keep up with the times are doomed to an early obsolescence. One data integration that is determined to stay relevant among changing tides of data management is SnapLogic. The San Mateo, California-based firm offers a cloud-based data integration platform that runs both on premise and in the cloud so it can connect any application or data source either inside or outside the firewall. The design, management, and monitoring tools run as multi-tenant applications from HTML5-based browsers and almost any SnapLogic functionality can be triggered by events or scheduled jobs via its REST API. Moreover, the product manages multi-structured data using JSON rather than forcing data flow through a tabular format. Fri, 15 Aug 2014 15:26:02 -0700 Platfora Brings BI to Hadoop Platfora is a three-year old company that is designed to provide real-time, ad hoc analysis of data stored in Hadoop. While most Hadoop enthusiasts are embracing real-time SQL query engines for Hadoop, such as Cloudera Impala, Platfora goes several steps further: it is a full-stack BI tool that comes with data wrangling, ad hoc visual discovery, and analytics, such as segmentation and behavioral analysis. Best of all, Platfora is equipped to blend all types of data, including transaction data from operational systems, interaction data from emails, text documents, and call centers, and machine data from Web and application servers and sensor networks. Platfora is an in-memory, distributed application that runs on Hadoop clusters, either on premise or in the cloud. Developers or highly skilled business use a visual design environment to create "Lenses"--personal data sets with a visual front-end that contain aggregated data from Hadoop. The Lenses run in Platfora's in-memory engine, providing subsecond query response time, allowing business users to analyze data at the speed of thought. lets business users pull in more granular data stored in Hadoop, if needed. Fri, 15 Aug 2014 14:28:15 -0700 X15 Software Tackles Log Data at Scale Silicon Valley startup X15 Software recently shipped a data analytics solution designed to help companies capture, store, and analyze petabytes of machine-generated log files from Web and application servers, among other things. The on-premises solution which runs on the Hadoop Distributed File System provided by major Hadoop vendors, such as MapR, Cloudera, and Hortonworks, supports real-time SQL queries against streaming log data. Called X15 Enterprise, the big data solution competes against log management tools, such as Splunk, which must first pull data from Hadoop into a proprietary engine to index and query the data. And since search-based tools don't leverage SQL, this limits their ability to query both traditional relational data and semi-structured log data in a single pass. Wed, 13 Aug 2014 14:35:42 -0700 Need Hadoop-based Analytics in the Cloud? Try Qubole Qubole is a new breed of analytic software company that runs on Hadoop in the public cloud. The ideal customer is one that already runs a lot of applications in the cloud and wants to accelerate the time it takes to make big data available to business analysts and data scientists. Qubole's big competitor is Amazon Web Services and its Elastic MapReduce (EMR) offering, which provides the Hadoop platform as a cloud-based service. Unlike EMR, Qubole was designed from scratch to support queries and analytics. Also, the service is geared t business analysts and data scientists, not Java and other developers, so it is easier to use. Tue, 22 Jul 2014 06:52:49 -0700 ScaleOut Software Sits at the Nexus of Operational and Analytical Computing ScaleOut Software provides an in-memory data grid that primarily provides fast reads/writes for high-speed operational applications, such as e-commerce, reservation systems, credit card processing, equity trading, smart grids, and cable network streaming. There are a lot of competitors in the space, such as Oracle Coherence, but ScaleOut and Apache Spark differentiates itself by supporting analytics on live, operational data, among other things. Sat, 19 Jul 2014 12:45:36 -0700 BI Survey 14 is the Largest Survey of BI Tool Usage in the World There is no better source for the trends in the use of business intelligence (BI) tools than the BI Survey conducted annually by the German-based research house, BARC. The BI Survey, now in its 14th year, is the world's largest independent survey of BI users, with more than 2,500 survey takers around the globe. This summer, I will help BARC's research staff evaluate the results of the BI Survey, which closes in a few weeks (late June.) The BI Survey tracks customer attitudes towards more than two dozen BI products. The published report discusses key purchasing and usage patterns, including total cost of ownership, market and customer penetration, query performance, support quality, business benefits, and technical challenges, among other things. Mon, 09 Jun 2014 09:05:50 -0700