|
Blog: Richard Hackathorn« June 2008 | Main | August 2008 » July 25, 2008SAS DataFlux Scrubs the EnterpriseAt the Boulder BI Brain Trust, we heard a presentation from SAS Dataflux about their company background, product offering, and future directions. They are a provider of data quality tools since 1997. See the complete blog item here.
July 24, 2008Microsoft Acquires DATAllegro: DW Appliance Market Jumps to Center StageWith Microsoft's announcement today of their acquisition of DATAllegro, the marketplace for data warehouse appliances (DWA) jumps to the central stage of IT drama. What began as a few emerging start-ups challenging a few big DW vendors has morph into industry game-changer. I can grasp why Microsoft would sell a wireless mouse here and there. However, a high-end MPP DWA puts Microsoft into the center of the appliance business by delivering soup-to-nuts solutions for large corporations. I never thought that Microsoft would ship real 'iron'. Well, I am now wrong. Perhaps a corporate name change is in order. How about Microhard? (Actually, there are several 'Microhard' companies: one that does OEM wireless modems, and another that does certification training of Microsoft networks.) I would also imagine that this action will solidify the definition of 'appliance' to mean a real end-to-end solution, which will require a mixture of hardware and software. Thus, product positioning as a software-only appliance will likely to be an endangered species. In his blog, Stuart Frost notes that acquisition will revitalize the venture capital industry to fund emerging database technology, as was the case two decades ago. This will make the small innovative start-ups more viable, and their conversations with larger corporations more interesting. In a phone interview, Stuart elaborated on several aspects of the acquisition. Although avoiding any actual amounts, he said that it was a 'very strategic price' that Microsoft paid for DATAllegro. They were initially approached by Microsoft to do a partnership. DATAllegro responded with ideas of putting SQL Server inside, instead of Ingres. The acquisition discussions proceeded from there. Stuart was excited to start a new life as part of the Microsoft team. He said that he was 'been there & done that' with several start-ups, and it was time to move into different challenges, such as guiding Microsoft into EDW. He emphasized that Microsoft is taking the long-term view of this endeavor and will be flushing out whatever capabilities are required for the EDW marketplace. Stuart was pleased that Steve Ballmer mentioned the acquisition in the context of Microsoft's initiative with enterprise search, a mixture that is quite exciting to large enterprises. More will be discussed at the Microsoft BI Conference in October. So, is Netezza next to be acquired? ...by Oracle? Related blogs by colleagues: William McKnight, Colin White, Krish Krishnan
July 21, 2008Teradata does Enterprise Data WarehousingI attended a two-day briefing at the Rancho Bernardo facility of Teradata. We had a full schedule of talks by cross-section of marketing and technology people. Here are a few glimpses of the event... Randy Lea, VP of Products and Services, started by sketching their corporate marketing message. We had a good discussion about the concept of Enterprise Data Warehouse (EDW). Teradata has argued that all analytic data should be centralized into a single EDW and then leverage the biz value embedded in the cross functional relationships. While that goal is still valid, it must 'bend' to the realities within all businesses, even to the extreme of having independent data marts to support applications required NOW. See the TDWI BI Journal article on Agile DW using an integrated sandbox within the EDW. Dave Schrader, Director of Marketing and Strategy, give a spectrum of business examples focusing on the front-line, back-office, middle management. It was a fascinating tour of innovations that are dramatically changing some industries. Dan Graham, Enterprise Integration Program Manager, drilled into the cloud of technical pieces of an EDW environment. Think PPT slides filled with hundreds of acronyms; however, think about the importance of executives to understand the biz implications of this swirling cloud. Dave Klumb, VP of Global Field Operations, gave an overview of the Professional Services organization at Teradata. It was interesting to see what their emphasis Todd Walter, CTO, took us a quick walk through new product features and ended with a great group discussion on 4-5 industry issues. For instance, we kicked around the biz tradeoffs of integration versus flexibility in the EDW. Tom Russell, VP of Data Architecture and Modeling Solutions, spoke on an 'integrated' data architecture for EDW. He reminded us that the original goals of data warehousing has been independence of data from application, logical centralization of data, and timely acquisition of data. In large DW shops, there are many BI tools, which should be integrated into the same base tables. The focus is the mapping between the usual logical/physical database design (e.g., ER diagram in ERwin) and the semantics of the BI applications (e.g., specfic star schema). Mike Goul, a professor at the Arizona State University, described the Teradata University Network (TUN). Teaching Info Sys requires a 'touch of reality' inserted into the classroom. TUN has over 2,000 faculty from 900 universities in 70 countries. There are case studies, BI tools like MicroStrategy, and million-row data warehouse. I wished other vendors were as receptive to the tangible support for universities. Mark Shainman, Senior Program Manager for Master Data Management (MDM), explained the nature and importance of MDM, along with Teradata's MDM offering. He argued that MDM is a critical piece of the EW infrastructure, which most do in a fragmented way. Mark urged that relationships among and within products, customers, suppliers, and other master data should be managed by a single infrastructure. Ron Swift, VP of Cross Industry Solutions, gave an overview of the horizontal offerings of Teradata, which has traditionally been vertically oriented (e.g., telcom, financial, retail). Ron noted that past innovations in an industry are now being applied in other industries. The common aspects of those innovation become the seeds for horizontal solutions, such as financal performance, customer mgt, and demand chain mgt. Ron shared some wisdom with the statement "all the reporting does not change our decisions", implying that learning from the past does not change how we react to the decision of today. Reflection... Teradata continues to be the major EDW vendor to large global firms. With their new lower end offerings, Teradata has the challenge of balancing their thrust toward centralized EDW with responsiveness to new biz requirements. There is sufficient maturity with EDW that there is increasing pressure to extend EDW with more agility, flexibility and responsiveness. Now that is a glimpse of the future for EDW!
July 7, 2008Kognitio brings flexibility to complex analyticsAt the Boulder BI Brain Trust, we heard a presentation from Kognitio about their company background, product offering, and future directions. They support high-performance analytics using a scalable MMP architecture. See the complete blog item here.
July 4, 2008IAP: The Word Cloud for the IAP WeekShawn Rogers blogged about Wordle.net and gave a word cloud of his blogs. This is what I got when I submitted my collection of blogs. So, if you want to know what the Independent Analysts Platform did this last week, just ponder the following illustration.
July 3, 2008IAP: Reflections on Independent Analyst PlatformThat was quite an experience! Doing 20 blogs in 3 days about the one-hour presentations delivered by 18 BI/DW vendors. The 'real-time' blogging has the advantage of focusing your attention, but it has the disadvantage of reducing your ability to reflect on and analyze the content. The result is that the blogs often had a 'vendor hype' tint to them. Perhaps I can do better next time... What stood out? The presentations where the CTO (or equivalent) cuts out the marketing pitch and clearly tells what was happening internally, who was pushing the limits, and where there were gaps. Several vendors displayed this candor. Most unfortunately did not. Some notable highlights were: + Composite Software with their Discovery Appliance. David Besemer gave an insighful demo. Rick van der Lans did a super job herding us cats and bringing all the logistics together. For a glimpse at the cast of characters, view our homemade video that congratulated Rick on the event. My blogs about IAP are collected here. There was definitely synergism flowing among this group of 'independent' analysts. If you are an analyst with similar interests, please contact us. The analysts attending are: Peter Aiken, VCU & Data Blueprint
July 2, 2008IAP: Ingres Thinking Software Appliances to Deliver BIMike Boyarski, Director of Product Mgt of Ingres, talked on Appliance-Based Computing for BI. He has working at Oracle for eleven years and then to On-Demand. Mike mentioned the long history of Ingres. However, the current goal is to be the enterprise-class open source database. A number of companies are contributing to the open-source base, such as DATAllegro contributing hash key encoding, sort performance and partitioned indexing. The academic institutions are doing significant database research on the code base. He remarked that the market fits better with the mid-market companies. When compared with MySQL, Mike inserts that Ingres is better in performance. Companies that are basing products on the Ingres codebase are: Pentaho/JasperSoft, Business Objects, DATAllegro, GoldenGate. Ingres tends to be a good technology alternative for the SaaS market. The rapid innovation of open-source software has become a problem for most data centers that require more stability. Mike defined Appliance-Based Computing as a “complete pre-integrated and standardized BI software stack”. He asserts that hardware is not part of the appliance, despite the sensitivity of a BI system to processor/network balance. The Ingress Icebreaker BI Appliance consists of the following stack: Jasper, Apache/Tomcat, Java, Ingres 2006, Linux. Mike characterizes that this system results in a capability similar to Business Objects two years ago.
IAP: Dataupia Frees Your DataJohn O’Brien, CTO and co-founder of Dataupia, and Samantha Stone, VP of Marketing, remarked that their tag line is to 'free your data' to provide open access to data for everyone. The goal is unlocking the power of data for business users. Dataupia has just experienced the one-year birthday from their first product shipment. Their product is the Satori Server, which John asserts is a true DW appliance, since it is shipped as a hardware/software integrated product. He said that their product is non-disruptive because their product brings the power of a MMP database to Oracle, Microsoft SQL Server, and IBM DB2 (called OMI as a shorthand for the three database engines). It works with the customer’s existing applications and existing databases. Pushing toward core IT processing, John said that their target market is the data warehouse mainstream that consists of: + long term data archiving John gave the example of the Dataupia architecture as being the backend to OMI to provide the parallelism to existing applications without changes. Their storage is custom designed based on records, rather than blocks. It is record-oriented data store. John asserts that their record-orient database has the same performance levels as column-oriented databases. They are not using compression at the data item level, but do utilize it for aggregations, which often result in 80% compression. John outlined the requirements of several current customers, such as Subex, Sendio, Tektronix (network traffic ) and Focus (teleco). The main advantage of Dataupia seems to be their transparency at the application level by being a backend to OMI database engines. They have extended their SQL functionality to correspond to the uniqueness of the front-end.
IAP: Teradata Continues To Improve Their StuffDave Schrader, director of Strategy and Marketing at Teradata, returned for a second hour and give an overview of what is new at Teradata. He quickly focused on what was new with Teradata, which he was divided into the following seven areas: 1) Data mart consolidation: emphasized the value of cross function data, so combining data marts has great value potential than separate ones. 2) New Platforms: quick overview of the product family: 550, 2500, and 5550 (A side remark was that Teradata now has three customers that have over a petabyte of data.) 3) Database Improvements: moving more functionality into the parallel database, 4) Master Data Management: i2 capability was folded into the MDM product. 5) Application Extensions: + Teradata Relationship Manager added AssetLink to support Market Resource Mgt (where are the dollars going), extended campaign mgt with an arbitrary number of steps, and support of mobile marketing/sales. + Financial and Performance Management partnered with Hyperion for more metrics, integrated with SAP, 6) Partnerships: The big 3 for Teradata are SAS, Microsoft, and SAP. SAS partnership focused on consolidation of their data sources and embedded SAS analytical processing into the Teradata database. SAP partnership focused on keeping the SAP interface but using Teradata for the larger cube creation and other analytics. Microsoft partnership focused on using Microsoft analytical services on top of Teradata, to provide high end solutions. 7) Education and Services: reaching out to various groups more.
IAP: Pervasive Moving Forward with Data IntegrationMike Hoskins, CTO and General Manager of the Data Integration for Pervasive Software, described his company as an established mid-level software firm. Pervasive started in 1982 as SoftCraft selling the Btrieve database, which is still being used in many places in the world. Another early data integration vendor, Data Junction, was acquired in 2003. Over the years, Pervasive has become a leader within the data integration market characterized by their 'agile embeddable integration'. Pervasive products are: DataIntegrator and DataProfiler. There is flexibility for embedding within a SOA environment. In the future, the Pervasive data services platform allows modular configuration through a data mediation services. A Metadata Studio is being developed to collect metadata from a variety of products and stored it into a common repository, to be managed from the studio. DataRush is being developed to solve the larger data integration problems.
HP: Neoview Designed for Heavy Operational BIGreg Battas, Chief Technologist for the BI Group, works for Ben Barnes in HP Software Group. He started with a basic definition of Neoview, which is integrated H/S warehouse platform by NP, built out of low cost commodity parts. It supports EDW, ODS and 'challenging' data marts, by having shared services, mixed workload, 3 to 300TB of raw data, complex query processing, and high SLA. Neoview is winning in situation of mart/DW consolidations, Teradata migration (replacement or dual vendor strategy, offload for high SLA), and outgrown Oracle implementations. Greg summarized their positioning as being the 'extreme sports' guys of BI. BI is evolving! It is no longer my mother’s data warehouse, which has the simple left-to-right flow from input data to output reports. Now, the DW has many operational systems feed and feed off of the DW. Doing things quickly is a critical requirement. Greg used the analogy that HP has been known as an 'arms dealer' that sells stuff to all the key BI vendors, without having a BI solution of their own. This has changed with the development of Neoview. Greg gave a concise history of the evolution of Neoview from a rewrite of Tandem for high-end performance requirements. The first test bed for Neoview was HP’s internal IT with Randy Mott’s team. Neoview is designed for operational BI for hundreds of terabytes with 24x7x265 always-on, fault tolerant, and black-box administration. Greg mentioned that, to get a rough estimate, he went to the HP.com site and configured a server with 100+ processors and many TBs, all of standard parts. Give a heart attack to the web analytics folks! Workload flexibility pipelined data flow to handle a wide spectrum from big complex queries to short, high-SLA queries to long-running queries, as shown next.
As an example of extreme DW requirements, a teleco firm required the ingest of 3B rows into 72 B rows of detail data, while also incremental extract of 3B rows for fraud detection, concurrent aggregate build, a massive affinity analysis, a large adhoc report, and 24 K queries from customer service reps who need 3-second responses. There are no windows to introduce new features or fix bugs. The world is not moving rapidly toward batch. The system must be up all the time. Neoview has several partner integration projects, such as embedding SAS algorithms embedded into the Neoview engine. Both ETL and ELT are bad approaches. What is really needed is to take the data transformations and other data integration pieces and place this DI processing close to the file/storage system. The normal query execution has dual access to the data. With the acquisition of Knightsbridge and recently EDS, HP has the ability to provide enterprise-level professional services to augment their product offering of Neoview.
IAP: Kalido Provides Context to Business InformationBill Hewitt, president and CEO of Kalido, showed how companies can get their hands around the changes in their markets. Does IT help companies get better? Bill summarized their focus as "Information should be delivered as a service, where producers and consumers will create/consume/distribute information in near real-time." By the time that data gets to the proper people, the information is too late, too old. Managers need the context and the relevance for their business. The Kalido Information Engine consists of: + Business Information Modeler Robert Dickson, principal resales consultant, continued with a detail demonstration of augmenting a Cognos tool with enhance metadata. In particular, the field of Reseller Type was added as an attribute of Reseller so that users can query on this new field. It is interesting that Kalido does not supply a database engine (since it uses SQL Server, DB2 and the like) or query tools (since it uses Cognos, Business Objects, and the like). It automates the metadata management layer above database/query products so that the data warehouse is more responsive to changing business requirements. P.S. During the break David Loshin and I discussed the Kalido presentation and felt that it was the best so far. The reason is that the CEO came and give the value proposition, all in 20 minutes. Then, a knowledgeable technical person gave a detailed demo with real product, all in 30 minutes. And, the demo actually worked!
IAP: Lumigent Technologies Secures/Audits Your Compliance PoliciesIt is Day 3 of IAP! I did not think I would make it, but here we are. There are only seven more presentation to go. Roger Hodskins, VP of Marketing and Alliances, described Lumigent as providing solutions that secure their business application data from unauthorized access, eliminate vulnerabilities, and assure privacy. The market niche is labeled GRC - Governance, Risk, and Compliance. Now that is a mouth-full. Does it really mean anything? Roger remarked that audit gets a bad rap. The purpose of an audit is to ensure that proper controls are in place within your company. Next, he mentioned compliance, which is usually imposed by government agencies. The levels of compliance are: regulations, governance framework, control framework, and finally the actual controls. Consequences of weak compliance ranges from going to jail to the loss of customer confidence. The steps in compliance are: + discover: uncover location/paths to be watched Roger used the analogy of being the surveillance camera, rather than a guard. A real story, first reports from their product indicated that the DBA was doing changes during production hours, which was a violation of company policy. A call from the VP to the offender stopped that practice by all DBAs within a week. The market for compliance solution is estimated by Forrester to be 1% to 5% of annual IT application expenses. Using this percentage along with other measures, the market for compliance solutions is in the billions! The magic occurs in three stages of: policy engine, auditing engine, reporting/alerting. Data is collected from three sources: + native auditing of the application (such as Oracle Financials) A unique feature of Lumigent solutions is to determine that a database operation was performed by a certain user, along with before/after values. This is possible because of close partnerships with application vendors (Deltek for government project mgt, Motorola for criminal justice, Trizetto for health care, and the like) Some customers are generating over a terabyte of compliance data per week! A large stock exchange has hundreds of millions of transactions per day. Pricing errors can create huge liabilities very quickly, which the Lumigent solution can detect quickly.
July 1, 2008IAP: Progress as DataDirect for Data ServicesAs the second part of the Progress Software briefing, Mark Troester, director of product marketing, talked about their role as a supplier of data access technologies via their DataDirect Connect, XQuery, and Shadow (from NEON Systems). DataDirect is an independent operating unit of Progress Software. Mark requested that his discussion be considered confidential, so this blog ends. Sorry...
IAP: Corizon Does Rapid Enterprise Applications With MashupsEdwin van der Sanden, CTO of Corizon, talked about composite applications (read 'mashups') within the enterprise. The architecture spans a User Interface, composition, reusable services, and connectivity into legacy systems and web services, as shown in the following figure.
The tool set consists of: Corizon Studio and Composer, the UI Service Provider, US Service Builder, and UI Service Extractor. Corizon offers three deployment options: stand-alone, embedded in packaged app, and as a portlet. A concise product overview is available here.
IAP: DATAllegro Highlighting Their Grid TechnologyMark Theissen, COO of DATAllegro, remarked that he is responsible for everything that touches a customer. The company received $19.6M of additional funding in May for a total of $65M. They have experienced 300% growth annually and have 100 employees. Mark distinguished their product as a second-generation of data warehouse appliance (DWA) that uses industry standard hardware and software, such as EMC storage, CISCO networking, Dell server, Ingres database, along with tool partners like SAS, Business Objects, MicroStrategy, Cognos, and Informatica. Mark asserts that they have "ultra shared nothing" architecture, as shown in the following diagram.
Mark said that a key competitive advantage for the company is grid technology that can move data at 500 GB per minute. They offer three different configurations: hub-and-spoke, multi-temperature, and in the future disaster recovery. Teradata integration is important because DATAllegro has perked the interest of many Teradata customers. The use cases are: batch accelerator, BI offload, and online archive. They have four Teradata customers currently. The key is to get the data out of Teradata quickly. When compared with the new Teradata appliance offerings, Mark felt that DATAllegro offers same capability at a lower price. Further, he felt that the Teradata offering are underpowered with reduced functionality.
IAP: Teradata Goes Active Enterprise IntelligenceDave Schrader, director of Strategy and Marketing at Teradata, presented the framework of Active Enterprise Intelligence (AEI). The role of the Enterprise Data Warehouse (EDW) is to integrate cross-functional data. AEI is the next step beyond EDW, involving the alignment of strategic intelligence with operational intelligence and the opposite. This results in value from cross analytics, along with value from AEI to support accelerated decisions. The AEI architecture consists of sex key features: active access, active loading, active events, active integration with front-end systems, active workloads, and active availability. Dave motivated the technical aspects with a variety of real world examples. He stressed the need for intelligent decisions at all levels of the company and noted that being intelligent is not always obvious. For example, telecomm firm monitor the balances of their subscribers to detect unusually large amounts and then proactively contacting subscribers to assist them in paying their balances. This results in lower debt and greater customer satisfaction. Technology is not the limiting issue. Of the triad of People-Process-Data, the people and process are the issues that need attention in today’s business environment. Teradata differentiates on their AEI techncial framework, which was illustrated with several customer case studies.
IAP: Microsoft Hides BI in SharePointKristina Kerr, Senior Product Manager at Microsoft, presented the topic of Business Intelligence. Kristina started with a classic Dilbert cartoon (dated 5-14-07) about the futility of using dashboards at least by the pointed-hair manager. With a background with Crystal Reports and Business Objects, she became interested in Microsoft’s efforts with dash boards. Customers are thinking bigger than BI and even bigger than ERP. The vision of Microsoft is to providing business insights to ALL employees leading to better, faster, and more relevant decisions. Typical day for information workers involves email and the like, but never do they explicitly deal with BI. The best BI is that which is hidden under Excel, email and such. She showed a table with two dimensions of Scope as personal to shared and Development as organic to intentional. The result in many current situation is confused or frustrating. At the personal-organic, it should be self-service. At the shared-intentional, then it should be performance management in terms of KPIs. Then, the result is empowered for personal-organic or aligned for shared-intentional. The implication is three levels of context for BI: personal BI, team BI, organizational BI. Kristina walked through an extensive scenario, starting with SharePoint scorecard for AdventureWorks sales. By clicking on various elements, relevant data would appear in other windows. SharePoint maintains a live data connection along with the proper permissions for the user. SharePoint also does a search across all metadata returning a list of objects, such as reports, people and a variety of things. Excel is extended with data mining functions, like analyze key influencers, detect categories, fill from example, forecast, highlight exceptions, and what-if scenarios. The point is that a lot of BI is being hidden within SharePoint, controlling security.
IAP: Progress as a Complex Event Processing VendorHub Vandervoort, CTO of Progress Software, presented. The company gets little recognition in the industry because 65% of revenue comes from OEM partners. He talked on the topic of Faster than a Speeding Bullet, which illustrated complex event processing within SOA as velocity within a pipeline. He illustrated the concepts and problems with a series of examples from manufacturing to equity trading. The obvious issue is how fast or fat is the pipeline. However, most overlook some key side-pipelines. A pipeline requires information provisioning that interacts with complex data with the packets flowing on the pipeline. Another side-pipe is the business visibility that reports on the status of the pipeline’s operation. One must alignment (balance) the velocity of the main pipeline with the velocity of these two side-pipes. Poor alignment causes higher error rates in the business operation. As an example, equity trading was 3-day settling period, which has been reduced to two hours. However, the update cycle on the data warehouse stayed with 24 hours. And, the reporting on those trades also stayed with daily cycles. The margin of safety must be as least 3x, implying that a two-hour latency in a pipeline requires at least a 40-minute latency in both the information provisioning and in the business visibility. Hub asserted that the margin should be 10x to 100x if a company is concerned about compliance. Why SOA? First, it is the basis for the event-driven pipeline. Second, it supports asynchronous, distributed and federated requirements. Third, it permits the non-invasive insertion of real-time side-pipes with less disruption. Progress Software is acquiring Iona Technologies to expand its service-oriented architecture product offerings. With its expertise in CORBA object request broker with pub-sub features, Iona has sold Progress products for years to bring smart endpoints together with smart networks.
IAP: Informatica Moving Beyond Traditional Data IntegrationWe have started the second day of the Independent Analyst Platform marathon! Karen Hsu, director of product marketing, gave the first presentation for Informatica. She started by describing a series of new features, such as the Velocity Methodology that combine operation and analytical data integration. Enterprise Data Integration (EDI) has coverage of most traditional data sources, along with cloud computing (external data resources based on SaaS), such as SalesForce.com and with Partner Trading Network (B2B). A new product is the support of operational data integration, where data latency is measured in terms of minutes and seconds. The point being is that EDI needs to support the entire spectrum of data latency. An interesting point was that their new capability for work flow management is required for operational data integration. Not convinced yet! But keep my mind open. Keep my mind open. Is Informatica spreading its wing beyond its data integration focus? It seems that one could have work flows with Informatica’s product. The recent acquisition of Identity Systems has provided capabilities with identity resolution. A good discussion followed, but I was unable to capture the key points. Robert Hipps, VP of R&D, then explained the B2B Data Exchange that was announced at the recent Informatica conference. Through an acquisition of ItemField in 2006, Informatica secured the ability to exchange standardized industry-specific structured data, such as HIPPA. It appears as another node within Power Center. It is all about ‘trusted integrated business partner data’. In summary... A key slide showed Informatica’s strategy as three vertical panels. The middle panel is EDI, which is the traditional data integration of the enterprise, with the additional of unstructured data in PDF, Word and the like. The left panel is the SaaS cloud that brings external data into the enterprise. Finally, the right panel is the B2B partnership that allows the enterprise to plug into the business partners of their value chain. Got the image? Hence, Informatica’s perspective of data integration moves beyond the enterprise to its greater context in the industry.
|
