Recently in 20091018 TD Partners Category

Curtis Kowalski, Chief Data Architect at Xcel Energy, talked consolidating multiple operational data systems (ODS) in an EDW.The goal is to retire all departmental ODSs and independent data marts into an EDW using Teradata, in 3NF. Xcel is doing this incrementally over time, starting with a 'lightly integrated' data warehouse. Hopefully this will enable the organization to the business value of integrated data.

Curtis presented a viable approach to move a traditional IT infrastructure into the new world of integrated data warehouse, especially when this integration involved low-latency data in ODS.

Fun romp through modern times! You had to listening carefully. Blink that you would miss a crucial twist of history. Best way to summarize Stein is with tidbits here and there...

 "Credit Default Bond is a cap pistol hooked to a nuclear bomb. We just do not know whether deficit spending, stimulus funding and the like is working. We just don't know... We just don't know... Ben paused to sip Throat Coat from Whole Foods. "What is the world going to be like when Iraq has a nuclear bomb? What is the world like for my son? Is global warming valid? We are not allowed to debate it? A serious serious serious problem in the collapse of education! I have observed the decline in our education. One fourth of high-graduate is able of only 'C' level college math. Let me tell you about a conversation I had with two young folks... about WWII, Germany, Russia, and where big trees grow! Sad! Our top economic advisers did us a huge disservice.

Ben finished with a tribute to our arm forces and police... "Where is salvation coming? The real stars are young people who are putting on body armor... I visit Walter Reed hospital and then national cemetery. A quote that I always remember... We asked God to bless this great country. In this world, God's work has to be our work.

I had a nice breakfast with the WhereScape folks this morning. The company sells tools to design and build data warehouses. That seems like a function already satisfied by many existing data modeling and ETL products. However, add the business requirement to build a data warehouse is a few days or even a few hours, and you have a different animal. We discussed industry directions and market strategy and did not have the time for a product demo... but will take time to see that demo on the exhibit floor. [will add more here later based on the demo]

Dan Graham and Todd Papaioannou presented Teradata's directions in Cloud Computing to the analyst group. It is interesting to note that Todd recently joined Teradata from Greenplum where he was Chief Architect.

They defined Cloud Computing as: on-demand self-service, with broad network access, with resource pooling, rapid elasticity, and measured service. It can be deployed as a private (behind your firewall), public or a combination. There is a penalty of around 20% to run complex workloads in a cloud environment.

Dan sketched the Teradata Agile Analytic Cloud for provisioning of user data marts, with the advantage of self-service, multi-tenant where the premium is on deployment time for that data mart. Use cases are: proof of concept, development, power-user sandbox, testing, and other short-term projects. To create new data marts, business users has several web portlets to define their requirements. The main advantage is the ability to join with data in the EDW in a managed environment.

In addition, Teradata is offering a cloud-like service that is Software-as-a-Service (SaaS) for demand-chain management in the retail industry. The pricing is based on the number of SKU within the database.

Oliver  Ratzesberger, Senior Director, Architecture and Operations of eBay, started by citing their data volumes: 50 TB/day in new incremental data, processing 50 PB/day, 10s millions of queries/day, 99.98% availability, over 15 PB (petabytes) of online storage, and so on. Those are big numbers!

They are defining the next version of their layered architecture for analytics customers, self-service, analytics as a service. Design for the unknown since 85% of our workload is new and unexpected. Exploration is the core of an analytical company. The metrics you already know are 'cheap'.

Because of intensive time-to-market pressures on the business, there is a renewed push for departments to have their own data marts. eBay needed to respond to these pressures while avoiding the usual problems with fragmented data and partitioned infrastructure. Oliver asserts that the TCO for the simplest data mart is at least $500K. In other words, a data mart cannot be cheap enough to justify its existence.

The solution is to build the same set of private data marts but as a virtualized system that shares all the data and processing capacity. By scaling across a large number of workload results in a much lower peak loads across the enterprise.

"Analytics as a Service 2.0" is an initiative for: self provisioning in five minutes, templatized marts from common problems for business units. You need: unlimited scalability in data volume, concurrent users, application complexity, queries; workload management, and virtualized support. This package is now offered by Teradata.

The benefits are to be able to react to requests for new analytics with 'agile prototyping. Users can 'fail fast' which allows user to try out new ideas quickly at a low cost.

When asked about the relevance of cloud computing to his initiative, Oliver said that the analytics run on the physical system to avoid massive data movement. However, the results of the analytics could drive applications that may run in cloud environments.

Another question was about maturing successful prototype applications into production status. eBay promotes as new subject areas or extension to existing subject areas.