This BeyeNETWORK spotlight features Ron Powell's interview with Steve Wooledge, Senior Director of Marketing at Teradata Aster. Ron and Steve talk about "big data" and how companies receive real value from big data when they’re able to apply analytics.It has now been a year since Aster Data was acquired by Teradata. Now you're Teradata Aster. Can you tell me how the integration has been with Teradata and what you feel that Aster has added to Teradata?Steve Wooledge:
I'll start with the technology side and then talk about the business integration side. On the technology side, Teradata has the best SQL engine in the world and has built leadership in the data warehousing market. We came in with a unique approach to take procedural processing through the MapReduce framework where we can parallelize code and distribute it across a cluster of commodity hardware. We integrated that with SQL so we have a SQL MapReduce framework that allows business analysts or data scientists to discover new insights from both structured as well as now multi-structured data, different data types that aren't as easily codified in a data warehouse. Teradata Aster brought more analytic flexibility and new analytic types on different datasets to allow companies to explore information and then take those insights and codify it into a data model in the warehouse. So Aster is really being leveraged by the Teradata install base as a data discovery platform with new analytic capabilities to find new insights from information that were difficult to find using just traditional SQL on only structured data. From a technology perspective, that’s how we're positioned and complement the data warehouse.
On the business side, we've taken our team, which was relatively small at the time of acquisition, and scaled it out. We've also hired specialists in the different geographic regions so we have people on the ground in Europe who are data scientists for analytics and business development. They work with the Teradata account managers to showcase the value that Teradata Aster brings to the customers there. In Asia we have a similar model, so we're staffing up internationally now and the integration has gone extremely well.
We've added this new analytic capability to Teradata, but on the flipside, we've taken some of the value they've developed in terms of integrating appliances. Last fall we announced our first integrated appliance that has Teradata Aster running in it. It's the Aster MapReduce platform. We've taken our software and integrated it with the Teradata stack. That allows us to bring in a performance-optimized appliance that plugs right into a customer's environment. We’ve built data adapters that allow us to move data in between systems seamlessly to be part of the Teradata analytical ecosystem.How have you seen big data evolve over the last year and why is big data important to today's enterprises?Steve Wooledge:
There's certainly a lot of hype around big data. When you hear “big,” you think size and volume. From our perspective, the big misnomer there is it's not just volume. There are all these new digital touch points with consumers that generate different data types that aren't necessarily structured, transactional, relational data. There's a lot of interactional data that comes off of social networks or machine-generated data from web servers, and sensors, and telematics. All these different touch points and data types are pretty complicated and huge in volume. People don't really know what insights are in that information.
Big data has been hyped because there's a huge opportunity now with all these new data types, and it takes new analytic techniques to extract the value from those data types. At that scale and at that complexity, SQL isn't necessarily the best way to discover new insights. In the past year, people have been starting to get past some of the technology hype. It's still there, but we're really focusing on the business value that people can get from the data. We want to shift the conversation to business value and analytics. Big data is merely a source of information. It's the analytics you apply to extract the value that's the next wave of big data.Definitely, analytics is where we're able to mine this information and then bring in what we need to help in the day-to-day operations.
That's right. It’s how you operationalize it in the business.What business value can be obtained from a Hadoop implementation with Teradata Aster?Steve Wooledge:
Hadoop has gotten a lot of awareness because it's an open-source phenomenon. It has taken a new technique of processing data called MapReduce processing, which is something that Google had popularized. They've implemented it on a file system. Hadoop provides a large scale-out infrastructure that can do data processing. We get into a lot of conversations with customers about the difference between Hadoop and Aster. Aster also took that MapReduce parallel processing framework, but we chose to implement it on a relational database instead of on a file system.
The benefit of Hadoop is that it's open source so it's easy to get, and it scales out on commodity hardware. Scale is the number one thing, and then because it’s a file system underneath, you can store any data type in its native format at scale. In a customer environment, where Teradata adds value is really on the analytics, and Hadoop provides value as a staging area for these new data types. If you're not sure yet what value there is in that information, you can store it without having to create a data model. You just move it into a file system and then have MapReduce do the processing. That is more batch-oriented in nature, which lends itself well to preprocessing where you can extract the relevant information out of those new data types. Then you can load the extracted information into a system like Teradata Aster where you join it with other structured or multi-structured data to derive the insights and operationalize it in the business. Hadoop complements the Teradata analytical ecosystem because it's an MPP scale-out architecture just like the Teradata Aster discovery platform and just like the Teradata integrated data warehouse.If a company has Hadoop and brings in Teradata Aster, what do you feel are the benefits? Are there more tools? Are there more capabilities?Steve Wooledge:
The value that we add on top of the MapReduce processing framework is the native integration we've done with SQL. One of the challenges that people have when they're implementing Hadoop is that while it scales well and you can throw a lot of data at it, the real value is extracting the information – the analytics. By bridging the gap, we've taken the MapReduce framework, integrated it with SQL, which allows the business analysts who understand SQL or are using business intelligence
(BI) tools like Tableau or MicroStrategy to leverage the data with new analytic functions through an SQL interface.
Teradata Aster has built these new analytic functions on top of the SQL MapReduce framework to solve specific business problems, things like marketing attribution, or pattern detection, or pathing analysis through a website. What's the path that visitors take when they enter your site to the point where they actually purchase a product? What are the different pages they look at? That type of analysis let’s you begin to correlate user segments and different business areas of interest.
What Teradata Aster adds to Hadoop is the SQL accessibility and analytic functions. Then, because we're built on a relational database, you can use SQL on the queries. If you want to do some advanced analytic function, you can use the procedural framework, which is MapReduce, integrated seamlessly.
Teradata Aster is also a user-friendly data discovery platform. As an analyst, you can use SQL. You can use any programming language – Java, Python, Perl, or what have you – to write analytics or you can leverage the out-of-the-box analytic functions that we provide. You get the best of all the analytic capabilities on a scale-out infrastructure that’s high performance. That's what Teradata Aster provides.Now you recently announced Aster SQL-H capabilities. Can you tell our readers what value those capabilities provide?Steve Wooledge:
As I mentioned, Hadoop provides a lot of value in terms of data staging and a scale-out architecture, and it’s a low-cost solution. Because it’s a young technology, one of the drawbacks is that the tools to access the data are very limited. Additionally, it does not have the performance of a relational database for a lot of different reasons. But there are a lot of our customers who want to store data and stage it in Hadoop. What our SQL-H provides is a bridge for the business analysts that want to query data that is stored long-term in Hadoop. They can write a query through SQL, and Hadoop appears as another table from which they can grab data.
We've integrated Aster SQL-H with a new Apache project call HCatalog. HCatalog is a metadata repository, or a library, that tells you where the data is stored across different Apache Hadoop projects including Hive, Pig, and Hadoop MapReduce. We can intelligently grab the data for the analysts that they need to write a query or run analytics functions without them needing to understand where the files are stored and the different directories in HDFS. It allows faster query performance. It allows you to leverage the scale-out, low-cost storage architecture of Hadoop, and it lets business analysts discover new insights through an SQL interface, which has never been possible before with Hadoop. We're the first in the industry to integrate with HCatalog in a meaningful way, and it’s part of our partnership with Hortonworks in the Apache Hadoop Community that has allowed us to engineer that solution and be the first to market with it.That really helps the business – they don’t have to hire all these specialists. Steve Wooledge:
That's a really good point. What are the current trends and challenges that you see right now in the big data space?Steve Wooledge:
Well, I think you hit on one of them. Hadoop is a new technology and people are excited about it, but there's a lack of skills. You need to go out and either retool, or retrain, or hire new specialists that have these skill sets. I think we have an opportunity to provide more value from these technologies like MapReduce processing and by packaging it up for our customers in a way that is more consumable from a business analyst perspective through the business intelligence tools that they've already standardized on and invested in.
There's also an opportunity to integrate the technologies more at the administrator level. If somebody wants to bring in these new analytic processing techniques like the Teradata Aster MapReduce appliance I referred to, it’s packaging them in a way that the system administrators or DevOps can integrate into their system. I think those two areas are where companies are looking for technology vendors to really provide more packaged solutions so that they don't have to go out and retrain, retool, or hire new people. They can leverage the skill sets that they have in-house, get the innovations out of the technologies that are there, and take advantage of the new cost paradigm of commodity hardware and scale-out MPP architectures. What do you see next for Teradata Aster and big data?Steve Wooledge:
I think we're investing a lot on the analytics side. I think that's where the real value is for customers. I mentioned the SQL MapReduce framework. We're extending that to make it even easier to develop analytic functions. We have a team of data scientists within our engineering team who are building, with our customers, analytic functions that solve some of these new business problems that are opportunities to extract value from new information. We want to continue to invest in those analytic functions and applications that we write on top of the SQL MapReduce framework and make it more accessible to people. We see SQL MapReduce becoming a standard in the industry because it does marry the best of both the declarative SQL world with the procedural programming and parallel processing world with MapReduce. We've made Aster available now as a downloadable product. If you go to Teradata Developer Exchange, you can download the Aster database
through a package called Aster Express.
We also have an integrated development environment for people that want to write new analytic functions and share these within the community. You'll see us investing a lot from an analytics perspective as well as integrating more tightly with the Teradata ecosystem so that customers who have Teradata can also get the benefit of these new analytic approaches that Aster brings so that they compete on analytics.
Teradata has always helped customers figure out the real business value and has built industry-specific applications or solutions that can help companies compete better on analytics. I think that's the value that Teradata Aster brings, and we’re going to focus our efforts on making it easier for customers to leverage the new data types and new analytic techniques and apply them in a meaningful way to solve problems.How are your customers using big data? Steve Wooledge:
There are a couple of key applications that we see on a recurring basis. Number one is what we call digital marketing optimization. As I talked about earlier, there are all these new customer touch points –social media, websites, text, mobile, you name it. There are all these new ways to communicate with customers or to market to them, and most customers we talk to say that's great but now they have ten more places to spend money and they are not sure which channels are the most effective. For every dollar they spend on marketing, they need to understand which method – for example, e-mail, search engine marketing, or website banner ads – is most effective. They need to know how they can string those things together to get a better view of their cost model and where they’re getting the biggest ROI. We have a number of customers that are using Aster in that way because we can handle all those different data types across those channels, and stitch it all together to give them the analytic insight of the best place to be spending their money. That's the number one application use case – digital marketing optimization or marketing attribution.
Another big use case is data discovery. People are trying to figure out the value in big data. How do they mine it, what new correlations, what new applications, what new data-driven insights can they find. Having a discovery platform that allows them to search for new value quickly is another huge application. There are other areas like social network analysis – identifying the influencers in a network. If you're Procter & Gamble, who are your brand champions that you want to give special incentives to because you know that they influence so many people in the consumer market. Another is fraud detection and prevention, trying to identify bad actors within the social network, or tracking how money is moving through a network to understand where there are touch points where the money laundering might be happening.
Finally, a huge area that's just starting is machine-generated data. For example, insurance companies are looking at telematics to personalize pricing for different insurance policies based on how fast you drive. There are a lot of applications like smart meters where we're just scratching the surface. Big data is just going to continue to get bigger, and there are going to be a lot of new applications that people are trying to figure out.Let’s talk about social media and networks like Facebook and Twitter. Could you provide us with a specific customer example?Steve Wooledge:
Yes, the Gilt Groupe is one our customers that use us for a lot of different things. One neat application they built is to monitor the Twitter feed of people talking about Gilt to identify the key words they're using, whether it's good or bad, or what products they're most interested in. They can monitor that and, it helps influence them in terms of what flash sales they should put on next and those types of things.Steve, thank you so much for telling our readers about how big data analytics can provide valuable insight for Teradata Aster customers.
Recent articles by Ron Powell