We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.


Aster: The Best-Kept Secret in Teradata

Originally published January 19, 2017

In this article, John Thuma, director of Aster strategy and adoption at Teradata, talks with Ron Powell, industry analyst and expert, about the many ways Aster can help companies get value from their Hadoop data.

Letís talk about some of the latest Aster announcements like Aster on Hadoop.

John Thuma: Aster on Hadoop is great. As you know, many companies have invested a great deal of money in their data lakes Ė their Hadoop infrastructures. †A year or two ago, Gartner released research about a survey that they did. That research indicated that the biggest struggle organizations are having is realizing value from their Hadoop assets and investment.

Aster on Hadoop allows companies to take advantage of their Hadoop assets. Itís one thing to store data, and Hadoop does great things to store data. What Aster on Hadoop does is that it allows you to actually analyze that data and turn it into value. Big data costs money. Big analytics makes money, and Aster on Hadoop helps bring that to fruition.

How about Spark?

John Thuma: Spark is exciting Ė itís the big topic of the day. But, like Hadoop, it is very challenging, especially as you get out to scale. It becomes more and more difficult. You have RDDs and different data structures. You have to implement either Python or Scala and you have to be very proficient in that.†

We have a Spark connector now for Aster that allows you to obfuscate a lot of that complexity and actually remove all the barriers of knowing whatís actually going on in the internals of Spark. You can still take advantage of all of the machine learning and all of the capabilities of Spark, but what we do is allow you to take care of the data prep and load on the Aster side. Then we stream data over a parallel connection, multi-threaded connection, over to Spark and you can run a machine-learning algorithm without having to know Spark at all. Itís actually one of our SQL MR algorithms called Run On Spark. It is very easy to use. What we do is we make Spark easy.

Thatís great. Another big area is Amazon Web Services Ė AWS. Itís nice to see you guys embracing AWS.

John Thuma:†Aster on AWS is a huge offering. Many people donít know that five or six years ago Aster used to be on AWS. Iíve read the installation manual on how to get it working. But now, you go to the AWS Marketplace, type in Aster, and there it is. You put your credit card in, sign up, and 15 minutes later, you have an Aster environment. So now, you donít have to wait. You can be agile. You can load some data that day. You can run some analytics that day, and you can get your answer that day. And that, to me, is amazing. And, because it is something you can turn off, you can turn it off as well. Youíre only paying for what you actually use.†

I love it, and itís a great way for Teradata to become borderless. Thereís borderless analytics. If you look at Teradata EDW, thatís also on AWS. And itís coming to Azure as well. And it might not be long before Aster is on Azure too. I am super excited about that.

You mentioned 15 minutes to have an Aster environment. †What kinds of skills are required to do analytics?

John Thuma: Well, thatís the beauty of it. If you are a Java programmer, analyst or something of that nature, youíre going to use SQL. If youíre a .NET programmer, youíre going to use SQL embedded in your code to communicate with the database. Well, we take advantage of the ANSI SQL interface to call our functions. And theyíre not user-defined functions (UDFs). Our SQL MR and SQL-GR Ė are actually laid out in such a way that they programmatically take advantage of the data the way it is laid out on the Aster file system. They run optimally against that data structure.

Let me give you a case in point. I was working with a jet engine company that had about 400 variables, about 400 columns of data and they had about 4 million records. It was a small data set for sensor data on a jet engine. I was talking to the product unit manager from this organization. I suggested we do a principal component analysis on this 400 variable table with 400 million records and see how quickly we get to eigenvectors and eigenvalues. He said that would never happen. So we ran it. Wrote the code in about 30 seconds (Select * from PCA). It took two-and-a-half minutes to run it. And his jaw dropped. He looked at it and said those eigenvalues are right. Interesting. He had four eigenvalues before it flattened out, so that variability became zeros. His jaw dropped on the floor watching this happen because heíd never been able to do that. So I suggested we do it with 4 billion records. He said, ďCan it do that?Ē I answered, ďAbsolutely. It will maybe take 15 minutes.Ē†

Another thing that is coming out that Iíd like to talk to you about Is Aster R. Aster R is a big deal. It allows us to take advantage of R at scale against highly partitioned data sets. Big offering this year.†

Well, if we look at real time, how does that all work and then when you go to the cloud is there some latency.

John Thuma: Not really because what our solution does is it generates a JAR file based on another SQL MR function that generates a PMML-like interface that generates the JAR file that can run on an edge node independent of your Aster environment. So you put this out on a Web server or something of that nature, and then you just call that particular API and it runs off of the Aster infrastructure.†

One of the questions I asked was latency and also run time and all the non-functional requirements that I would expect out of something like this. Real-time analytics has to have millions and millions of instances hitting it per second. And, sure enough, the product team had graphs and they came back and showed me the numbers. It was in the tens of millions per second where itís coming back instantly. Weíve done a lot with Aster this year. I think itís the best-kept secret in Teradata.

Okay. Letís close with AskAster. What is AskAster?

John Thuma:†Thatís my new pet project. Itís kind of crazy. I posted out on Twitter and on LinkedIn that I would teach Aster to anyone on the planet in 30 minutes. And, sure enough, I got quite a few people who wanted to learn. I set it up, and I worked in peopleís different time zones and all this other stuff. We actually sat down and did 30-minute conversations on WebEx. I took them through a machine-learning use case. Real-time. They said things such as, ďWow, I didnít know you could do this.Ē†

It made me think ďAskAster.Ē So I created a hashtag called AskAster. So if youíre in a meeting, and you have a question about Aster Ė itís for customers Ė you can tweet your question to #AskAster. There are 7 or 8 people that will be monitoring that, and it will launch in December. So theyíre going to be able to get real-time help. And I think itís an industry first in the technical space. I know this kind of thing exists in some service industries and airlines, but #AskAster is coming in December. Iím excited about that.

Thank you, John. Itís always a pleasure learning more about Aster.

  • Ron PowellRon Powell
    Ron is an independent analyst, consultant and editorial expert with extensive knowledge and experience in business intelligence, big data, analytics and data warehousing. Currently president of Powell Interactive Media, which specializes in consulting and podcast services, he is also Executive Producer of The World Transformed Fast Forward series. In 2004, Ron founded the BeyeNETWORK, which was acquired by Tech Target in 2010.† Prior to the founding of the BeyeNETWORK, Ron was cofounder, publisher and editorial director of DM Review (now Information Management). He maintains an expert channel and blog on the BeyeNETWORK and may be contacted by email at†rpowell@powellinteractivemedia.com.

    More articles and Ron's blog can be found in his BeyeNETWORK expert channel.

Recent articles by Ron Powell

 

Comments

Want to post a comment? Login or become a member today!

Be the first to comment!