Revolutionary Approach to Big Data: A Spotlight Q&A with Gary Nakamura of Terracotta

Originally published June 4, 2012

BeyeNETWORK Spotlights focus on news, events and products in the business intelligence ecosystem that are poised to have a significant impact on the industry as a whole; on the enterprises that rely on business intelligence, analytics, performance management, data warehousing and/or data governance products to understand and act on the vital information that can be gleaned from their data; or on the providers of these mission-critical products.

Presented as Q&A-style articles, these interviews conducted by the BeyeNETWORK present the behind-the-scene view that you won’t read in press releases.

This BeyeNETWORK spotlight features Ron Powell's interview with Gary Nakamura, General Manager of Terracotta. Gary and Ron discuss how Terracotta’s in-memory approach enables enterprises to store, search, distribute, persist, manage and monitor data with microsecond access to large volumes of data.

Gary, for our readers that may not be familiar with Terracotta, let's start by having you give us a brief overview of the company and your acquisition by Software AG.

Gary Nakamura: We’re a Software AG company, and I'm the General Manager of the Terracotta Business Unit. We were acquired last year by Software AG to be the cornerstone of their next generation data management strategy. We're very excited to be part of the Software AG family and think we have a very compelling value proposition for folks that are trying to solve their big data problems today.

Gary, it seems that almost everyone has a slightly different definition of big data. How do you define big data?

Gary Nakamura: I'm not sure that I have anything unique as far as a definition of big data. It’s no secret that there is a significant explosion of data being collected by mainstream businesses in all industries. It stems back to consumerization of IT, mobile devices, tablets, ultra lightweight notebooks, music, videos, etc.

The challenge of big data is mostly around the business problems that it creates. How do you extract value from that data? Can the existing or prevailing technologies help the end customer solve these particular problems? I do not believe that big data would exist if traditional or prevailing technologies could solve these problems. If they could, people would just be using a regular database and/or a data warehouse to solve analytics problems. The big opportunity for customers who are trying to extract the value from their business data is to be able to look at all of their business data and be able to take the value from it either in real-time or in batch format. Again, the technology problems stem from the volume. The sheer scale of the data is creating opportunities for technology companies to solve these particular problems.

I’d also like to make another point about the volume of big data. It will become the norm. It's not a phenomenon that's going to peak and then somebody's miraculously going to figure out how to manage the amount of data that's being collected and the volume will come down. Eventually big data is going to become just “data.” It won't be “big” anymore because it'll just be what you have to deal with. I do believe it’s big now for some people. I think the statistic is that on average in the Fortune 500 companies, their data is growing at 40% per year. On the high end, it’s flipping over to 100% per year. “Big” will become the norm, and then the questions will be how do you analyze it, how do you extract value from it, and how do you make it go faster – all of the same problems that exist right now for “big data” situations. It's just the natural evolution of what's happening in our world.

At the same time, the demand of the end users to be able to access this data or get the results or reports back at a high velocity is also increasing simultaneously.

The third problem is around variety – how to store this data, whether it's structured or unstructured, and the different formats that it comes in. How do you extract the value? That's the big technology problem, and that's why you're starting to see venture capitalists invest in this area. As I said before, it is a significant problem and, again, the prevailing technologies are not well suited to solve this problem.

Gary, you mentioned that if relational databases could handle big data, we wouldn't have this big data phenomenon. Based on that, it sounds like your solution is revolutionary and not just evolutionary.

Gary Nakamura: That is correct. We had a significant breakthrough with regard to our solution in the market – BigMemory. We have figured out a way to store terabytes of data on commodity RAM without the sacrifice or the residual effects of volatility of that kind of memory. The solution that we provide and the reason we say “in-memory data management for the enterprise” is that we don’t just store the data in-memory. We allow end customers to persist, search, distribute, manage and monitor their data – all the enterprise-class things that customers expect from a data management solution. Again, it's all in-memory. That's the big revolution. Instead of tens of millisecond access from a database, it’s microsecond access to large volumes of data.

When you speak of large volumes of data, how much data is that?

Gary Nakamura: We're talking about tens of terabytes in-memory, so as much as a commodity machine can hold and multiples of those. Today, I think the biggest commodity machine you can buy is two terabytes of RAM on an HP machine or Dell machine. We've have a couple of those in our labs, we have tens of the one-terabyte machines, and we have hundreds of the 500 gigabyte machines. The reality of it is that the price of RAM has come down significantly and will continue to come down. The opportunity to store data very, very close to your application for this microsecond-level access is significant.

Is RAM going to replace disk?

Gary Nakamura: Yes, we believe that is the case. Our value proposition is around storing data in RAM. The problem comes back down to volume and velocity. There are two business problems stemming from large-scale data or big data. One problem is the volume – how do you hold it or store it? The next problem is access. The current database or data warehouse solutions simply cannot deliver the data fast enough to solve today's business problems. That's why you're starting to see a significant fragmentation in the data storage and data management space.

Why do you feel Terracotta is built for handling this kind of a problem today?

Gary Nakamura: I base it on the experience of our customers, whether its Telstra, or Visa, or PayPal, some of the banks that we've signed up, large manufacturing companies that are trying to track diagnostic information about their products or others in biotech/pharma. Our customers are telling us that this solution is built for their problems today, but also built for their problems tomorrow and well into the future. Some of them are starting with tens of terabytes in-memory right out of the gate, and they are planning on hundreds of terabytes of data in-memory. Some of them are even talking about petabytes of data in-memory. They use Terracotta for fast access to large volumes of data, that's the first thing.

The second thing is our product BigMemory is designed to work with the “state of the shelf.” What I mean by “state of the shelf” is whatever is the best version of the RAM that you can buy off the shelf, or a standard Java virtual machine (JVM), or a standard database such as Oracle or Teradata. It's whatever's available. We fit very nicely into the existing enterprise environment. We don't need a specialized JVM. We don't need specialized RAM or hardware. We don't need specialized anything. It's all standard-based stuff. That is why we believe that we're in a very good position not only to solve significant customer business problems, but also to deliver against these business problems in the future.

Gary how does in-memory stand up against other similar offerings like solid state?

Gary Nakamura: Solid-state drive (SSD) is an order of magnitude faster than disk, and random-access memory (RAM) is orders of magnitude faster than solid state. The question is whether or not solid state is an interim step. I think if you look at some of the storage companies that are building flash-based storage solutions or appliances that compete against the likes of EMC and/or NetApp, they have a combination of solid state and RAM. What they do very well is store data on SSD, and then they leverage RAM to speed up that data. They're using RAM to get the microsecond access. What we've figured out is a way to move most of that data onto RAM so you don't have the latency of SSD and/or disk. I do believe that there is a place for SSD. I don't believe it's going to go away, but I also don't believe it's going to solve the velocity problem that customers are experiencing.

The velocity of data is really critical today. Can you give us some examples of how your customers are using Terracotta to handle the velocity issue?

Gary Nakamura: I'll just give you an example of a transformational difference from a business perspective. Our customer is a large credit card company, and they are storing terabytes of transactional data in-memory to detect potential fraud. Using BigMemory, they have been able to take their existing fraud detection service, which had an average fraud detection of an hour, down to less than 4 seconds. The reason it’s transformational is very simple. If you're detecting fraud in an hour, the person who actually made the fraudulent transaction is probably at home looking at their new toy. With the detection down to less than 4 seconds, that person is not even out of the store and the online transaction can be canceled very quickly. That's the transformational aspect. As it relates to credit cards, it’s hundreds of millions of dollars of value proposition.

It certainly makes a lot of sense to identify the fraud immediately instead of taking up to an hour or even longer. For example, I recently received a call asking me to confirm or identify a possibly fraudulent transaction that had taken place eight hours ago.

Gary Nakamura: Absolutely, and that is the fundamental problem. In many cases, with identity theft, eight hours could mean 20, 30, or hundreds of fraudulent transactions. At the end of the day, the credit card company is on the hook for those. That use case also transcends to things like the military. The military collects data and security information at a very significant pace. It’s not so much about fraud, but they would like to be able to detect other things specific to their data. They would like to do real-time detection because it could be a life or death situation where they don’t have 45 minutes to make a critical determination.

I’d like to ask another question about the credit card company example you provided. What did it take to implement Terracotta? Did they have to replace their existing hardware?

Gary Nakamura: It really didn't take much. I mean that's the other obsession that we have as an organization. We have designed BigMemory as a solution that is very simple to implement, and any Java developer can pick it up and integrate it into their application. It's very simple to get BigMemory up and running. In some cases, if they're already using some pieces or a library that we support, it could be as easy as a couple of lines of configuration to get up and running. In the case of this payment company, we were able to essentially engage and get them in deployment in less than six weeks. We did this just before Black Friday, which is a very critical transaction time for large credit card companies. They needed to be able to have high performance, but they also had to meet all their SLAs. With our solution, they were able to breeze through that and all of the other peaks through the year-end shopping season.

It seems to me that the hundreds of millions of savings just drop right to the bottom line and provide a tremendous value proposition.

Gary Nakamura: Right, and it was commodity hardware. They didn't have to buy anything new. That was the beautiful part about it.

Some of these customers had come to a point where there was no amount of money that they could throw at a database or hardware that would solve the problem at hand. They recognized that pouring more money into it was not going to solve the problem. That’s why they turned to us.

Again, hundreds of millions of dollars of risk mitigation is significant. It’s not incremental – it’s transformational to their business. We are seeing this time and time again with our solution across multiple industries.

Gary, this is excellent. I really appreciate you taking the time to give our readers insight into Terracotta’s revolutionary BigMemory solution.

  • Ron PowellRon Powell
    Ron, an independent analyst and consultant, has an extensive technology background in business intelligence, analytics and data warehousing. In 2005, Ron founded the BeyeNETWORK, which was acquired by Tech Target in 2010.  Prior to the founding of the BeyeNETWORK, Ron was cofounder, publisher and editorial director of DM Review (now Information Management). Ron also has a wealth of consulting expertise in business intelligence, business management and marketing. He may be contacted by email at rpowell@wi.rr.com.

    More articles and Ron's blog can be found in his BeyeNETWORK expert channel. Be sure to visit today!

Recent articles by Ron Powell



 

Comments

Want to post a comment? Login or become a member today!

Posted June 4, 2012 by Douglas Laney

Great interview, Ron, and great insights Gary. Cool to see the industry finally adopting the "3V"s of big data over 11 years after Gartner first published them. Thought your readers might be entertained by a copy of the original article I wrote in 2001 first positing them: http://blogs.gartner.com/doug-laney/deja-vvvue-others-claiming-gartners-volume-velocity-variety-construct-for-big-data/. --Doug Laney, VP Research, Gartner, @doug_laney

 

Is this comment inappropriate? Click here to flag this comment.