The Analytics Game Changer: Internet of Things

Originally published January 18, 2016

This BeyeNETWORK article features Phil Bowermaster’s interview with Bill Franks. Phil is an independent consultant and analyst specializing in big data and analytics, and Bill is chief analytics officer at Teradata. Phil and Bill discuss the how the Internet of Things is going to revolutionize the world of analytics.

Recently you’ve been talking about the Internet of Things. What’s it all about?

Bill Franks: The Internet of Things (IoT) gets a lot of play from the technology perspective, but what I’m excited about is the data and the analytics perspective. There is an absolutely huge amount of data coming at us already, and it’s going to be growing from the Internet of Things. The Internet of Things is going to dwarf many of the other sources of big data, and companies have a lot of preparation to do so that they will be able to analyze and drive value from IoT data.

Can you give some examples of Internet of Things analysis?

Bill Franks: The most common example is known as predictive maintenance. It uses a lot of sensors that are embedded either within an engine or some other type of equipment, such as electronics like ATMs. In preventive maintenance, computers can monitor the patterns of the sensor readings and identify those that may lead to a failure at some near point in the future. There is also work being done around the use of drones. They take images of, for example, people playing on a sports field. Your kids could be playing this weekend. With no infrastructure investment, you could put up the drone, have it take film. You can analyze that film and get the player movement. 

Another example is crop optimization. Take a picture of your field. Send out a drone. It will take a measure of the current temperatures. More importantly, they’ll actually analyze the coloration of the leaves in very micro areas of the land and identify where more or less water is needed, or more or less pesticide. And they’ll custom download to your farm equipment what you need to do for each of those. And you’ll go through and customize the treatment.

These are just some of the things – and we’re not even touching the surface of things like personal fitness bands, other devices that capture information about us and smart grids for the utilities. It is all over the place.

Sounds like it’s a huge range of devices producing a wide range of different kinds of data. How do you have to handle that data differently than data from traditional sources?

Bill Franks: It is so big and it is so massive that you really have to develop some protocols to aggressively filter it and limit it to what is actually important. For example, do I really need a temperature reading every millisecond when I’m monitoring the temperature of the room we’re sitting in to have a thermostat or even the overall heating system of the hotel take action? The answer is, “Not really.” I’m probably okay every 10 seconds or even one minute. You can slow down the cadence.

There is also aggressive filtering you can do. You don’t need to report that it is exactly 72.0 degrees every ten seconds. You can report it is 72.0 the first time it hits that, and then you report again whenever that temperature changes. If that happens to be 10 hours from now, you could infer that to every other point in between. 

There are a lot of different types of filtering and limiting that has to be done to make that data usable. A lot of the sensor readings that are possible may not add value for your applications and may not be worth capturing at all.

With so many different kinds of sensors and devices spread out over such a great space, I can only imagine that there are lots of privacy issues that are arising from this?

Bill Franks: There absolutely can be. Some of the data can be incredibly sensitive. When we think about the things in medical equipment at the hospital or the information on my fitness device. Many would consider the data collected by your cell phone (which is a “thing”) about your location to be sensitive. There is definitely going to have to be some thought given to some of those privacy issues. 

One other angle of this is that “things” often then have operating systems. They’re connected to the Internet. What about virus or intrusions, especially as they become outdated. It’s going to become a very big issue that I don’t think people have thought about. Once you have 200 outdated things in your home, it’s not going to be that different from if you had a computer running a 10-year-old operating system that hackers could hack into in seconds based on the outdated nature of it.

Because of the complexity of the things themselves and, therefore, the data they produce, are there different multiple levels of analysis that have to be done with that kind of data?

Bill Franks: Layers. If you think about your house, you might have a smart heating system for the whole house that attempts to not just keep one thermostat at one spot in your house at one temperature, but various rooms all at a similar temperature, pushing heat and opening and closing vents in the right rooms to keep it stabilized. But above that, you might have an overall smart home system that incorporates media equipment and things around your kitchen. Those then can roll up to your smart home and when combined with your neighbor’s smart home data helps the power company start to understand at that local level what is being utilized and how to get power. But then another layer up from that, you have the things in the grid that are then feeding the homes that could be analyzed to better optimize how all the neighborhoods get the power. There are multiple layers of analysis and usage of this information.

What kinds of solutions is Teradata putting in place to capture and store and analyze this IoT data?

Bill Franks: First of all, Teradata has always been built for huge, high-volume data. IoT is obviously another example. JSON, one of the most common formats for machine sensor data, is now a native data type in the Teradata Database. We announced Teradata Listener, which is targeted – among other things – to capture this type of data. We have Hadoop platforms that integrate with our overall architecture, which would be a great landing place for the raw data. We’ve really got a lot of things going on in addition to ramping up our professional services people who know how to handle this data.

I understand that you’re overseeing changes in how Teradata provides analytic services. Can you elaborate on that?

Bill Franks: We have had success working with customers around the analytics. As a result of that success, we had the same problem  and I’ve written about it in my book, The Analytics Revolution  that other companies have. We ended up with various pockets of people delivering analytics. We’re working on tying them together into a cohesive team and getting it set to scale up to yet another level. I’m pretty excited about the opportunity to see that through in the coming year.

For organizations that have a lot of “things” – they’ve got devices out there with sensors producing data – they want to get started, what’s the best way? 

Bill Franks: You have to get hold of some of that data, and you have to start looking at it. You don’t need all of the data. You don’t need a massive amount. Get enough to start to test some theories. I think it is good to partner with a company like Teradata. We can provide some of the technology and the people to analyze it those first times so you don’t have to put a lot of upfront money into just experimenting before you understand the value of it. Let us help prove the value. Then after you understand the value, we can help you implement. But I think the key is to start with the experimentation. Start with some small-scale pilots to really prove the value because the cost to capture and house all of this on an ongoing basis is quite large. So you want to make sure that you do it the right way and justify it.

Where does the Internet of Things analysis go over time? Is it a game changer or is this just a flash in the pan?

Bill Franks: I think it is going to be a game changer. Honestly, I’m not even sure that any of us know yet exactly how game-changing it is going to be. I think it’s going to be a bit of an evolution, just like the Internet. I don’t think people predicted how much it would change the game. 

One example that everybody is enamored with today is the idea of the self-driving cars that are already doing hundreds of thousands of miles. That is 100% data and analytics that make that happen. It is the cars collecting information on what is around them to see where the other cars or obstacles are, looking at the video of the lane to know where the lane markings are, or deciding when to brake. That requires a massive collection of data from sensors. It’s detailed and complex analysis to decide when to steer, when to brake and all of those things. I would think most people would consider it pretty game-changing to have a self-driving car pick you up at the airport and get you to your hotel or other destination. That’s all about IoT and analytics, and I think that’s why I get excited. I think there will be a lot more examples like that coming forth in time.

Big change is coming and the Internet of Things is driving it. Thank you for sharing your perspectives with us.

  • Phil Bowermaster
    Phil Bowermaster is an independent analyst and consultant specializing in big data, business intelligence and analytics. Phil is the founder of Speculist Media, which produces blogs, podcasts, and other social and traditional media exploring the role of technology, particularly data technology, in shaping the future. He works with select clients in developing and executing content strategies related to big data. Phil can be reached at

Recent articles by Phil Bowermaster



Want to post a comment? Login or become a member today!

Be the first to comment!