Valuable Intelligence from Big Data: A Q&A with Keith Cooper of Connotate

Originally published December 19, 2012

This BeyeNETWORK spotlight features Ron Powell's interview with Keith Cooper, CEO of Connotate. Ron and Keith discuss how Connotate’s scalable solution provides companies with competitive intelligence, financial intelligence, market research and more. They also discuss the results of a survey commissioned to uncover big data perceptions and outcomes. 

Keith, for our readers who are not familiar with Connotate, can you give us an overview of the company?

Keith Cooper: Connotate provides technology and services that automate Web data extraction and monitoring. So if you want to gather data from one website, or 500,000 websites, or be notified when something changes on one or all of those sites, you can use Connotate to do it automatically.

Your goal as a company is to make the World Wide Web anyone's proprietary database. What do you mean by that?

Keith Cooper: Well, executives use databases to organize information so that they can easily query it to gain insights. Most of that information resides inside the company in CRM systems, in sales reports, in churn reports and the like. Connotate's goal is to help you gather and organize information from outside of the company to add to your insights. Now, I'll use an example. If you're a product manager, let's say for Tylenol, and you want to query your database as to how many cases of Tylenol you sold to Distributor X in New York and determine if that is up or down from last month. That's really valuable information. You have all of that information inside of the systems that you have in the company and you get the result. But if you find out that the sales are down to this distributor, then you need to determine why they are down. Your query of your company’s internal systems doesn't give you the answer. It gives you some information, but you also need the answers to other questions, such as: Did they lose some retail accounts? Did they increase their prices? Did the competition reduce prices on competitive products? Do they have enough inventory? Knowing all of those things would lead to a better answer, and all of that information is outside of the company. Connotate can automate the collection of all of that information from wholesaler websites to retailer websites and put that information into a logical database so you can see the “why.” In fact, you can probably see the “why” before your distributor can see the “why.” That's the sort of thing that differentiates it from an internal database.

It sounds to me that Connotate is already able to mine external big data sources and provide intelligence for an enterprise. Is that correct?

Keith Cooper: There are thousands and thousands of use cases for the data that's out there on the Web. But at the end of the day, what we try to do is make it very easy, inexpensive, and efficient to get the data and put it in a format so that you can utilize it and gain the insights that you need.

I understand that intelligent agents power your solution. What is an agent and why are they important?

Keith Cooper: The agent is what we call our proprietary code when it's given instructions to gather specific data from websites. For example, you can create an agent to gather all the product and pricing information from a company's website. You can create another agent to get the same information from another competitor's website, and you can compare the data side by side. You can then instruct that agent to alert you when the inventory or price changes. You don't need to bring in all the information again and again. Agents are important because they're smart enough to filter out the garbage, extract precisely the data you need and then transform it into a format that can be imported into your internal systems.

So it's almost like competitive intelligence on steroids.

Keith Cooper: Exactly. 

Can you give us some examples of the types of intelligence Connotate provides to your customers?

Keith Cooper: Through data we help customers optimize sales, form customer acquisition and retention strategies, and also identify new product and market opportunities. Some of those use cases are, for example, for competitive intelligence like you mentioned – being able to track your competitors for news, management team changes, job postings, pricing changes and inventory changes. 

There are sales intelligence use cases such as monitoring government RFP sites for new construction bidding opportunities and getting that information before anybody else does. 

There are human resource intelligence use cases such as conducting criminal background checks on prospective employees, for example. 

There are also market research intelligence use cases like going out to social media sites, aggregating product reviews and bringing them back so that you can understand how you're doing relative to your competition. 

There are also financial intelligence use cases, and there certainly is just the straight aggregation of news and financial information. For example, some of our customers like Associated Press, McGraw-Hill and Thomson Reuters utilize our technology to gather terabytes of information every day from websites around the world to get that information.

Do you have a set of best practices that you can share with the customer once they engage you so that they can get a better understanding of how to do this?

Keith Cooper: Oh yes, absolutely. We have a professional services group that does just that, and we've tried to institutionalize the process a bit. We have quarterly business reviews with our customers to make sure that they are using the technology in the best way. During these reviews, they give us feedback to help us develop our roadmap. We have user group meetings regularly, and we have customer support at all times if a customer has questions or a query as to what sort of data they might be able to get from websites. We've done this millions and millions of times at this point, so we have a lot of insights that we can share.

If there was just one nugget that you could share that you’ve seen that has really amazed you regarding the types of insight gained by customers, what would that be?

Keith Cooper: Well, I'm always most impressed with how people are utilizing the social media commentary that's increasing in scale and is out there en masse, and bringing it back with our technology and utilizing one of our partners. We have many partnerships with technology companies that do text analytics or sentiment analysis on words being spoken. Companies are able to garner great insights as to how they're doing, how their product is doing, and how their company is doing from the words of their prospects and customers. The combination of our powerful technology to gather the information and bring it back in a format that can be analyzed along with our partners’ technology, which can then break that down and give insights from it, is very, very valuable for any company and any brand.

That actually is one of the biggest needs I'm hearing from our audience regarding social media sites like Facebook and Twitter. They would really like an easy way to make sense of all that data to impact their bottom line.

Keith Cooper: Facebook and Twitter are obviously the big gorillas out there in the way of volume. But a lot of that volume is not necessarily relevant to your particular product or to your particular industry. Often blog sites, customer comment sites, or even product reviews within companies’ websites are the most valuable to product managers or executives trying to make decisions about product lines. Our technology is really good at getting that information and bringing it back in a format that can be usable.

Keith, how quickly can a new customer have something up and running with Connotate?

Keith Cooper: Well, the technology is very easy to use. We have designed it so that we can do that for you as a service. You tell us what data you want from what websites, and we will put our professional services team on it. Depending on how many websites and how much data, it can be gathering information within hours or days. We can also train non-programmers within any organization to create these intelligent agents on their own so they can collect that data on premise in their own company. The training only takes a couple of hours. Depending on the type of data, one of these agents can be created literally in minutes to begin collecting data.

Connotate recently commissioned a survey to uncover big data perceptions and attitudes. Can you share some of the outcomes, and were there any unexpected findings from that survey?

Keith Cooper: We just got it back, parsed through it and published some of the results. We had a really good response. A significant number of executives took the time to fill out the survey, approximately three times as many as last year, so close to a thousand respondents, which is terrific. That increase tells us there's obviously a lot of focus and a lot of interest in the category. The survey revealed that most people are paying attention to big data. Many people don't quite know how to define it, not that that’s all that important. But what is really important is executives are trying to figure out if it is worthwhile to spend time and money on infrastructure to have people focus their time on this. 

We're trying to help folks understand how they can get ROI from big data. We've put on a series of webinars that focus on specific use cases. Obviously, competitive intelligence is one, sales intelligence is one, and we have another one coming up soon. We've tried to put those actual use cases out there so customers can understand what to do with that data. We also found that there are concerns that there's too much data and folks don't quite know what to do with it. Also, there were a lot of people that said they wanted to wait and see to find out how best to look at and utilize big data.

Based on your survey, what is your overall perception of what this means for the big data industry?

Keith Cooper: Well, I think there's a real pent-up demand for solutions that can address some of those problems. One of the things that we've tried to do with our technology is help automate that data extraction and transformation, and to do that at scale so that the amount of big data can be manageable and utilized in a very productive fashion. The challenges of manpower and volume are intertwined. If a project continues to grow and the only option a company has is to throw more people at it, they'll never achieve the ROI. Ultimately that's inefficient. The automation that we provide is really the only way to achieve a scalable and improved ROI. That is our strong suit – scalability and reliability. The market is beginning to recognize that, and we're fortunate and growing quickly because of that.

A major drawback for organizations is skilled resources for big data projects. It seems that many of your partnerships address that issue, right?

Keith Cooper: Yes.  Again, we have partnerships with companies that can help you better understand what the data says using text analytics, sentiment analysis, or visualization tools to help graph data, and that's very valuable. I would advise any company to think real hard about what it is they want and why before getting started, and we'd be happy to speak with anybody about the opportunities that they might have.

Keith, thank you for providing our readers with an understanding of Connotate and how you can help companies sort through big data to derive useful and valuable insights.

  • Ron PowellRon Powell
    Ron, an independent analyst and consultant, has an extensive technology background in business intelligence, analytics and data warehousing. In 2005, Ron founded the BeyeNETWORK, which was acquired by Tech Target in 2010.  Prior to the founding of the BeyeNETWORK, Ron was cofounder, publisher and editorial director of DM Review (now Information Management). Ron also has a wealth of consulting expertise in business intelligence, business management and marketing. He may be contacted by email at rpowell@wi.rr.com.

    More articles and Ron's blog can be found in his BeyeNETWORK expert channel. Be sure to visit today!

Recent articles by Ron Powell



 

Comments

Want to post a comment? Login or become a member today!

Be the first to comment!