Blog: James Taylor Subscribe to this blog's RSS feed!

James Taylor

I will use this blog to discuss business challenges and how technologies like analytics, optimization and business rules can meet those challenges.

About the author >

James is the CEO of Decision Management Solutions and works with clients to automate and improve the decisions underpinning their business. James is the leading expert in decision management and a passionate advocate of decisioning technologies business rules, predictive analytics and data mining. James helps companies develop smarter and more agile processes and systems and has more than 20 years of experience developing software and solutions for clients. He has led decision management efforts for leading companies in insurance, banking, health management and telecommunications. James is a regular keynote speaker and trainer and he wrote Smart (Enough) Systems (Prentice Hall, 2007) with Neil Raden. James is a faculty member of the International Institute for Analytics.

March 2010 Archives

I was doing some research the other day and found (re-found really) Kurt Thearling's page of Data Mining Techniques. I had forgotten how useful this was and thought I should re-post it for those of you looking for a nice online summary of core data mining techniques.

Posted March 31, 2010 6:38 PM
Permalink | No Comments |

I recently got the survey results from the annual data mining survey that Karl Rexer of Rexer Analytics runs. You can get the summary here or the full results from Karl but here are my thoughts:

  • Data mining is everywhere. The most cited areas are CRM / Marketing and Financial Services with a big lead over Retail and Telecom. Healthcare did poorly, no surprise.
  • Data miners most frequently work in are Marketing & Sales, Research & Development, Risk.
  • Data miners' most commonly used algorithms are regression, decision trees, and cluster analysis - way ahead of the others. Text mining was back in the pack, interesting given the amount of text mining coming presentations we saw at Predictive Analytics World.
  • Half of data miners say their results are helping to drive operational processes. This is encouraging as I think this is by far the most effective way to use predictive analytics.
  • Batch scoring with the results getting stored in the database came top of deployment approaches at 30% with interactive real-time scoring at 21% and 16% putting the model into some overall software project.
  • 60% of respondents say the results of their modeling are deployed always or most of the time. This is still not good enough - nearly half are not getting deployed.
  • The top challenges facing data miners are dirty data, explaining data mining to others, and difficult access to data. However, in 2009 fewer data miners listed data quality and data access as challenges than in the previous year. 34% also have problems with IT.
  • Open-source tools Weka and R made substantial movement up data miner's tool rankings this year, and are now used by large numbers of both academic and for-profit data miners.
There's lots more in the survey so go get it and read it.

Posted March 26, 2010 2:03 PM
Permalink | 1 Comment |

In a recent post, Big BI is Stuck: Illustrated by SAP BusinessObjects Explorer, Stephen Few took issue with the claims of SAP BusinessObjects Explorer. I have not spent any time with the product so I am not going to discuss his specific criticisms but I was struck by a caution he added in the post:

Don't mistake what I've written as a case against Big BI in favor of Small BI. It is entirely possible for large BI vendors to provide effective tools for data sense-making [analytics]. To do this, they need to switch from a technology-centric engineering-focused approach to a human-centric design-focus approach, and base their efforts on a deep understanding of data sense-making. Most of the small BI vendors have done no better in cracking this nut than the big guys. They might be more agile due to their small size and thus able to bring a new product to market more quickly, but when they approach the problem in the same dysfunctional way as the big guys, they fail just as miserably. Just like politicians who sell themselves as "not like the guys in Washington," new players in the BI space often point to the failures of the big guys and then go on to do exactly the same. I am not making a case of small vs. big, but of clear-headed, informed, and effective vs. an old paradigm that doesn't work for the challenges of data sense-making.

It seems to me that part of what Stephen is getting at here is a need to focus not on the technical capabilities but on the ability of a tool to support better decision-making. I see his post as pointing out a key reason I believe companies must begin with the decision in mind, figuring out what kinds of analytic insight will help improve a specific decision and drilling back into their data from there. In contrast, most companies today start with the data and go forward - and most BI tools (big BI, small BI, in-memory BI, SaaS BI) work this way too.


Posted March 19, 2010 2:37 PM
Permalink | 1 Comment |
In a great post on 8 things to keep in mind on predictive analytics, some folks from Diamond Management & Technology laid out some things to keep in mind that I really liked. Here they are with my comments - you can get more detail on each from the series of posts with which they followed this initial one.
  1. Understanding the cost of a wrong decision helps target investments
    Absolutely, though I still think that finding a decision you can tie to an executive's compensation plan works better.
  2. Strategic and operational decisions need different predictive modeling tools and analysis approaches
    .. and deployment approaches. I divide decisions into strategic or direction-setting ones, tactical or day-to-day management ones and operational or transactional ones. Particularly with the latter, which are crucial, you need to think about how the models will be deployed if they are to add value.
  3. Integration of multiple data sources, especially third-party data, provides better predictions
    Yup, but don't just integrate your data - begin with the decision in mind and integrate to support it.
  4. Since statistical techniques and tools are mature, by themselves they are not likely to provide significant competitive advantage
    True. It is their ability to turn YOUR data into YOUR insight that does.
  5. Good data visualization leads to smarter decisions
    .. at the strategic and tactical level and to better models at the operational level - decision making at the operational level is too high-speed, too automated for much in the way of visualization to be useful a the moment of decision.
  6. Delivering the prediction at the point of decision is critical
    Yes!
  7. Prototype, Pilot, Scale
    Of course - don't forget to scale the deployment piece too
  8. Create a predictive modeling process & architecture
    Yes. And map it to your IT development process if you want to impact operational decisions embedded in your enterprise IT infrastructure.
A great list!

Posted March 4, 2010 6:16 PM
Permalink | No Comments |