Yet there are too many low-grade tools out there. Sentiment analysis done right is about much, much more than simply scoring tweets and reviews. Sentiment analysis done right discovers business value in customer, consumer, and constituent content and behaviors, whether online, on-social, or in enterprise feedback.

The Sentiment Analysis Symposium, May 8 in New York, is the place to learn more.  This is an authoritative conference that brings together experts and practitioners from research and industry. You'll have an unmatched opportunity to learn about state-of-the-art technologies, how they are applied, return on investment, and how to choose from among the many available options.

Options help you understand social conversations and also direct and indirect feedback (such as surveys, contact-center notes, and warranty and insurance claims), online news, presentations, even scientific papers: Any information source that captures subjective information.

Advanced analyses monitor and measure sentiment and often much more, linking sentiment to demographics, customer profiles, behaviors, and transactional records. They help business analysts (and marketers, market researchers, customer service and support staff, product managers, and other users) get at root causes.  These are the explanations of behaviors captured in transaction and tracking records. Sentiment analysis means better targeted marketing, faster detection of opportunities and threats, brand-reputation protection, and the ultimate aim, profit.

Social Media revolve around feelings, attitudes, and emotions. Facebook and Twitter are major sources of sentiment (and also of complementary social connectedness data). Facebook and Twitter accounts have profile data attached to them, but nothing that matches the detailed, usably-structured information you can find on LinkedIn. Google is the ultimate information-access engine, capable of bringing together information from a huge variety of disparate sources, including sentiment information such as product, restaurant, and hotel ratings, although when corporations wish to find, mine, and exploit sentiment they need to turn to deeper BI and analytics tools.

There's no one-size-fits-all sentiment solution, not Google or one of the several as-a-service solutions out there or any of the capable analysis workbenches or social-media analytics tools. Instead, there's a whole spectrum of sentiment sources and analysis possibilities.

These are a sampling of the topics that will be covered at the May 8 Sentiment Analysis Symposium. You will meet and learn from experts, strategists, practitioners, researchers, and solution providers - experienced and new users and those evaluating solutions. A sample of speakers for the event includes the American Red Cross, Fidelity Investments, Thomson Reuters, American Express, Kraft Foods, and the Wall Street Journal.

For a crash course on technology concepts, you should also attend the May 7, half-day Practical Sentiment Analysis tutorial, taught by Prof. Bing Liu. (Check out this profile of Bing that appeared in the January 27, 2012 New York Times.)

To register please visit sentimentsymposium.com/registration.html today. See you there!

Tom is a text analytics early adopter and long-time proponent. He was one of the first to apply natural language processing (NLP) techniques for ad-hoc consumer research although he chose to focus on professional services in the years when market-research oriented solutions such as Buzz Metrics and Cymfony (later acquired by Nielsen and TNS, respectively) first emerged. Tom is also behind a Next Generation Market Research movement and says that Anderson Analytics' solution is based on the firm's years of research and experience.

Tom has responded to a few questions.

Seth: That Anderson Analytics has been quietly working on developing a text analytics software product is a welcome surprise. How did that come about?

Tom: We've always done a little internal development to fill gaps. I've been relatively open about my opinions on the state of text analytics software in general, and that there are no perfect tools out there. It's more about selecting the right tool or combination of tools for the right job and then knowing how to use them. I realized as early as late 2005 that the software out there really isn't developed with the analyst in mind, and that developing something seemed to make sense. Text Analytics has changed a lot since 2005, and it will continue to do so. So I doubt it comes as a surprise to anyone following this field that we're now in development of something more elaborate. Our feedback so far has been very positive.

Seth: There are many text tools on the market, so why now?

Tom: Well I've spoken to market research directors at several Fortune 500 companies. Interestingly, most of them had similar experiences and opinions in regard to text analytics that I had. Many had tried, or requested proof of concepts from large vendors in the text analytics industry and had been underwhelmed by what they saw, especially considering the price tag. It was clear to me that there was still a lot of room for something created by those who both understand text analytics and the needs of market research professionals.

Seth: Will this be a stand-alone, do-it-yourself tool or part of a larger service offering?

Tom: Probably both. Initially an 'OdinText Lite' intended as DIY option, I also think text analytics can add value to some of the other offerings of full service research firms. We also envision slightly different modifications depending on intended use. This in my opinion is one of the major failings of what some of the other large vendors are offering, tools that supposedly can handle any type of text regardless of source. If you build something for everything, then how accurate and useful can it possibly be for a domain expert? In this case the intended expert is the customer intelligence expert, not so much the PR or advertising executive.

Seth: I heard you've already been approached by one large agency regarding some sort of investment or partnership?

Tom: Well yes. However, Anderson Analytics is well positioned to get a quality product into the hands of our customers. That said I do try to keep an open mind if someone brings something additional to the table which can add value.

Seth: Thanks, Tom.

Tom: We're on the Web at OdinText.com if readers would like to get in touch.

We will explore the Why and How of content analytics at next week's Smart Content conference. Whether or not you can attend, five articles I've recently published will help you come up to speed. They're accessible -- not technical -- Five Easy Pieces if you will, with a great deal of richness, to be explored, beneath their surfaces.

Check out --

• Content Management Finds Meaning
• This is Content Intelligence, According to 4 Experts
• Content strategy by the numbers, via semantics
• Six Definitions of Smart Content
• Semantics and Analytics Unlock Value in Social and Online Content

We'll hear more about success stories in diverse business domains at the conference; we'll explore all things content analytics. But even if you can't join us, key a watch on the topic. "Unstructured" content, from online, social, and enterprise sources, is a next big thing for BI and content management both as boundaries expand to create a world of semantically unified business information.

The information governance concept, beloved by corporations and consultants, barely applies. It's a challenge creating standards and maintaining content-production rules, more a drag than a benefit, given the highly competitive, fast changing, almost chaotic content marketplace. It's semantic and analytical technologies, which help you find and exploit patterns relevant to your goals, whether expanding readership or automating sense-making, that allow content producers and consumers to keep up, to create findable, flexible, and reusable content and to generate business-linked insights.

Smart Content, the conference, is really just a next step in the BI/analytics/applications market education and match-making I've been doing for years. The opportunity is huge -- business and technical -- a consequence of the value content analytics can bring to news and social media and Web and enterprise content.

We'll cover a spectrum of approaches -- as applied in media & publishing, advertising & on-line commerce, marketing and PR, finance, research, and the Semantic Web -- enhancing the value of news and social media and Web and enterprise content -- with links to enterprise information management, content strategy, BI, text analytics, and search.

We'll start with Visionaries Panel with Dries Buytaert, CTO at Acquia and creator of Drupal, Natasha Fogel, EVP at Edelman StrategyOne, and Mark Stefik from XEROX PARC, followed by Jeff Fried of Microsoft explaining What Business Innovators Need to Know about Content Analytics.

We'll have talks by Rachel Lovinger, content strategy lead at Razorfish, Darrell W. Gunter, EVP/CMO at Collexis, and Randall Snare & Elizabeth McGuane of iQ Content, Dublin -- preceded by a series of lightning talks that will help attendees learn about the gamut of innovative smart-content solutions -- and followed by five Application Spotlight talks. Then stick around for a networking reception.

Smart Content will take place Tuesday, October 19 at the Executive Conference Center at 48th & Broadway in Manhattan. Learn more, and register today, at smartcontentconference.com. Register by September 10 for a $200 early-bird discount.

As my colleague Laurel Earhart, Smart Content marketing director, puts it, Smart Content is designed for decision makers, implementers, solution providers, and also investors. We're expecting great things to happen!

Lastly, I'm quite happy and appreciative to have TechTarget and the BeyeNETWORK as a Smart Content media sponsor, and the support of other prominent media and solution providers in the content management and analytics space. I'd love to have TechTarget readers and community members join us for what is sure to be an excellent program, due of course to the quality and expertise of the Smart Content speakers.

Please visit smartcontentconference.com for more information and to register. (Early-bird rates run through September 10.) Thanks!

I hope you can attend. We have a great program lined up. I'm especially looking forward to the lightning talks, a series of quick demo-presentations, and to "Selecting a Social Media Analysis Platform/Provider" with moderator Suresh Vittal from Forrester Research and social-analytics gurus Nathan Gilliatt and Marshall Sponder. Note that registration is 50% off for academics, government, and non-profits.

And this is a first note about this year's Text Analytics Summit, slated for May 25-26 in Boston. I'm program chair once again and will present an introductory workshop the afternoon of May 24. Please consider joining us!

Do get in touch about the conferences or anything else related to BI, text analytics, or sentiment analysis -- my coordinates are on-line -- or follow me on Twitter for updates.

Seth

For a forthcoming BeyeNETWORK article, Perspectives on Text Analytics in 2010, I asked solution-provider executives about the top challenges and opportunities they foresee for the coming year.  Eight out of ten responses cited social media and sentiment analysis: solutions that harvest opinions, attitudes, mood, and other subjective information from news, social media, surveys, and other forms of enterprise feedback.

Claire Thomas, text analytics lead at SAP, calls sentiment analysis "broadly applicable to various industries and initiatives." She captures the rationale for the up-coming Sentiment Analysis Symposium, a conference I am organizing, slated for April 13 in New York. 

Claire and SAP have not endorsed the symposium.  I'm quoting Claire and other industry leaders simply as evidence that a practical, solutions focused sentiment-analysis forum, bridging technology and business concerns, is sorely needed.

Behind the need:

IBM SPSS Vice President Olivier Jouve notes that "Twitter, Facebook and other Web 2.0 media are the new critical sources for marketing" and a range of other enterprise functions.  These functions include what's often now called "social CRM," customer relationship management that taps social sources.  I predict that category will be short-lived, just as early-2000's "e-commerce" is now just one facet of modern, comprehensive business solutions.

The fact is, "more companies are looking at 360-degree views of customer feedback, and social media is a critical early warning (before a customer buys) and customer support (when a customer is having issues) indicator of customer experience," as Clarabridge CEO Sid Banerjee puts it.

I'd guess we can all agree with these statements, but how to handle these and traditional information sources?

According to Lexalytics CEO Jeff Catlin, in 2010, "sentiment will complete its transition to a 'checklist' feature that everyone who works in this space will have to provide.  All of the vendors (big and small) will claim to have sentiment."  So there's a problem evaluating sometimes over-stated claims to choose an appropriate solution.

What's appropriate?  Many user organizations can get by -- for the present -- with what Attensity CTO Ian Hersey characterizes as "social media aggregation and lightweight analytics (e.g., buzz analysis, media monitoring)." 

The Sentiment Analysis Symposium will be for organizations with needs that range from focused to sophisticated, to users who, in Ian's words, want "to incorporate the social media into the same analytical models as they use for their internal data and, more important, plug that social media into business processes."

These enterprise scale users require solutions that, in Olivier Jouve's words, handle sources that are "voluminous, cryptic, multi-lingual and deeply interconnected" via "sophisticated data collection mechanisms, advanced multi-lingual analysis and the infrastructure to manage daily terabytes of data."

This being the BeyeNETWORK's text-analytics channel, I've quoted text-analytics industry leaders although I expect media-monitoring, listening platform, and brand/reputation management users, agencies, and solution providers will be well represented at the Sentiment Analysis Symposium.

So check out the event on-line -- follow @SentimentSymp or me, @SethGrimes, on Twitter for updates -- and you have until February 3 to submit a speaking proposal by the way -- and do send me your questions and comments.

Software

Attensity has released a new version of its Voice of the Customer product, Analyze for VOC Version 5.2, that includes accuracy enhancements, a new sentiment scoring feature, a RESTful real-time integration architecture, and analysis enhancements include normalized time-series charts and calculated values according to the company. Version 5.2 introduces new out-of-the box reports for sentiment, Net Promoter Score (NPS) issues, customer churn and competition, according to Attensity.

Attensity has also released E-Service Version 6.1, an enhanced version of the company's application suite for customer service and support organizations.

Clarabridge launched Clarabridge Social Media Analysis (SMA), which the company characterizes as "the industry's first advanced text analytics software that allows companies to integrate social media content into their existing internal enterprise feedback to create more useful customer analysis," in September. The solution uses social media content from Alterian Techrigy's warehouse of social media content, with data from blogs, Facebook, Twitter, YouTube, MySpace, and other social media sites.

Linguamatics released I2E 3.1 in October. According to the company, key enhancements include support for enterprise deployment, NLP-based querying of a greater choice of document types, a new I2E Chemistry option with substructure and structure similarity search powered by ChemAxon, extended results reporting, flexible hyperlinking from extracted entities to web resources such as gene identifiers, glossaries of biomedical terms, and chemical structure visualization.

Orchestr8, a developer of semantic tagging and text mining software, in September announced a new technology to complement their content analysis service, AlchemyAPI. Visual Constraints is designed for extraction of structured data (product info, pricing, descriptions, etc.) from Web pages. A new, October AlchemyAPI release extends extraction capabilities to quotations and named-entity coreferences.

Resources

The UK National Centre for Text Mining has posted presentation slides from the October workshop on Text Mining for Scholarly Communications and Repositories.

An e-book by Graham Wilcock of the University of Helsinki, "Introduction to Linguistic Annotation and Text Analytics," according publisher Morgan & Claypool, "provides a basic introduction to both fields, and aims to show that good linguistic annotations are the essential foundation for good text analytics."

Conferences

Text Analysis Conference (TAC 2009) workshops will be held November 16-17, 2009 at the National Institute of Standards and Technology in Gaithersburg, Maryland, co-located with the Text REtrieval Conference (TREC), November 17-20, 2009.

A May 1, 2010 workshop on text mining will be held in Columbus, Ohio in conjunction with the 2010 SIAM International Conference on Data Mining (SDM 2010). The workshop is devoted to techniques of machine learning in conjunction with natural language processing, information extraction and algebraic/mathematical approaches to computational information retrieval.

The 1st Information Retrieval Facility Conference is slated for May 31, 2010 in Vienna, followed by the 3rd IRF Symposium, June 1-4 2010. The conference aims to provides a forum for researchers in information retrieval, Semantic Web technologies for IR, Natural language processing for IR, and large-scale or distributed computing for those areas. They symposium will especially focus on methodology and evaluation in patent searching and retrieval.

The North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL HLT) 2010 conference will take place June 1-6, 2010 in Los Angeles, California. The conference covers a broad spectrum of disciplines working towards enabling intelligent systems to interact with humans using natural language, and towards enhancing human-human communication through services such as speech recognition, automatic translation, information retrieval, text summarization, and information extraction.

The 48th annual meeting of the Association for Computational Linguistics (ACL 2010) will be held in Uppsala, July 11-16, 2010. The ACL workshops will be held July 15-16.

The 33rd annual ACM SIGIR Conference is slated for 18-23 July 2010, Geneva, Switzerland. SIGIR is the major international forum for the presentation of new research results and for the demonstration of new systems and techniques in the broad field of information retrieval (IR).

The 23rd International Conference on Computational Linguistics (COLING 2010) will be held in Beijing, August 23-27, 2010. There will be pre-conference workshops on August 21-22 and post-conference workshops on August 28.

Management student> I have been struggling differentiating some terms and understanding them more clearly. Therefore, my questions are related to that confusion. I would also like to hear your opinion on these two technologies (BI as software and enterprise search) and their uses of text analytics.

MS> What is the difference between text analytics and text mining? Is it related to structured vs. unstructured data? Or is text mining a subset of text analytics?

Seth> There isn't a significant difference. I find that text mining is used in areas that have applied the technology longer and that apply data mining. Examples include life sciences and intelligence (e.g., counter-terrorism). Text analytics is more often used in business.

MS> Is content analysis the same as text analysis (if we look at textual documents, not rich data)?

Seth> To me, "content" generally indicates managed information that is typically found in a repository and that is often published on the Web. In this sense, e-mail and IM messages, survey responses, contact center notes and transcripts, and other forms of text generated during business operations are not content. In this sense, content analysis that concerns text is a subset of text analysis.

But "content" does also cover video, audio, and other media as you note. Content analysis would include these forms where text analysis wouldn't, as you understand, beyond work with textual tags.

MS> Is there a difference between text analytics done by search technology and BI applications?

Seth> Text analytics that backs up search is meant to support information retrieval: indexing, summarizing, and ranking documents in response to a search query. TA enables semantic indexing by topics and themes and relationships in order to go beyond indexing based solely on keywords. TA in support of search can also enable smarter, and natural-language, query processing. The example I'll give is that you can enter "map oslo" in Google and get a map of Oslo, because Google is doing a combination of named entity recognition for the geographic area, Oslo, and pattern matching that understands that "map " is a request for a map.

TA in BI (outside use of search for BI) is different. A complete definition of BI include treatment of information in textual and other forms, in databases, repositories, and on the Web. Search is a BI tool, and so is information extraction (a text analytics technique; information = entities, facts, topics, themes, etc.) into structured databases -- some see IE from text as equivalent to ETL for traditional databases -- and also analysis in the sense of data mining of text-extracted information. So when, for instance, you visualize a relationship network that includes people, companies, etc., based on text-extracted named entities and links (relationships, events, etc.), that's TA at work for BI.

MS > What are the fields that use text analytics the most? (any industries in particular?)

Seth> Life sciences and intelligence (including counter-terrorism) were the earliest use cases with serious work going back to the late '90s and they're still very strong domains for TA. But now we're seeing use in a spectrum of business applications as well.

Seth> Let me refer you for this question and the next to a report I recently published, which you can download for free at http://altaplana.com/TA2009 .

MS> How would you describe text analytics market?

[Seth> In my paper, I estimate a 2008 diversified, global market for text-analytics software and vendor provided professional services at $350 million, representing 40% growth from 2007. I foresee sustained growth rates of up to 25% for 2009.]

MS> There is a lot of talk about eDiscovery where text analytics plays a crucial role, but it is also one of the main markets for search technology. Are these two technologies (is it ok to call text analytics is a technology?) coming together?

Seth> I believe that in e-discovery, the principal application of TA is (still) in support of search in the sense that I wrote about above, creating richer indexes that allow legal researchers (litigants) to respond faster and comprehensively to discovery mandates. TA is only starting to be used by legal professional for investigatory purposes, for what you could call "making the case." Compliance and fraud investigations, and risk management, are starting points in this type of use. But I don't think the technology is being used systematically by litigators yet. I do think we'll see a lot more of this investigatory type of use.

I hope you've found our Q&A useful! As always, if you have questions or comments, do get in touch.

Software

Orchestr8 released on June 18 a significant upgrade to its AlchemyAPI content analysis online service. According to the company, the update includes expanded language coverage (adding Portuguese and Swedish), enhanced text categorization, and integration with Linked Data standards. "AlchemyAPI is a web-based service that enriches a publisher's content through automated tagging, categorization, and semantic analysis available as both a free online API and commercial subscription service."

Attensity Group announced on July 8 the availability of its new, hosted Survey Advantage service at a $5,000 per month point of entry. Attensity Survey Advantage "enables departments within large organizations and government agencies to measure, chart and understand customer sentiment and top issues expressed in customer feedback surveys."

Book

Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit by Steven Bird, Ewan Klein, and Edward Loper was published in June 2009. "This book offers a highly accessible introduction to Natural Language Processing, the field that underpins a variety of language technologies ranging from predictive text and email filtering to automatic summarization and translation. You'll learn how to write Python programs to analyze the structure and meaning of texts, drawing on techniques from the fields of linguistics and artificial intelligence." Visit O'Reilly for information.

Conferences

A joint conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing will be held August 2-7, 2009 in Singapore. ACL-IJCNLP 2009 will cover a broad spectrum of technical areas related to natural language and computation.

The 2009 conference of the German Society for Computational Linguistics and Language Technology (GSCL) will include a workshop on the Unstructured Information Management Architecture (UIMA), September 30, 2009, in Potsdam, Germany. "Participants are invited to present applications realized using UIMA, general experiences using UIMA as a platform for natural language processing, as well as technical papers on particular aspects of the UIMA framework. Alternatives to and comparisons of other frameworks - e.g. GATE, LingPipe, etc. - with UIMA are of interest, too."

The third IEEE International Conference on Semantic Computing is slated to be held September 14-16, 2009 in Berkeley, California. ICSC 2009 is "an international forum for researchers and practitioners to present research that advances the state of the art and practice of Semantic Computing, as well as identifying the emerging research topics and defining the future of the field."

Recent Advances in Natural Language Processing RANLP 2009 is slated for September 14-16, 2009 in Borovets, Bulgaria, preceded by September 12-13 tutorials and followed by associated workshops September 17-18.

The ACM Eighteenth Conference on Information and Knowledge Management (CIKM 2009) will take place in Hong Kong, November 2-6, 2009. The conference is sponsored by ACM SIGIR and SIGWEB.

Language and Technology Conference 2009: Human Language Technologies as a Challenge for Computer Science and Linguistics (LTC 2009) will take place November 6-8 in Poznan, Poland. "Human Language Technologies (HLT) continue to be a challenge for computer science, linguistics and related fields as these areas become an ever more essential element of our everyday technological environment... [creating] a favorable climate for the intensive exchange of novel ideas, concepts and solutions across initially distant disciplines."

Text Analysis Conference (TAC 2009) workshops will be held November 16-17, 2009 at the National Institute of Standards and Technology in Gaithersburg, Maryland, co-located with the Text REtrieval Conference (TREC), November 17-20, 2009.

Mining User-Generated Content for Security (MINUCS 2009) will take place December 9, 2009, in Venice, Italy, colocated with the First International Conference on User Centric Media (UCMedia 2009) in Venice, 9-11 December 2009. "The aim of this workshop is to bring together researchers from academia and industry who develop technologies for mining open-source user-generated textual data on the Web, as well as end-users interested in exploiting such technologies for knowledge discovery. The emphasis is placed on large-scale text mining systems..."