Discovering the Value “Inside the Conversation”

“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”

                                                                                                                        Albert Einstein

In my previous post I discussed the fact that in the worlds of marketing and sales, structured data is dwarfed by the vast amount of unstructured data that exists. Research shows that at least 80% of all data that exists today is unstructured. (Help me determine the real number as it relates to sales. Take the 5-minute survey here).

Structured data is represented in fields and values in your marketing automation, CRM and other sales and marketing systems. Since this data exists in a structure, it can be reported, analyzed and clearly presented by numerous tools.

Unstructured data is contained in emails, conversations, documents, chats, text messages and social interactions. It is the data “inside the conversation”. Today, some of this unstructured data may be captured, but most isn’t. Even the data that is captured is not analyzed or acted upon because analyzing it requires human interpretation; someone has to read it; someone has to listen to it. We just don’t have the time to do that; therefore we discard the value associated with 80% of the data that exists! What a waste.

Solutions are beginning to emerge that analyze these massive amounts of rich unstructured data. The most well known is IBM Watson. Watson uses natural language processing (NLP) and machine learning to reveal insights from large amounts of unstructured data.[1]

The difficulty with systems like Watson is the complexity of implementation. These NLP systems typically require vast amounts of training to be able to understand and interpret unstructured data. The training involves context of the industry, the market, the vocabulary (dictionary) and the specific problem being solved. For example, training the system to understand symptoms and indicators of certain diseases to assist a doctor in diagnosis. Often the project of training an NLP system involves person-years of effort by expensive data and linguistic scientists…even when aided by machine learning. These projects require huge infrastructure costs due to dictionary models that are difficult to scale. Therefore, solutions involving NLP have been out of reach for small and medium businesses due to the sheer cost, questionable accuracy and complexity of implementation and maintenance.

However, that paradigm is changing rapidly. New advances occurring in the domain of deep meaning analysis promise a world where NLP systems require minimal to no training. This means effortless and rapid, (and inexpensive) implementation. Finally, access to the hidden value inside the conversation will be available to everyone.

This change is coming soon, so now is the time to prepare. In my next post I’ll cover a few steps that you can take now to be ready for the not-to-distant future!

[1] http://www.ibm.com/smarterplanet/us/en/ibmwatson/what-is-watson.html

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published.