There are two entities which are totally unavoidable for all of us:
- One is ‘limited information’, we live in a world of limited information as we do not have access to all the data that we need in order to help us understand the situation and scenarios of a concern.
- The other entity is ‘variety’ or variations, all data we look at is non-uniform and therein lies the problem and it is precisely, limited information and variety, occurring at the same time.
We talk about AI, we talk about statistical computing, and we talk about machine learning. At the end of the day we need to understand the context of the interest by seeking results from a small sample of data, then we can generate it to the entire population, build models, build predictive analytics capabilities so as to surface patterns.
Things are different at this very moment and time. We have impressive technologies and advanced capabilities for us to process Big Data. Therein lies the Big Data platforms that we so often hear about like Spark, distributed computing, supercomputers, high-performance computers and so on. Hence, this concept of data is very important, because the availability and the volume of accessible data will decide on how one may want to implement as well as kick-start one's AI strategy.
All this brings us to the subject of discussion “An effective chatbot strategy”. Here, Johnson Poh is trying to paint a landscape of the fundamental concepts relating to Chatbots. Generative based chatbot models are beginning to emerge but are tough to implement
From retrieval based to generative based – Chatbots
Data troves of messaging conversations will make a chatbot system smarter and more human-like.
Messaging App active users > Social Network active users
Messaging has become a major way of interaction, and companies want Chatbots to be a part of the conversations.Google Trends
A lot of day in and day out applications are going to move to the Chatbot landscape. We are pushing the concept of firms one step ahead, in fact, Satya Nadella CEO of Microsoft made a bold statement on the fact that “Chatbot is probably going to be our new application platform or the new app store”. Moving ahead, people will shift from building applications that require multiple levels of engagement like downloading, opening the app & subsequent learning of using the app, to everything being hosted in the Chatbot platform.
Virtual assistants will continue to grow smarter with each use, holding context based follow up conversations. The sophistication of retrieval based bots depends on a composite of machine learning classifiers.
A range of Machine Learning Classifiers (Supervised)
|Rule-Based Techniques||NLP Techniques||ML Model Classification techniques to
enhance model precision
|Examples :||Examples :||Examples :|
|Decision Tree Learning||Text classification||Association Rule Learning|
|Question Answering||Language modeling||Inductive Logic Programming|
|Speech Recognition||Support Vector Machines|
|Machine translation||Bayesian Networks|
|Documentation||Similarity & Metric Learning|
|Summarization||Similarity & Metric Learning|
As a matter of fact, Microsoft Azure has a bot framework that allows one to spin out a Chatbot by just submitting simple FAQ’s within the company website and one can build a bot in a very short span of time.
Chatbots with strong business and user value have a combination of NLP techniques and ML classifiers in a closed domain.Closed Domain
- Limits Conversation scope to specific goals
- Solves the problem in a targeted context
- Pointed conversations
Most of the bots are functioning in the closed domain where the structure and content are well defined so that the bot is able to retrieve most relevant answer on a query.
How quickly can we move from closed domain retrieval based Chatbot to closed domain generative based Chatbot?
With more people using AI personal assistants, more data will power neural network models and improve the value of generative bots.
“Give a man a fish and you feed him for a day. Teach him how to fish and you feed him for a lifetime” – Lao Tzu
Same is true with generative bots.
- Volume –Huge amount of training data required
- Structure – Robust taxonomy and ontology
- Model – Create a robust technique for learning – a combination of ML classifiers and recurrent neural network (RNN) modeling, which is effective in NLP. Content - Closed domain with specific goals to solve specific problems.