NLP and NLU – what’s the difference?
In connection with chatbots one stumbles upon terms such as NLP (Natural Language Processing) and NLU (Natural Language Understanding). What do these terms entail and what is the difference between them?
Natural Language Processing – Searching a Text for Clues on Meaning
To put it bluntly, NLP stands for “flat” techniques in the development of software that searches a text for clues in order to approximate its intended meaning. Among those clues are certain keywords. Many simple chatbots work with such techniques: They search the user input for keywords, and in response deliver a predetermined answer that was saved under said keyword. In online shops – the area of chatbot applications – customers ask a lot of questions that include word such as “price” or “costs”. A chatbot that is based on NLP would most likely refer to a price list. A more sophisticated variant of NLP could lead the chatbot to identify two terms at once. If keywords like “price” or “costs” appear in conjunction with the name of a product, the chatbot would be able to reveal the price of said product.
The bottom line (advantages and disadvantages): NLP is a quick software-powered way of analyzing text for its meaning. Identifying keywords in simple questions delivers a satisfying success rate. NLP hits its limits when faced with more complicated questions or when instead of keywords, intentions have to be identified.
Natural Language Understanding – Comprehending the Meaning
By contrast to NLP, NLU (Natural Language Understanding) deals with comprehending the meaning of a question or a statement in detail. This includes:
- identifying subject and object, as well as other optional expressions
- detecting the process or the context of a question
- and connecting pronouns to their corresponding nouns
Semantic and context sensitive pragmatic analyses have to be employed for this. Questions like “What milk chocolate do you have?” -(answer)- “How much does it cost?” can be answered satisfyingly because the chatbot detects that “it” refers to “milk chocolate” through the context of the question, while the user does not have to mention the desired object by name. More complex questions such as “Can I return this item if it’s already open?” can be answered that way. A “normal” chatbot that “only” operates on NLP would not stand a chance when faced with such questions.
Requirements for NLU are therefore:
- an extensive lexicon with the different meanings of words (as most words can hold several meanings)
- detailed sentence parsing (the disassembly into components)
- and an extensive context modelling
Perfect understanding of human communication might not be obtainable today. We at Kauz, however, improve these techniques continuously. This results in our chatbots learning to understand more questions and to answer them more satisfyingly on a daily basis.