What is natural language processing? A beginner’s guide

During a conversation, how often do you think about your language? Are you aware of grammar rules, or consciously analysing the meaning behind your language?

Humans don’t tend to think about the intricacies of language when communicating. But for machines, it’s a different affair. For a machine, human language can get confusing. Quickly.

That’s where natural language processing enters the fray. But what is natural language processing, and how does it allow machines to hold conversations?

What is natural language processing?

Natural language processing is an area of artificial intelligence. Sometimes shortened to ‘NLP’, it’s concerned with how humans and machines interact. It focuses on teaching machines to understand natural language.

To clarify, natural language differs somewhat from formal, Queen’s English rhetoric. It’s how people talk and use language in normal, everyday interactions. (Which is markedly different than how we’d write an essay, for example.)

So, what is natural language processing? It’s all about letting us interact with machines using the same language that we use when interacting with other humans.

A look behind the curtain

In the early days, the answer to ‘what is natural language processing’ was ‘rule-based processing’. This means that to understand our language, machines needed step-by-step instructions. These rules would outline how to analyse and respond to natural language input.

But Moravec’s paradox — where things that come naturally to humans are the hardest to teach machines — came into play.

Machine language is nothing like natural human language. Telling machines what to do requires very specific rules and instructions.

Natural human language, meanwhile, is ambiguous. It relies not just on words and rules, but idioms, double meanings, context and shortcuts. As such, creating rules for the intricacies of human language is a tall order.

So, the latest natural language processing relies heavily on machine learning. This means that machines now learn how to understand language by comparing masses of examples. From these examples, the machines identify patterns. Then, they apply those patterns to new language input to derive meaning.

When is it useful?

So, what is natural language processing used for?

One of the most common examples is chatbots. Without natural language processing, chatbots would be unable to understand or respond to the messages you send them. Not unless, that is, you phrased them with an exactly set syntax.

Voice assistants like Alexa or Google home are another example of NLP at work. They couldn’t understand your voice commands without natural language processing.

If you’ve ever used (or argued with) autocorrect or grammar and spell checkers, you’ve interacted with NLP. If the computer cannot understand human language, it couldn’t alert you when you’ve written something incorrectly.

Natural language processing also enables hands-free computing, making it useful for improving accessibility. With it, people can tell the machine what to do verbally, or even have the machine transcribe their spoken words. It’s also a component of machine translation.

These are just a few examples of the ways that we use natural language processing today.

Natural language processing

What is natural language processing? It’s teaching machines to understand us when we write or talk in human languages. Instead of, that is, only understanding us when we use machine languages. (Also known as code.)

Teaching machines to understand us is a major challenge for artificial intelligence engineers. But with machine learning, progress in natural language processing is well underway.

Useful links

Types of AI: distinguishing between weak, strong, and super AI

What is Moravec’s paradox and what does it mean for modern AI?

What is machine learning? A beginner’s guide