Everything You Need to Know About Chatbots

Since Lexsys was founded around 25 years ago, we’ve encountered more than a few technological developments in our field. As a provider of language services and related content, we’re particularly fascinated by AI chatbots that understand natural language. They’re surely one of the most exciting breakthroughs we’ve seen in recent years, and a perfect example of how cutting-edge technology and linguistic expertise can work hand in hand. 

In this article, we’ll cover the following:

  • What chatbots are and how they work 
  • The benefits they offer  
  • What makes a good chatbot 
  • The technical and linguistic challenges they present. 

What exactly is a chatbot, anyway? 

As the word suggests, a chatbot is a type of robot that can chat with people online and answer questions they either type in or speak out loud. State-of-the-art chatbots use artificial intelligence for this purpose. 

These days, it’s hard to imagine everyday life without chatbots. They often pop up in the corner of your web browser, offering to help with your urgent queries. You see them in messaging applications like Facebook Messenger or WhatsApp, and they also take the form of digital assistants like Apple’s Siri or Amazon’s Alexa, who promise to make our lives easier in various ways.  

How does a chatbot work? 

There are basically two types of chatbot: 

  • Rule-based chatbots are programmed to match specific answers to specific questions. They don’t actually understand language, however, which significantly limits the ways in which they can be used. To parse a user’s written or spoken question, they break it down into its individual components and process them according to set rules while accessing a database to come up with an appropriate response. 
  • AI chatbots, on the other hand, make use of natural language processing (NLP). This means they understand user input and keep learning from it, which helps them constantly improve. Besides answering simple questions, they can also set up and cancel appointments and even send out invoices on their own. 

The advantages of chatbots 

Chatbots can be a boon to companies and customers alike. While they’re no substitute for personal customer service, they are great at supplementing such activities. In fact, chatbots offer a variety of benefits: 

  • Not only can they support customers in a similar way to human employees; they can do this around the clock, including outside normal business hours. 
  • They can lighten the load on customer service employees by answering frequently asked questions
  • They communicate with customers via the channels they prefer and often pool the resulting data in a central repository (a CRM system, for example) that can be accessed by employees from different departments. 
  • They qualify leads and ensure that only potential customers with real promise are forwarded on to employees in sales. 

What makes a good chatbot? 

If your company wants to take advantage of chatbots, they’ll need to meet certain requirements. A potential customer who keeps getting the response “Sorry, I’m afraid I can’t help you with that” will soon take their business elsewhere. 

Computational linguistics and natural language processing 

As we touched on above, intelligent chatbots guided by AI are capable of processing natural language. They understand and can simulate both written and spoken human speech, and can even carry out simple automated tasks.

The expertise needed to make this a reality can be found in the area where linguistics and computer science overlap, which is why it’s referred to as computational linguistics or linguistic data processing. 

An AI chatbot starts by turning human input into statements and intentions. Once it has identified the purpose of a given message, it determines an appropriate response and provides it to the user. This process is then repeated until the conversation ends.

Natural language processing essentially follows the same procedure, with slight variations depending on the model at hand. The Saarbrücker Pipeline Model, for instance, consists of six steps: 

  1. Speech recognition: Spoken input is first converted into text.
  2. Tokenization: Each sentence of input is divided into words and groups thereof.
  3. Morphological analysis: Personal forms and case markers are analyzed and words are reduced to their basic forms.
  4. Syntactic analysis: The individual words are analyzed to determine their structural function (subject, object, article, verb, and so on).
  5. Semantic analysis: Meaning is attributed to the sentences or parts thereof.
  6. Dialog and discourse analysis: The relationships between the consecutive sentences are identified.

Dealing with grammar mistakes and typos 

As if all that weren’t enough, chatbots also need to be able to handle input that contains errors in grammar or spelling and spoken input from non-native speakers with accents, as well as from people who talk in a local dialect. Not everyone speaks the Queen’s English and when typing in a hurry on a phone, one’s fingers don’t always find the right letters. 

How chatbots combine linguistics and technology 

Thanks to advances in deep learning and machine learning, the algorithms chatbots run on are now much better (and faster) at comprehending human speech and the intentions behind it. 

Let’s take a look at a simple example: weather forecasts. There are myriad ways to ask what you can expect in the days ahead: 

  • “How’s the weather going to be tomorrow?”
  • “Is it going to rain tomorrow?”
  • “Will it rain on Tuesday?”
  • “Will I need my umbrella tomorrow?”
  • “What’s the chance of rain tomorrow?”
  • “Any rain tomorrow?”

Along with an extensive vocabulary that covers all the relevant situations, a chatbot needs to be familiar with a wide range of syntactic structures, various modes of expression, and different registers – from laid-back to strictly formal. In the case above, the bot will also require information regarding the user’s location. 

All this means that chatbots need access to a comprehensive database of sentences that mean the same thing, but use different phrasing. A compendium like this takes time to build, as well: A good chatbot should know at least 50 expressions for a given intention. 

Meanwhile, there are two types of linguistic experts who play key roles in designing chatbots and making them user-friendly: 

  • Conversation designers give chatbots a voice, shape the progression of their dialog, and compose the texts they use. They also determine the personality of each bot by choosing whether to give it a sense of humor, the ability to use slang, and formal or informal responses.
  • Computational linguists are the ones who lay the groundwork for chatbots. They develop the lexical resources a chatbot uses in a particular language. In this context, the term “lexicon” refers to both the entire vocabulary of the language in question and the specific terms used by the company and industry in question. In other words, a chatbot for a car dealer will be very different from one used by hair salons.

Lexsys: the partner you can count on in chatbot projects  

Here at Lexsys, we can provide you with a wide variety of content and language services that will support you in dealing with your communication-related challenges. We speak your customers’ language and can help you make sure your message gets across just as it should – including in the automated responses your chatbots provide.

We’d love to hear what your experience has been with chatbots and what you think of them in general. Perhaps you’d like to leave us a comment below…?

Want to learn more about how Lexsys can help you overcome the challenges chatbots present? Get in touch with us today, and we’ll be happy to set up a consultation.

About the Author