How we designed and developed an AI chatbot for the first time - Artificial Intelligence (AI) Chatbot in Malaysia, For You

How we designed and developed an AI chatbot for the first time

An illustration of Malaysia’s popular attraction — Bukit Tinggi, Pahang

It is mid-2018. The air here at XIMNET is abuzz with a nervous excitement — we had just landed our first chatbot project.

‘Artificial intelligence — the future of tech!’

‘NLP — the next big thing in business automation!’

Up until now, these were just things we had watched grow from the sidelines, with equal amounts of skepticism and trepidation. Just words we’d throw around the office: “AI? Yeah, we’ll see.” But now we were going to have a stake in it.

But where do we begin?

Choosing a Platform

ven way back then, there were plenty of Natural Language Understanding (NLU) platforms to choose from, and no shortage of advocates for each provider.

Google’s Dialogflow seemed like an obvious choice: Creepily accurate answers to our sometimes nonsensical questions is basically their whole business model.

On the other hand, IBM’s Watson had already made big waves with their Jeopardy win, and was now poised to revolutionise healthcare (a promise still pending).

And what about Microsoft’s Azure Bot?

In retrospect, it really didn’t matter which one we chose — we ended up trying them all anyway as we ventured deeper and deeper into our chatbot journey, eventually going beyond the platforms to explore the underlying models for ourselves (Huggingface is a great resource for this).

Our takeaway: You can’t go wrong with either platform. Do a quick comparison and just get started.

Identifying the Problems

Despite what our optimistic selves would have us believe, chatbots are not magical, omniscient beings. By this point, everyone in the team had had their own horror stories about having to deal with an ineffective chatbot — creating a meaningful one was going to be a deliberate effort.

Having a long history of building solutions for the web, we knew we needed to prioritise.

Enter, the 80–20 rule. For the uninitiated, here’s a quick breakdown from Investopedia:

The 80–20 rule, also known as the Pareto Principle, is an aphorism which asserts that 80% of outcomes (or outputs) result from 20% of all causes (or inputs) for any given event. In business, a goal of the 80–20 rule is to identify inputs that are potentially the most productive and make them the priority.

In chatbot terms, if we identify the right 20% of problems to address, we will be able to satisfy the needs of 80% of users. But how do we find those problems? Here are a few things we did to get the conversation going:

Profile the users: Who will be using the chatbot? Is it going to be on a public-facing channel, or will it be behind a log-in screen? Taking a user’s point of view serves as a good starting point into identifying what would be most helpful without the bias of being on the inside.

Speak to the stakeholders: When speaking to stakeholders, there are bound to be specific problems that they need addressed. Is there a painful but key business process, long overdue for automation? What are the repetitive tasks bogging them down from more productive work?

Get down to the sources: How are the users currently communicating with the brand? If there are any logs available (e-mail trails, Messenger chat histories, etc.) it will be good to get a look through as this will not only help in identifying problems, but also serve as a guide to building a feasible solution.

Once we had pinpointed the problems we wanted to address, it was time to get to work.

Establishing a Foundation

While we were designing our chatbot, it was very tempting to become mired in the world of utterances, intents, entities, dialogs and responses. And for a while, it felt like duty for us to educate and encourage more people to do the same.

But reality doesn’t quite work that way, and we found that it made discussions tedious and frustrating for everyone involved. If we were going to come up with something great, we had to start communicating effectively, and fast.

In the end, we boiled down the problems into a series of process flows: The idea being that each dialog could be essentially reduced to a flow chart. Sure, it does leave out quite a bit of the nitty-gritty (e.g. How does a user tell the chatbot to get recent transactions?), but it gives all stakeholders a chance to be involved even at the early stages of chatbot building.

Developing Human-First

Over the course of building our chatbot, it started to become painfully apparent that the old adage is really true: ‘No two humans are alike’. From slight variations in word choice to complete changes in sentence structure, when it comes to developing intents, you really need to catch ’em all. It pays to be thorough, and it is always a good idea to involve more than a few brains.

On the other hand, when it comes to responses we found it best to keep it short and sweet. While it is always tempting to provide all the information at one go in a chatbot message, often times all this does is overload the user, resulting in skimming and subsequent confusion. As a rule, it is much better to provide only information that is necessary, directing users to additional information when prompted.

And as always, when it comes to anything human — test, test, and test.

We launched our first chatbot — Affin Hwang AM’s NADIA in late November, 2018 to great results. Over the years we have built new chatbots and improved upon our overall solution: Teaching our chatbots new tricks every now and then, and in turn, learning some new tricks ourselves.

XIMNET is a digital solutions provider with two decades of track records specialising in web application development, AI Chatbot and system integration.

XIMNET is launching a brand new way of building AI Chatbot with XYAN. Get in touch with us to find out more.

about the article

This article is first published on XIMNET Medium. More resources available here.

XIMNET is one of the leading tech agency for AI Chatbot in Malaysia.