What is your name?

Asking a persons name in Conversation allows you to personalise responses later on. However actually getting a name from a person which looks good in personalisation is quite hard.

First thing to consider is how you ask the question. Take these examples.

  1. What is your name?
  2. Can I get your name?
  3. How do you like to be known as?

All three of these can illicit different responses from the end user, some not the name at all. Possible responses can be.

  • (1) Simon O’Doherty
  • (2) Why do you want to know my name?
  • (2) No
  • (3) Hugh Jass

None of these answers are ideal. The first one will look silly in personalisation. The next two need to be addressed. The last one is just silly.

So at this point is very important to shape what you send to the user. Actually everything you write, especially in process flow you need to be mindful of how you shape your message.

For this example we are going to use “Hello. Before we begin, can I get your name?“. This is liable to get responses as shown above about “why” and “no”. But you will also find that people will often not even read what is being said. So you could also expect a question.

To deal with this, we can create a simple entity to look for common key terms.

conv-0911-1

The reason to use an entity over intents is that in a process flow, the scope of questions is very narrow, and we don’t want to interfere with any other intent training that is required.

We can take the following actions on each entity.

@NameOptions:why
Output: “I need your name so I can be more personalised. Can I get your name?” and ask again.

@NameOptions:question
Output: “Before you ask me a question, can I get your name?” and ask again.

Don’t try to be everything to everyone!

I have seen people try to have Conversation answer any question at any time. This can cause serious issues in process flows, and make it seem more like a search engine.

When creating a process flow, don’t assume that the end user will do everything. Test with your users first to see where you need to customise behaviour.

 
@NameOptions:no
We don’t want to force the person to give their name. So we can customise this with the following advanced code:

{
    "output": {
        "text": "No problem, you don't need to tell me your name. Please ask me a question"
    },
    "context": {
        "name": ""
    }
}

We are setting the context variable name to blank, so that personalisation doesn’t have a name.

Condition: input.text != “”
This node is to actually load the persons name using the following advanced code:

{
  "output": {
    "text": "Hello <? input.text ?>. Please ask me a question."
  },
  "context": {
    "name": " <? input.text ?>"
  }
}

Condition: true
Output: “I didn’t quite get that.” and ask again.
We use this final node in case someone enters nothing. We just send back to ask again.

Here is the sample script for above.

Avoid too much personalisation!

When personalising responses, you should avoid doing it too often. Having the persons name appear infrequently has much more of an impact.

Try to create multiple possible responses and have 1 out of every 4-5 personalised and random responses.

Outstanding Issues

So while this will work, there are still a number of issues you need to factor in.

Too much information: What happens if they answer “My name is Simon O’Doherty”. Conversation cannot handle this, and will use the whole text .

To work around this, you can try passing the users input to AlchemyAPI and Entity extraction to get the persons name. Running on the text above gives the following output (truncated):

"entities": [
        {
            "type": "Person",
            "relevance": "0.33",
            "count": "2",
            "text": "Simon O'Doherty"
        }
    ]

There are still issues with this though, which you will find when you experiment with it. Another option is a simple UIMA dictionary which detects names. I did an example of this long ago, although using WEX or Watson Knowledge Studio may make it a lot easier.

gbenj8n

Bad information: Apart from the joke or offensive names, allowing the user to personalise responses means your system is open to abuse. So you will need to do some sanitisation at the application layer to stop this.

Response formatting: In the example script you will see that context name is being set by ” <? context.name ?>”. However when you test, you will see the preceding space is removed. Without this, you will get some odd formatting like “Hello .” if no name is specified. You need to correct this at the application layer.

Tools of the trade – UltraEdit

So I use a number of applications to help in building and testing Watson. But the main ones I use are as follows.

  • Ultraedit
  • SPSS Modeller
  • Excel
  • Languages: Java and Python
  • iPad (no seriously :))

I want to touch on UltraEdit for this one. I primarily use it for the following.

  • Making mass fixes to data that SPSS can’t easily handle, and languages take too long.
  • Making the JSON readable
  • Extracting meaningful data from JSON to work with.
  • Putting that data back easily.

It is a pretty powerful text editor which allows you to do very complex search/replace/modifications to data. I am of the opinion if you prefer a different text editor, then go with what you know. But here are the main scripts I use for UltraEdit.

When you open the conversation JSON file it will look a little bit like this.

conv-0907-1

Just a mess. There is two scripts I picked up from UltraEdit that allow you to decompress it and compress it back to what you see above.

JSON – readable

if (!UltraEdit.columnMode) {
if (!UltraEdit.columnMode) {if (!UltraEdit.activeDocument.isSel()) {
UltraEdit.activeDocument.selectAll();
}
var jsonText = UltraEdit.activeDocument.selection;
var json = JSON.parse(jsonText);
var jsonTextFormatted = JSON.stringify(json, null, 2);
UltraEdit.activeDocument.write(jsonTextFormatted);
}

Will turn it into this.

conv-0907-2

To turn it back again.

JSON – compress

if (!UltraEdit.columnMode) {
if (!UltraEdit.activeDocument.isSel()) {
UltraEdit.activeDocument.selectAll();
}
var jsonText = UltraEdit.activeDocument.selection;
var json = JSON.parse(jsonText);
var jsonTextFormatted = JSON.stringify(json, null, null);
UltraEdit.activeDocument.write(jsonTextFormatted);
}

Now there are purists out there that can live with reading JSON. I’m sure all they can see is blond, brunette, redheads. For me I prefer to have my data with a lot less noise.

To that end I created two macros for extracting the entities and intents from the conversation file, into a format you can open in excel/SPSS.

Conversation – get intents

if (!UltraEdit.columnMode) {
if (!UltraEdit.activeDocument.isSel()) {
UltraEdit.activeDocument.selectAll();
}
var jsonText = UltraEdit.activeDocument.selection;
var json = JSON.parse(jsonText);UltraEdit.newFile();
var intents = json.intents;UltraEdit.activeDocument.write(‘”Question”,”Intent”\n’);
for (var i = 0; i < intents.length; i++) {      var intent = intents[i].intent
var examples = intents[i].examples      for (var j = 0; j < examples.length; j++) {
UltraEdit.activeDocument.write(‘”‘ + examples[j].text + ‘”,”‘ + intent + ‘”\n’);
}}}

Creates a CSV file you can work with.

conv-0907-3

Conversation – get entities

if (!UltraEdit.columnMode) {
if (!UltraEdit.activeDocument.isSel()) {
UltraEdit.activeDocument.selectAll();
}var jsonText = UltraEdit.activeDocument.selection;
var json = JSON.parse(jsonText);UltraEdit.newFile();var entities = json.entities;
UltraEdit.activeDocument.write(‘Entity\tValue\tSynonyms\n’)
for (var i = 0; i < entities.length; i++) {var entity = entities[i].entity
var values = entities[i].valuesfor (var j = 0; j < values.length; j++) {
UltraEdit.activeDocument.write(entity + ‘\t’ + values[j].value + ‘\t’);var synonyms = values[j].synonyms
var syn = ‘ ‘
for (var k = 0; k < synonyms.length; k++) {
syn += ‘”‘ + synonyms[k] + ‘”,’
}
UltraEdit.activeDocument.write(syn + ‘\n’)
}}}

This one creates a Tabbed separated file (TSV) with the entities in a nice readable format.

conv-0907-4

I also have scripts to put the intents and entities back into the conversation script, but I don’t plan to release at this time. As it is so easy to make a mess, and I won’t be supporting these.

You can pick up the scripts here.

Handling low confidence answers in Conversation.

So I will be switching between, newbie to expert in no particular order. This post assumes you already know how to use conversation.

In earlier versions of Watson, it was designed to handle high, medium and low confidence answers in different ways. With conversation this all works a little bit differently.

For this example, I am using the NLC demo intents. The first thing you have to do is find your low confidence.

Unlike earlier versions of WEA, the confidence is relative to the number of intents you have. So the quickest way to find the lowest confidence is to send a really ambiguous word.

These are the results I get for determining temperature or conditions.

treehouse = conditions / 0.5940327076534431
goldfish = conditions / 0.5940327076534431
music = conditions / 0.5940327076534431

See a pattern? 🙂 So the low confidence level I will set at 0.6. Next is to determine the higher confidence range. You can do this by mixing intents within the same question text. It may take a few goes to get a reasonable result.

These are results from trying this (C = Conditions, T = Temperature).

hot rain = T/0.7710267712183176, C/0.22897322878168241
windy desert = C/0.8597747113239446, T/0.14022528867605547
ice wind = C/0.5940327076534431, T/0.405967292346557

I purposely left out high confidence ones. In this I am going to go with 0.8 as the high confidence level.

Once you have those, you can create your conversation. Your first node is to check that later nodes won’t fail.

conv0905-1

The next nodes check to see if a low confidence or medium confidence is hit. For low, it won’t respond. For medium, it will display text, then continue from the next condition to find the actual answer.

conv0905-2

When you test it, you get the following results.

conv0905-3

You may still need to test with users to tweak the upper value.

Using this fall through method makes it easier to maintain with your intents. You can use a nested option if only certain intents need to be hit.

Here is the Sample file.

Gotchas: When writing numbers in conditions, always start it with a numerical value like “0.8”, not “.8”. Otherwise you will get a Dialog node error: EL1049E:(pos 24): Unexpected data after ‘.’: ‘8’

Do you even need a chat bot?

With everyone rushing out to create bots, I am reminded of the Jurassic Park quote: “Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should”. 

Despite what people will tell you, conversation or chat bots for that matter are not needed for everything.

For example where a simple web form will do, it will out perform against a chat bot. But if that same form could have most of it answered by one question, then a chat bot may be a better solution (or maybe you just need a better form).

Or simply changing your internal processes may negate the need for a chat bot. For example, imagine after analysing your support calls for training the bot you find that over 90% of calls are due to one printer. Do you create the bot, or just replace the printer?

Or if your customer knows your domain language then a search engine or Retrieve and Rank may be a better solution.

IBM normally does all this checking through what we call a Cognitive Value Assessment (CVA). A well done CVA reduces all head aches on projects. Even if you don’t go with IBM, you should realistically examine your business process to determine if you even need a chat bot, and not just jumping on the bandwagon.

Why Conversation?

Most of the chat bots out there are just messaging frameworks, that allow you as a developer to interact with existing messaging systems. How you interpret, talk, react is all handled through code.

Out of the box, where Watson Conversation excels is the ability to take a knowledge domain (ie. Customer) that doesn’t directly match your Domain knowledge. With a handful of questions you can have your conversational bot answering questions it has never seen before.

Conversation also makes it relatively easy to write out your conversational flow (also known as chat-flow & process flow, depending on how long you worked in Watson).

Conversation is meant to piggy back on existing messaging frameworks to build intelligence into them with ease.

The danger of making things easy, is that people skip the theory and go straight to the development. The older people in the audience will remember when Visual Basic (or Lotus Notes for that matter) came out. It became easy to create applications, but most were travesties in UI, maintainability and functionality.

My focus going forward is to cover more the theory end to resolve this. As I am already competing with a number of blogs + videos in relation to Conversation.

Clean Slate

It’s been a couple of years since I last touched the blog. I thought I would start from scratch. I have everything backed up in case someone is looking for something (stats say otherwise).

I thought I would focus on conversational bots, as most of my work has been in this area. Mainly Watson related though.