Using LUIS inside FormFlow

Using LUIS inside FormFlow – Bot Framework

FormFlow is highly powerful and highly useful when you want to collect a set of predefined field data from user and don't wish to create separate dialogs for that. But there are some limitations to it. One such limitation is that, when prompted for an integer or any specific type data type question, say for e.g. "What is your age?", the end user might tend to enter a response like "My age is 23". But here since the bot is expecting an integer type input, it won't understand the user's response and throw in an error and re-prompt the question. This highly disrupts the conversion flow, user experience and makes the bot less likely to be re-used.

So what we need is to add more intelligence to the fields inside the FormFlow and LUIS is the perfect cognitive service which does this. As it is we use LUIS in a BOT (most of the time), now we need to make the FormFlow fields also be processed through LUIS.

The approach we are gonna take is call the LUIS API in the validate method of the form flow field.

Add a LuisModelEndpoint key to your web.config file. The value would be the LUIS endpoint URL exposed by your Luis app. You can find the endpoint by going to the Publish tab of your Luis app.

Your web.config will look something like this :

<appSettings>
  <!-- update these with your BotId, Microsoft App Id and your Microsoft App Password-->
  <add key="BotId" value="YourBotId" />
  <add key="MicrosoftAppId" value="" />
  <add key="MicrosoftAppPassword" value="" />
  <add key="LuisModelEndpoint" value="https://westus.api.cognitive.microsoft.com/luis/v2.0/apps/YOUR_MODEL_ID?subscription-key=YOUR_SUBSCRIPTION_KEY&verbose=true&timezoneOffset=0&q="/>
</appSettings>

Now let's add the Luis response class so that we can deserialize the API response. Create a class called LUISOutput and paste in the below code.

public class LUISOutput
{
    public string query { get; set; }
    public LUISIntent[] intents { get; set; }
    public LUISEntity[] entities { get; set; }
}
public class LUISEntity
{
    public string Entity { get; set; }
    public string Type { get; set; }
    public string StartIndex { get; set; }
    public string EndIndex { get; set; }
    public float Score { get; set; }
}
public class LUISIntent
{
    public string Intent { get; set; }
    public float Score { get; set; }
}

Inside your FormFlow class, add a method named GetIntentAndEntitiesFromLUIS which will query Luis for the user input and find in the entities and the intents. The code would be:

public static async Task<LUISOutput> GetIntentAndEntitiesFromLUIS(string Query)
    {
        Query = Uri.EscapeDataString(Query);
        LUISOutput luisData = new LUISOutput();
        try
        {
            using (HttpClient client = new HttpClient())
            {
                string RequestURI = WebConfigurationManager.AppSettings["LuisModelEndpoint"] + Query;
                HttpResponseMessage msg = await client.GetAsync(RequestURI);
                if (msg.IsSuccessStatusCode)
                {
                    var JsonDataResponse = await msg.Content.ReadAsStringAsync();
                    luisData = JsonConvert.DeserializeObject<LUISOutput> (JsonDataResponse);
                }
            }
        }
        catch (Exception ex)
        {

        }
        return luisData;
    }

The base structure is ready. Now you could call this method in the validate function of your individual form flow fields. For example, lets take a Patient registration form flow which asks for 'Name', 'Gender', 'Phone Number', 'DOB'. If I want to apply luis to the Name field, the code would be :

return new FormBuilder<RegisterPatientForm>()
        .Field(nameof(person_name),
        validate: async (state, response) =>
        {
            var result = new ValidateResult { IsValid = true, Value = response };

            //Query LUIS and get the response
            LUISOutput LuisOutput = await GetIntentAndEntitiesFromLUIS((string)response);

            //Now you have the intents and entities in LuisOutput object
            //See if your entity is present in the intent and then retrieve the value
            if (LuisOutput != null && Array.Find(LuisOutput.intents, intent => intent.Intent == "GetName") != null)
            {
                LUISEntity LuisEntity = Array.Find(LuisOutput.entities, element => element.Type == "name");

                if (LuisEntity != null)
                {
                    //Store the found response in resut
                    result.Value = CultureInfo.CurrentCulture.TextInfo.ToTitleCase(LuisEntity.Entity);
                }
                else
                {
                    //Name not found in the response
                    result.IsValid = false;
                }
            }
            else if (LuisOutput != null && Array.Find(LuisOutput.intents, intent => intent.Intent == "None") != null)
            {
                //In case of none intent assume that the user entered a name
                result.Value = LuisOutput.query;
            }
            else
            {
                result.IsValid = false;
            }
            return result;
        })

That's all, now on user input, the formflow will call luis and based on the response, the values would be set into the form fields.

You can find the complete project solution in my GitHub repo.

Using LUIS inside FormFlow – Bot Framework

Comments

More from this blog

Containerizing Python Durable Functions: A Step-by-Step Guide for Docker and AKS

Microsoft Foundry (New) vs Foundry (Classic) — What Actually Changed?

How Fast is ASP.NET Core Web API? A Performance Analysis

Comparing List Loop Performance in .NET: From .NET Framework to .NET 9-preview

Secure Your Azure Functions: Use Managed Identity for AzureWebJobsStorage

Command Palette

Comments

More from this blog