Bots may have been known for decades, but they were used mostly in simple applications rather than for serious business cases. However, recently they became more intelligent, complex, and versatile. Nowadays bots can do much more, with multiple communication channel support and features such as natural language understanding or user intention recognition. They are a part of a cloud revolution which brought software to a new level, providing great availability and scalability for low development effort and maintenance cost. This article is focused on the Bot Framework – Microsoft’s cloud solution for bot development.

A New Way to Communicate with the User

We are used to communicating through web or mobile communicators – millions of people using Facebook Messenger proves that. The good news is that bots can converse with us through some highly anticipated channels. That brings your company communication to a new level. Bot service is available 24 hours per day and can be designed to handle typical questions asked by people, which may reduce the staff needed to maintain such channels. You can provide instant support for your customers, partners and new leads.

Another advantage of this kind of communication interface is the fact that it doesn’t need an expensive and laborious process of bringing up an application UI. A bot usually communicates with the user using plain text through the channel that already provides a user interface. Depending on the channel, it has also a set of built-in components that can be used to display bot responses in a more attractive way. You can use images, videos, audio clips, buttons and lists of items not only to present the content, but also to interact with the user.

Put a Bot in Charge

Before starting a new bot project, you need to consider if it really is a solution for your business case. It’s not recommended to start bot development just because it’s a hot topic. However, in some cases, this kind of software can save a lot of time, money and resources. The following list of bot example use cases might help in making the decision:

• Answer for typical questions
  • A bot can make use of Q&A knowledge to receive user question and provide an appropriate answer.
  • Questions can be matched to correct answers using a LUIS (language understanding intelligent service) cognitive service.
  • Reduced time can be spent by help desk staff answering typical questions.
  • Example use cases are help chat, contact pages and web stores.
• Alternative system interface
  • By integrating a bot with external systems (e.g. Outlook, Jira, CRM, SharePoint) a bot can become an alternative interface to work with these systems.
  • A bot can simply ask some questions and gather the answers given by the user to submit data that normally would be filled in on a form.
  • Example use cases are creating support tickets, uploading SharePoint documents, making calendar appointments, and providing translations.
• Entertainment & education
  • A bot can be also used to entertain and educate its recipients by sending various kinds of content to the user.
  • It’s a good idea to use media types like videos, audio, images and links to knowledge base articles.
  • Example use cases are workout coach, recipes book and product adviser.
• Notification bot
  • A bot can be scheduled to initialize conversations at appropriate time, notifying the user about some actions or reminding about things he should do.
  • It’s important to remember that sending proactive messages is not always possible – it depends on the channel used for communication.
  • Example use cases are meeting reminders and timesheet reminders.

Why Do You Need a Bot?

When I joined a team assembled to develop a bot, I was confused – is it really the kind of software desired by enterprise customers? Then I realized I’m asking the same questions I asked about web applications eight years ago. Since then, a lot of desktop and on-premise systems were migrated to the cloud. Web applications became a crucial part of this process. Maybe it’s also a good idea to give bots a chance?

Bot Framework is cloud-based technology which allows building a bot streamlined with ready-to-use components and services responsible for connectivity, intelligence, and integration. We can then assume that a bot built using this framework has some qualities and features characteristic of cloud-based software:

• Cost efficiency – Azure cloud allows selecting a fitting application service plan, depending on your needs. For applications generating small traffic, the cost is very small or even free.
• Scalability and reliability – you don’t need to bother with what will happen if the traffic rapidly increases. Applications hosted on Azure can scale up to handle higher traffic and scale down when it’s over. The availability of cloud services is also typically much higher compared to in-house IT infrastructure.
• Convenience – by hosting an application on the cloud, we don’t need to care about the infrastructure. This is also significant cost reducer, ruling out the necessity of purchasing, managing and administering a local server room.
• Easy provisioning and deployment – there are tools allowing you to define the details of application services and other resources needed to host it. As an example, you can use ARM templates with Cake build script to build your code, provision resources on target Azure subscription with correct settings, and deploy compiled application from your local command line or build server.
• Integration with other systems – cloud computing makes it easy to connect web application with other systems, sharing the data or events amongst each other. That allows building well-integrated applications with relatively small effort.

However, cloud services may also have some disadvantages:

• Cold starts – an application might get expropriated to release resources for other applications running on the server. That typically happens when the app remains idle for a long time. It’s still available, but the first time it’s called after a long idle period, you might experience a longer loading phase. This disadvantage is undesirable especially for bots, where a quick response is a key feature. However, a cold start is typical for lower app service plans and can be fixed by upgrading the plan.
• Security – a typical web application stores some data in the database. Usually the database is also hosted in the cloud, which mean it’s potentially vulnerable.
• Localization – the fact that cloud servers are located in other countries may become a problem for some customers, depending on their security policy.

Bot Framework Architecture Overview

The Bot Framework divides the bot for some services. This partition allows the developer to focus on bot functionality while the framework handles communication with the user and conversation history storage. The following diagram in Figure 1 provides a general overview of typical bot architecture:

Figure 1. General services architecture of a basic bot

Bot Service (Bot Channel Registration)

The Microsoft Bot Framework comes with out-of-the-box support for number of channels, e.g. Skype, Skype for Business, Microsoft Teams, Facebook Messenger. Bot Service allows defining which channels are enabled for a bot and handles communication with these channels. All messages received by this service are getting posted to bot web API, returning the message that is getting sent back to the user.

Web API Service

The Web API Service provides an endpoint responsible for answering user activities. It’s the developer’s responsibility to implement this logic. However, the Bot Framework makes this job easier by introducing the concept of bot dialogs and a set of interfaces and types. The Bot Web API is also a service in which integration with external systems and services should be implemented.

Bot State Service

The conversation between a bot and a user is the most basic feature the Bot Framework provides. To make it efficient and fast, a conversation state is stored to the Bot State Service each time a new activity is being recorded. It helps to keep the asynchronous nature of Bot Web API and Bot Service. Bot state can be cached in memory or stored in persistent storage. Thanks to the fact that a bot state, kept in a database, is restorable at any point of time, it is possible to develop a bot capable of continuing previously initialized conversations.

Note: In the beginning, Bot Framework shipped with the default State Service which was built into Bot Service. However, it’s going to be deprecated in a future release. It’s recommended to use other storage services like Azure Storage or Cosmos DB for bot state, especially for production bots.

Analyzing Bot Requirements

Bot conversation design might seem to be an easy task – you just need to provide an appropriate answer for predictable questions. However, things are not as simple as they might look, especially for complex projects. A person analyzing the bot requirements needs to think about bot behavior to provide a quality user-friendly experience. The following list presents example scenarios for which the bot should be prepared:

• Starting a new conversation
• Renewing a previously unfinished conversation
• How to react when a new user joins the conversation
• How to react when a user leaves the conversation
• Human-like reactions, e.g. saying hi, goodbye, telling jokes
• Reporting that something went wrong e.g. in case of external system outage
• Handling unrecognized user intents, e.g. fallback to a real person
• Handling alternative utterances of user input
• Processing files uploaded by the user

The most intuitive way to define the bot scope is to design its conversation using dialog scenarios as shown in Figure 2:

Figure 2. Example conversation flow scenario

Bot Development Challenges

Microsoft Bot Framework allows development of great bots. However, it’s just a set of tools – we don’t get a ready-to-use bot right after creating the project. Typically, a bot needs development to bring some functionalities. It’s good to realize what kind of challenges we might be faced with.

Bot Intelligence

If the bot is about to converse with the user, it should have a form of intelligence. The most basic intelligence can be implemented programmatically using if or switch instructions. However, for more complex scenarios it’s usually not enough. Luckily, Microsoft provides the Language Understanding Intelligent Service (LUIS) – a powerful cloud service providing natural language understanding features. This service allows anyone to build a custom cognitive language understanding model in an intuitive way – simply by providing some utterances and matching them with correct intents. Figure 3 shows an example of the design interface.

Figure 3. Defining LUIS intents and utterances.

In addition to that, Microsoft Bot Framework comes with out-of-the box support for LUIS services to provide easy integration. It’s worth keeping in mind that LUIS has a limited quota for a free usage per month, and it might be necessary to upgrade to a paid consumption plan.

Technical Limits

The architecture of Bot Framework is based on web services communicating with each other in an asynchronous way. That is the reason why the user does not receive bot responses immediately. The exact response time depends on the channel, but it’s typically up to a few seconds. However, if bot services are configured for free or basic consumption plans, the response time might elevate to tens of seconds during the cold starts.

Multi-language Support

A basic bot can be implemented to communicate in any language, as long as messages sent by the user can be recognized by its logic. Things get more complicated for bots using LUIS service – it has support for a limited set of languages, and only one of them can be selected per each service instance (Figure 4). This means that, for multi-language bots, there may be a need to register and maintain multiple LUIS services, which might increase the costs.

Figure 4. LUIS app registration form with culture selection

Integration with Other Systems

In some cases, the functionality of a bot depends on integration with other systems. This is potentially a part of the code that might cause long response times. Therefore, the integration between the bot Web API service and an external service needs to be developed carefully, focusing on good performance. If the integration requires time-consuming queries, it might be a good idea to delegate these tasks to other cloud services like Azure Functions or Azure Web Jobs to cache the results in storage available for the bot.

Use Channel Features

It’s worth taking advantage of the features each channel offers to bots, especially because it’s easy to develop. However, you need to be aware that the Bot Framework doesn’t guarantee all components will be available and will look and behave the same way on all available channels. You can check and compare channels features on Bot Framework Channel Inspector.

Developing a Delivery Bot: from Theory to Practice

Creating a new intelligent bot was a challenge when the Bot Framework was initially introduced. However, as the framework is being constantly improved by Microsoft, the development became much easier. Let’s see how to create a new bot with the language understanding feature called Delivery bot.

To start, login to the Azure Portal and create a new resource of type Web App Bot. Alternatively, you can also create a Functions Bot – it would work the same way. For this example, stick to the Web App option shown in Figure 5:

Figure 5. Creating a new Web App Bot resource

In the next screen, you will fill in some important provisioning details. Let’s go through them (Figure 6):

Figure 6. Bot provisioning details form.

• Bot name: the name of the bot, that will be presented to the user. Note that it must be unique across Azure, so you will need to choose a different name.
• Subscription: Azure subscriptions selector
• Resource group: you can add a bot to an existing Azure resource group or create a new one; in this case, create a new group to have all resources separated from other existing ones
• Location: the region in which the bot will be hosted; it’s recommended to pick a location that is close to the target audience to provide shorter responses times.
• Pricing tier: you can pick either free (F0) or paid (S1) plan for your bot.
• App name: internal web application name determining the URL address of the bot messaging endpoint
• Bot template: allows you to initialize a bot messaging endpoint with ready-to-work code; for this example, choose Language understanding (C#) template that will also initialize a new LUIS application.
• App service plan/Location: allows to create or select existing service plan, determining what compute resources will be available for Bot Web API Service. It is recommended to choose a service plan located in the same region as the location of bot resources being created.
• Azure storage: the storage that will be used to keep the state of the bot; in this example, create a new one.
• Application Insights: optional analytics for bot, collecting bot telemetry

Having defined the bot configuration, you can simply click the Create button. After several minutes, the bot is up and running, ready to have a simple chat. You can take a quick look at the new resource group created on you Azure subscription to see the newly created resources shown in Figure 7:

Figure 7. Bot resources on Azure Portal

The bot consists of the following resources:

• Storage account – for bot status storage,
• Web App Bot – bot registration with some advanced settings such as channels configuration,
• App Service – bot messaging endpoint web application,
• App Service Plan – a service plan for bot App Service (only enlisted if a new service plan was created),
• Application Insights – for bot analytics and telemetry.

Teach Your Bot to Understand the Business

Before starting a new conversation, you must put in some development effort to let bot understand your business cases. In this example, the bot is targeted for a logistics company. That’s why you need to teach it to understand some example questions the customer might ask. To do this, you need to dive deeper into the LUIS service. You might have noticed that this service is not available in resources listed on Azure Portal in the new resources group. That’s because LUIS is a separate service and doesn’t get included as an Azure resource. LUIS applications can be explored on the luis.ai portal after signing in using the same credentials as for the Azure Portal. However, as Europe and Australia regions have their own subdomains (for performance reasons), the LUIS service localized in the European region is accessible on eu.luis.ai portal. You can learn more about LUIS regions here.

When you sign in to the LUIS portal, you’ll see the service you just created in the applications directory shown in Figure 8. Note that you will need to change the URL based on your region.

Figure 8. Delivery bot LUIS service listed on applications view

The learning process in LUIS service is based on intents and utterances. Surprisingly, the service provisioned by Azure portal already has some intents – you can explore them in the Intents section after clicking on the app name (Figure 9):

Figure 9. The intents of the new LUIS service

These intents were predefined to provide some basic bot interactions and are already in use. However, it’s time to enhance the language understanding service with some new intents. New intents can be added using the Create new intent button. Add two intents with the following details:

Intent name: SendParcel
Utterances:

• Send a new package
• I want to send a new parcel
• How can I send a new parcel?
• What should I do to send a parcel?

Intent name: TrackParcel
Utterances:

• Where is my parcel?
• I want to track my parcel
• What is my parcel delivery status?
• When will my parcel be delivered?
• When will you deliver my package?

Figure 10 shows the interface used to create the intents.

Figure 10. Creating new LUIS intent with various utterances

Once the two new intents are created, you’ll see them in the Intents list as shown in Figure 11.

Figure 11. The new intents

Train and Publish LUIS

By adding some new intents, you might have noticed the Train button, visible in the header of the LUIS web application, changed the status icon color to red. This means that there were some changes since the last time it was trained. By clicking on this button, you start the training process which normally takes a few seconds. After successful training, you may notice that a status icon of Train button changed to green. Additionally, the application shows the following message for a few seconds (Figure 12):

Figure 12. Message shown after successful LUIS training

Once the training process is done, you can test it using the test tool available on the LUIS portal by clicking the Test button in the header. However, the fact that the service is trained doesn’t mean that changes you made are already available for the bot. Before making use of new intents, you need to publish a new version of LUIS. To do this you need to go to the Publish section and click the Publish to production slot button.

Introducing New Intents to the Bot

Now it’s time to start using the new LUIS intents in your bot. This means you need to make some changes in the bot messaging endpoints code. The easiest way to get access to the code is to open the Build section of the Web App Bot resource available in the bot resource group on the Azure Portal as shown in Figure 13.

Figure 13. Bot management build section

As you can see, there are two options here:

• Open online code editor
• Download zip file

If you are planning to work on some advanced features it is recommended to download a zip file. It contains a standard .NET solution, so you can develop your bot features using Visual Studio. In this case that’s not necessary, and you can simply use the online code editor. After opening it and exploring the code, you might recognize that bot functionality is based on a single dialog implemented in the Dialogs/BaseLuisDialog.cs file. Modify it with the following code:

This class is implementing by a special kind of dialog by inheriting the LuisDialog type. This type is provided by the Bot Framework API and allows implementing a bot dialog powered by LUIS. It requires obtaining some details about the LUIS application. Luckily, LUIS application details were stored in application settings by the Azure provisioning process and can be easily retrieved from Application Settings. If you still want to find the details of LUIS application on your own, you should navigate to LUIS portal, choose your application, go to Publish tab and check the Resources and Keys section for your region. All details you need are available in Endpoint value of the existing key:

• LuisAPIKey – the value (key string) of the existing key (highlighted by red color),
• LuisAppId – the identifier of the LUIS app (can be also found in Settings tab; highlighted by blue color),
• LuisAPIHostName – the domain name of the server hosting LUIS app (highlighted by green color)

Figure 14 shows these details.

Figure 14. Identifying LUIS app details on LUIS portal

The implementation of the LUIS dialog is based on public methods decorated with LuisIntent annotation. This decorator requires an intent name parameter. Once the method is properly decorated, it becomes an event handler of matching LUIS intent in scope of the current dialog.

Under the hood, the Bot Framework is forwarding each message sent by the user to the LUIS service and waits for the response. The Language Understanding service matches the input message with all intents and returns a score table as a response. Finally, the Bot Framework receives the response and triggers the method associated with the best-scoring intent.

Having modified the bot code, you need to build it. To do this, open the console section available in online editor.

You will find the button to open console here as shown in Figure 15:

Figure 15: The button to open the console

Type in build.cmd and press Enter. This command directly rebuilds a bot messaging endpoint app deployed on App Service. Figure 16 shows how the console window should look.

Figure 16. Building the app

Once the build is complete, you will see a success message at the bottom of the window similar to that shown in Figure 17:

Figure 17. The successful build

Testing a Bot

After these simple steps, the bot is ready to test. You can use a test chat component available in the Test in Web Chat section of the Web App Bot resource on the Azure Portal to start a test conversation. Figure 18 shows a successful test.

Figure 18. Test conversation with Delivery bot

Next Steps

The bot you have created is just a mockup. To make it more useful, you would need to integrate it with a real logistics system to allow tracking and sending parcels. The Bot will probably have more complex dialogs, allowing it to ask for some details such as parcel identity number or recipient details. These scenarios can be realized using another LUIS cognitive feature called entities, which allows it to recognize some important input data in the user message. The production-ready bot should also be available on a public channel, configurable in the Channels section of Bot Web App resource.

Summary

Bots turned out to be a totally new way of creating software, and there are many areas in which they can fit. Despite the fact that the Microsoft Bot Framework approach uses well-known technologies such as the web API, the analysis and development approaches are significantly different than in case of classic web applications. That is why bot development is an exciting and challenging job. Additionally, it doesn’t necessarily need expensive tools, as the development can be done completely on Azure Portal using a web browser. In the end, a successfully finished bot project brings a lot of satisfaction for its creators, project owners, and end users.