Playing with Microsoft Bot Framework

Part of my job description is our CI/CD and it kind of implies that I’m interested in keeping the build green. It doesn’t mean that I immediately jump in whenever some unit test fails, but I’m definitely keeping an eye on unreliable ones.

Whenever master branch stays red long enough, this is what starts to happen to each failed test in it:

  1. Look for test failures history in Google BigQuery (select Name, Result, count(*)...).
  2. If test behaves like a random results generator, create a case for that.
  3. Skip the test in master branch and put the case number as a reason.
  4. Find out who created the test (git blame) and assign it back to the author.

Pretty simple. And boring. I can automate that, but it’s not always clear who is the author of the test. After all, people resign, update each other’s tests, refactor and destroy git history on special occasions. I was thinking about doing something with machine learning to solve that, but it feels like an overkill. Creating a bot, on the other hand, who would ask me to double check when it’s uncertain, sounds more interesting and actually doable. Even if I’m never going to finish it.

However, I’ve never wrote any bots before, so for starters I’d like to check what it actually feels like.

Choosing a bot framework

Easy googling reveals that Microsoft Bot Framework seems to be default choice for bot writing. There’s also BotKit, but having Microsoft behind the tool might mean that there will be more examples for it. Plus, Bot Framework supports both C# and JavaScript – the two languages I’m the most comfortable with, so the choice was easy.

On the surface, developing with Bot Framework seems simple. For instance, in its object model there’s a Bot object, which has Conversations with Users through different Channels. Single Conversation consists of one or more Dialogs, and Channels are basically the tools User uses to interact with the bot: chats, Facebook, Skype, Slack, etc. Quite logical. There’re also some other object types and concepts behind it, but you get the idea.

However, despite such object model does look logical, writing with it is not quite intuitive. Few times I took existing bot example, made a tweak or two, and then dive into documentation trying to understand why exactly it doesn’t work. And I can tell you, the level of details I had to get into was incomparable to triviality of the change I made. But maybe it’s just me.

Writing “Hello World” bot

“Hello World” equivalent in the bot world is the bot echoing every message back to its sender. For this one I used Nodejs, but C# with .NET Framework or .NET Core is also a valid option.

This is how it looks. First, initialize the project and install botbuilder  package:

And then, the bot itself:

Because it’s going to talk to user via console, we had to create a connector for that (line 2). Then, we create a bot (4), providing it with the connector (5) to a channel and default handler (6) for any conversation that user initiates. In this one our bot simply echoes user messages back, prepending them with Hello world and words.

Kinds of bot conversations

Even though in the previous example a word “Conversation” has never came up, that line-long arrow function was actually the conversation. In fact, MS Bot Framework has two types of them: waterfall and dialog based.


Waterfall conversation is simply a linear sequence of functions, which guide user through set of questions and answers. For instance, assume we’d want to write a pacifist bot – a guardian of pacifist chat, who asks a set (of one) of questions anyone who wants to join. Without changing previous hello-world example that much, here’s how we could do that.

The whole conversation can be put into two steps: a question and a reaction on its answer. The only question the bot is going to ask is “Are you against the war?”. Because that’s a question with limited number of answers, I used builder.Prompts.choice function to make it as hard as possible to answer anything other than yes or no.

Here’s how conversation with the bot could look like.

What’s cool, depending on Channel and Connector type, Prompts.choice will try to use UI and tools that given Channel provides. For instance, ChatConnector with Bot Emulator attached could’ve rendered that as two buttons, eliminating the need to type anything at all.

Dialog-based conversations

Creating a conversation based on Dialog objects is more powerful, as with this approach bot starts behaving like a state machine. As it moves between its dialogs it any direction, for end user it might feel like a non-linear conversation.

Dialog itself is an object with miniature waterfall conversation in it. Dialogs can start each other, or stay idle, waiting to particular keywords in ongoing conversation and kicking in when they notice something relevant.

Let’s get back to the original reason why I started to think about writing a bot. If my bot is going to help me with monitoring unreliable tests, these are the kinds of dialogs I might need to have with it:

  1. See the list of commands that bot understands.
  2. Tell it to start monitoring unreliable tests.
  3. Configure a threshold – how often a test is allowed to fail before it’s considered unreliable.
  4. Tell it to stop monitoring tests.
  5. Check current status.

What’s interesting, in some scenarios dialog #2 will automatically lead to dialog #3. After all, it’s hard to start monitoring for unreliable tests without being configured first. Another interesting fact is that this bot implies some sort of a state. Firstly, a threshold (dialog 3), which can be associated with current user, and secondly, whether or not monitoring is running, which we can associate with the conversation itself.

This is how we can do that.

Default conversation handler

Let’s add default conversation handler to kick in when I type messages that bot does not understand. Some sort of a help message in response would be enough.

There’s nothing fancy here except for setting a storage where we’ll keep the state. It’s volatile and will be destroyed when bot exits, but for now it’s good enough.

Handling ‘status’ command

What should we do if user types status as we suggested? Start a dialog!

As you can see, this dialog looks like a miniature version of waterfall conversation, where the only interesting things are reading the values from userData and conversationData and specifying when dialog should start.


‘Stop’ dialog

This one is pretty straightforward. If conversationData‘s flag is not set – do nothing. Otherwise – turn it off. In real implementation it also would stop some background processes.

‘Configure’ dialog

This one is a little bit more interesting. I want to ask user to enter a number between 1 and 100, representing percentage of false positives, after which a test is considered unreliable. I don’t want to do reading/parsing myself, so let’s allow builder.Prompts.number to handle that.

This dialog also has a trigger keyword, so I can start it any time I want. But it’s more interesting to allow another dialog to do that.

‘Watch’ dialog

Here we instruct the bot to end conversation if isStarted flag is already set, or begin a new dialog if configuration is needed. Then, when child dialog finishes, WatchDialog dialog regains the control and continues to the second step.

Having a conversation

Pretty cool.

As a bonus, here’s one more trick. Without any single change in conversation flow, I can replace ConsoleConnector with ChatConnector, add small web server to handle HTTP calls and use Bot Framework Emulator to have a nice chat with my bot.

Bot Emulator


Apparently, writing bots is fun. The biggest thing I haven’t touched yet is connecting a bot to Microsoft Cognitive Services, which would allow me to do some pretty cool stuff.

For instance, you noticed that dialog triggers are pretty rigid, right? If I type Start instead of Watch, corresponding dialog won’t kick in, even though it’s obvious in a given context what the intent was. As a solution, I could’ve connected user intents recognizing module to LUIS – MS service for understanding the language. I saw few examples of how it’s used, and that’s really impressive. For instance, LUIS was able to guess that ‘Hello’, ‘Howdy’, ‘Hey’ and ‘Wazzup!’ probably have the same intent. Or ’10s’ and ‘ten seconds’ are equivalents. Or ‘yesterday’ is a date. And so forth. Wow.

Anyway, I’m still thinking whether or not I should write that bot for unreliable tests, but at this point the bot itself is not a challenge anymore. The only difficult part would be writing connectors for bug tracker, git client and unit test results storage, so in addition to talking bot could actually do something.

Leave a Reply

Your email address will not be published. Required fields are marked *