Layer-based automated testing, part 3: emulating request/response scenarios in tests

This article is about using BDD and SpecFlow for testing request/response scenarios like message boxes, modal windows, interaction with peripheral devices and similar cases. I'm using the sample from previous article but in fact this article is valid for every testing system using BDD and SpecFlow.

The problem: "when" which is not an "act"

A good practice for writing unit tests is to follow "Arrange-Act-Assert" pattern. It means that every test should consist of 3 phases: "arrange" (setting the initial conditions), "act" (executing what is actually tested) and "assert" (verifying the results). Following this pattern helps to structure your test code more clearly, and make it more atomic. That is why mocking libraries, including Moq, tend to enforce this pattern.

BDD methodology recommends following similar pattern which goes by the name "Given-When-Then", where "Given" is initial conditions, "When" is the behavior which is being tested, and "Then" is the results you expect.

However when you are trying to implement "Given-When-Then" for more complicated scenarios, you'll discover that sometimes "Given-When-Then" is not that easily converted to "Arrange-Act-Assert". For some of your "when" steps, the implementation look more like "arrange" than "act" and require different order of execution of steps. Which you can't allow because on BDD level they are clearly "When" and not "Given".

For example, let's discuss the scenario of simple confirmation messagebox:

Scenario: Calling person
    Given the following persons are available
    | Name              | HomePhone | MobilePhone |
    | Lucius Tarquinius | 3435423   | 524643      |
    When I select person with name "Lucius Tarquinius"
    And click button "Call"
    Then dialog "Are you sure you want to make a call?" appears
    When I click "OK"
    Then number "524643" is called

We are writing code-driven layer tests, so we can't test message box itself. The standard practice in this case is to separate the untestable functionality by interface, like it is recommended for example here:

http://stackoverflow.com/questions/8560865/messagebox-and-unit-testing
http://stackoverflow.com/questions/17523736/handling-messagebox-in-nunit...

OK, so we are creating this interface:

    interface IMessageBoxService
    {
        bool ShowYesNo(string message);
    }

and then we are trying to mock it:

        [When(@"I click Yes")]
        public void WhenIClickYes()
        {
            var messageBoxService = new Mock<IMessageBoxService>();
            messageBoxService.Mock.Setup(t => t.ShowYesNo(It.IsAny<string>())).Returns(true);
        ...
        }

In code IMessageBoxService is actually used like:

    if (_messageBoxService.ShowYesNo("Are you sure you want to make a call?"))
    {
        _callService.CallPerson(SelectedPerson);
    }

And here's the problem: step "And click button "Call"" is written in scenario before the step "When I click "OK"", and it already makes the portion of code with "ShowYesNo" to be executed. At this moment, mocker is not yet configured (we set it later on step "When I click "OK""), so ShowYesNo returns false instead of true. Your test fails when it should pass.

Possible solutions

Make tests more atomic

In fact, many think that scenarios like the one described here are too large. They are worse in terms of localization of a problem and reusability. To be more strict and atomic in our approach, we should write not one test, but two:

Scenario: Calling person
    Given the following persons are available
    | Name              | HomePhone | MobilePhone |
    | Lucius Tarquinius | 3435423   | 524643      |
    When I select person with name "Lucius Tarquinius"
    And click button "Call"
    Then dialog "Are you sure you want to make a call?" appears
 
Scenario: Calling person - OK
Given the following persons are available
    | Name              | HomePhone | MobilePhone |
    | Lucius Tarquinius | 3435423   | 524643      |
    And dialog "Are you sure you want to make a call?" appears for person with name "Lucius Tarquinius"
    When I click "OK"
    Then number "524643" is called

And this way we can work around this problem on "When" step, before first actual resolve has happened. But I don't think this is a good practice:

  1. Longer tests are needed anyway. You can't just avoid them. They can show problems which shorter tests just won't be able to show. See the first article in series for detailed discussion.
  2. Splitting tests in such a way is unnatural and it would be hard to understand by customers. And SpecFlow, among other benefits, gives you a clear way of documenting the scenarios. So why we should make it less clear by starting scenarios from intermediate steps like "Given dialog A appears for person with name ..."?
  3. Every time you need to think about putting your system in intermediate state, and it's a non-trivial task.

Write workarounds in test implementation

OK, but SpecFlow is quite flexible! Why we can't just remember the information on "When" steps, and execute them later, in any order we need? In practice, it means that we can delay actual execution until the end of "When" block, to be completely sure that all configuration is done:

        [AfterScenarioBlock]
        public void AfterScenarioBlock()
        {
            if (ScenarioContext.Current.CurrentScenarioBlock == ScenarioBlock.When)
            {
                // do "when" actions here
            }

But this will also cause a lot of problems:

  1. Tests will fail not at actual step, but at the end of "when" block
  2. You'll need to remember not only the conditions of steps, but also its relative order. In the end it will cause you to create a complicated event system through which different step classes are interrelated. And this system will become more and more complicated the more different kinds of steps you'll get.

Write multithread tests

It is theoretically possible to completely emulate the situation with messagebox thread-blocking behavior by starting tests from another thread. Again, I would not recommend this approach. You will get lots of issues with thread synchronization and even error output. See for example:
http://stackoverflow.com/questions/12159/how-should-i-unit-test-threaded...
http://stackoverflow.com/questions/41568/whats-the-best-way-to-unit-test...

Make modal interfaces asynchronous

So, looks like the best way to solve this problem is by making all modal interfaces asynchronous. This way we are not actually blocking the flow of business logic by calling messagebox. When modal dialog is closed, event is called and end-of-dialog handlers can start working. This way we don't need to precondition user reaction. In the implementation of step, we just raise the event of the end of messagebox, and code associated with this event starts to run. It can be implemented using traditional C# asynchronous interaction (using events) or using new async/await pattern. In this sample async/await pattern is used:

    public interface IMessageBoxService
    {
        Task<bool> ShowYesNoMessage(string text);
    }

Actual implementation of MessageBox can still use modal dialog:

    public class MessageBoxService : IMessageBoxService
    {
        public Task<bool> ShowYesNoMessage(string text)
        {
            bool ret = false;
            System.Windows.Application.Current.Dispatcher.Invoke(
                new Action(
                    () =>
                    {
                        var messageBoxResult = MessageBox.Show(Application.Current.MainWindow, text, "", MessageBoxButton.YesNo);
                        ret = messageBoxResult == MessageBoxResult.Yes;
                    }));
            return Task.Run(() => ret);
        }
    }

Now we call it like:

        private async void OnCallClicked()
        {
            var res = await _messageBoxService.ShowYesNoMessage("Are you sure you want to make a call?");
            if (res)
            {
                _callService.CallPerson(SelectedPerson);
            }
        }

"Await" splits this method in two, and the second half is called only when the task of showing messagebox is completed. We can emulate such behavior in tests using TaskCompletionSource:

    public class MessageBoxServiceStub : IMessageBoxService
    {
        public Mock<IMessageBoxService> Mock = new Mock<IMessageBoxService>();
 
        public bool ShowYesNoResult = false;
 
        private TaskCompletionSource<bool> _tskShowYesNo = new TaskCompletionSource<bool>();
 
        public void FireResultYesNo(bool value)
        {
            _tskShowYesNo.SetResult(value);
            _tskShowYesNo = new TaskCompletionSource<bool>();
        }
 
        public async Task<bool> ShowYesNoMessage(string text)
        {
            Mock.Object.ShowYesNoMessage(text);
            var ret = await _tskShowYesNo.Task;
            return ret;
        }
    }

And then, in our corresponding test step we fire the event on dialog's end:

        [When(@"I click ""OK""")]
        public void WhenIClickOK()
        {
            messageBoxServiceStub.FireResultYesNo(true);
        }

Here we are not setting up some mock: we are actually raising the event, at exactly the same moment and in exactly the same way modal dialog does. This approach is straightforward and allows "When" condition to be an "Act", so you can create long sequences of "When" conditions without breaking the "Arrange-Act-Assert" pattern. Some would argue that it's not a good idea to change your application's interfaces to solve some problems with tests, but I think this is not the case. First, in general it's not a good idea to write business logic (or view model logic) expecting synchronous response from user - which may not happen at all, or come with a significant delay. Second, with asynchronous implementation you don't really care how your dialog behaves: will it be modal or not. It can be changed without any updates to logic it calls.

Of course this approach is valid not only for dialogs, but for any scenarios of interaction with peripheral devices, external services, etc. You shouldn't underestimate the power of asynchronous calls. It can greatly help not only with scalability and responsiveness of UI, but with testability too.

AttachmentSize
sample.zip101.03 KB

Comments

Nice information

Thanks