Tag: SpecFlow

When and Where to Automate Testing

~ Adam ~ Leave a comment

A year ago I undertook an interesting piece of R&D to write Selenium tests for our main UI. I watched the pluralsight course, learned the difference between WebDriver and the IDE, and started building my Page Object Model. My simple test took the best part of three weeks to build and executed in around half the time a decent QA would take if you gave them a double shot of espresso.

I patted myself on the back, demonstrated the work to our Product Owner, and then advised that we should shelve the work because I wasn’t confident to hand over the mind boggling complexities of waits, framesets, and inherited view models into general practice.

Over time I lost confidence in the project. If it had taken me days to generate even the simplest of tests how would a junior developer fair when asked to automate complex financial scenarios. Quietly I put the idea to the back of my mind and concentrated on other more pressing matters.

That was until last week. We’ve been doing some work on our BACS integration and as part of the regression test I’d enlisted the help of one of our QAs to mock each response code and import it into the system. The process was tedious, repetitive, and I hated myself when I had to tell him he’d missed a vital check off each scenario.

As I was speaking to him cogs in my head began to turn. I’d gone off the idea of large scale test automation because of the complexity of our UI but the BACS processing system doesn’t have a UI. I could knock something together in a couple of hours which would create customers, mock BACS files, and schedule our JobServer. Even more powerful, if I used a technology like SpecFlow our QA could write the tests he wanted, I could automate them, and we’d be able to iterate over every scenario within a couple of minutes. Even more exciting was the idea we could send the feature files off to our Product Owner and banking partner and ask them to verify the behaviour was correct.

Later that week, after we’d proven our project to be a success and found a handful of low priority bugs to be corrected I started to wonder why this automation project had delivered such value where the previous UI automation had failed. I decided this was because:

  • The BACS process was an automated mechanism already so the test automation steps were simpler
  • The UI had been designed for human use, it wasn’t a chore to run through but it was complex for Selenium to navigate the controls
  • Mocking and importing BACS files was repetitive, slow, and tedious

The project turned out to be a huge success, we’re already planning on how we can expand the solution to cover other payment integrations such as SEPA.

The next time you’re considering whether or not you should automate an area of testing consider the nature of the tests. Do they use complex UIs? Do you have to repeat very similar tests over and over again? Do they currently take up a large portion of your testing cycles?

Try to automate the hidden process which run over and over and you wished you could test every time if you had the time!

Multiple Binding Attributes in SpecFlow

~ Adam ~ Leave a comment

I recently discovered something rather nice in SpecFlow, I was implementing a scenario like this

Scenario: Save a user
Given I have a user with the name Joe Bloggs
When I save them
Then the user should have a FirstName Joe
And the user should have a LastName of Bloggs

I wanted to provide the flexibility in the assertions so our QA could decide how he wanted to phrase the text in the scenario. Logically however we’d want the same binding for each variation.

Here’s what I came up with:

[Then(@"the user should have a (.*) (.*)")]
[Then(@"the user should have a (.*) of (.*)")]
public void ThenTheUserShouldHaveA(string field, string value)
{
  var user = GetUser();
  Assert.AreEqual(user.Properties[field], value);
}

However this didn’t work, I kept getting a field of “FirstName of”. I discovered however that you can reverse the binding attributes to give priority.

Updating the attributes to

[Then(@"the user should have a (.*) of (.*)")]
[Then(@"the user should have a (.*) (.*)")]

This change gave the of binding precedence and ensured that both scenario steps worked correctly.

SpecFlow what is it and why should I care?

~ Adam ~ 1 Comment

I was introduced to SpecFlow a year or so ago by a friend of mine during a presentation at Agile Yorkshire. He described it as a tool for BDD and I was immediately struck by the revolutionary concept of human readable tests. It’s taken me a little time, some projects and many follow up questions but I feel I’m really beginning to grasp the power of this easy to use tool.

So what is it? SpecFlow is a framework which translates text into a series of steps which are compiled as a test.

This makes more sense with an example. Let’s say I’m the developing the software for a cash machine.


Given I have £150 in the bank
When I withdraw £100
Then I should have £50 remaining in my account

SpecFlow will then generate three steps for you to implement.

[Binding]
public class Bindings
{
  [Given(@"I have £(.*) in the bank")]
  public void GivenIHaveInTheBank(int p0)
  {
    ScenarioContext.Current.Pending();
  }

  [When(@"I withdraw £(.*)")]
  public void WhenIWithdraw(int p0)
  {
    ScenarioContext.Current.Pending();
  }

  [Then(@"I should have £(.*) remaining in my account")]

  public void ThenIShouldHaveRemainingInMyAccount(int p0)
  {
    ScenarioContext.Current.Pending();
  }
}

I’ve filled these in quickly to give you an idea of how the process is intended to work.

[Binding]
public class Bindings
{
  private readonly CashMachine _cashMachine;

  public Bindings(CashMachine cashMachine)
  {
    _cashMachine = cashMachine;
  }

  [Given(@"I have £(.*) in the bank")]
  public void GivenIHaveInTheBank(decimal initialBalance)
  {
    _cashMachine.Balance = initialBalance;
  }

  [When(@"I withdraw £(.*)")]
  public void WhenIWithdraw(decimal withdrawlAmount)
  {

    _cashMachine.Withdraw(withdrawlAmount);

  }

  [Then(@"I should have £(.*) remaining in my account")]
  public void ThenIShouldHaveRemainingInMyAccount(decimal expectedRemainingBalance)
  {
    Assert.AreEqual(expectedRemainingBalance, _cashMachine.Balance);
  }
}

Behind the scenes SpecFlow has built a test using your test framework of choice (MSTest, NUnit, XUnit or a variety of others). These tests can be run within Visual Studio as any other unit test. Here’s my scenario running in Resharper:

 

Resharper Executing Test

 

I think most people would be impressed by this. SpecFlow has converted human readable text and enabled us to write a simple unit test for it. We can easily add a second to make sure that our new state of the art cash machine can also dispense pennies.


Given I have £100.86 in the bank
When I withdraw £1.99
Then I should have £98.87 remaining in my account

We can add additional complexity such as overdrafts


Given I have £100 in the bank
And I have an overdraft of £50
When I withdraw £125
Then I should have £-25 remaining in my account


Or functionality to check that I cannot go overdrawn



Given I have £100 in the bank
And I have an overdraft of £0
When I withdraw £125
Then I should be prevented from withdrawing money
And I should have £100 remaining in my account




I’m sure you can see the value of having simple, understandable definitions of what a particular area of code should do instead of the complicated, developer-only Unit Tests which litter many of today’s solutions. If you wish these executable tests can form the basis of your specs, documentation and even training. Your Product Owner can get involved and validate the tests are correct, if they’re particularly tech-savvy they may even write a few themselves!


What I particularly love about SpecFlow however is the way it empowers today’s QAs. From the moment the first bindings have been created your QA team can be let loose to create new tests, verify edge cases and prove alternate scenarios until they are happy that the feature works as designed. Tests can now be created alongside the features being developed, Acceptance Tests can be created directly from the specification which the QA team can use to validate each feature and scenario as it is being developed. These same tests then form the basis of your regression suite for years to come.


Should any of these tests fail the QA team can provide the developer with not only the human readable replication steps but with with an executable test which can be debugged on the dev’s own PC.


I find this very exciting, we’ve started using SpecFlow for a number of projects and I have every intention of using it for more. If you’re interested in finding out more visit the SpecFlow website or read their Quick Start Guide.