Redefining User Acceptance Testing

User Acceptance Testing or “UAT” is a critical part of any project. It’s where the customer takes your software into their environment, tests it and reports any issues which must be resolved before they take it into live. Development teams then resolve these issues, send the application for retest and (hopefully) get a signed off version.

I have a deep seated dislike of this concept.

I should clarify that I have no problem with customers doing their own testing – in fact I encourage it, we’re software professionals and don’t work in our application’s domain. High quality customer collaboration is essential if we are to produce a good product. No, what I have an issue with is the concept of a development, testing, and bug fix phase.

If Agile and Scrum techniques have taught us one thing it’s that we need to move away from “development phases” and towards continuous delivery with ever increasing value.

Leaving bug detection and resolution until the end of project is bad!

What then is the role of UAT in modern development methodologies? How do we achieve a signed off version without digging ourselves into these waterfall defined project stages we were supposed to have abandoned years ago?

By making UAT a continuous process alongside the development.

In my view as soon as a project begins, while the stories are still being written and the mock ups created the latest and greatest version of your application should be installed into the customer’s UAT environment. Then, while you’re working away creating the new features the stakeholders in your client’s business are testing the suitably of what you’ve already developed against their needs.

As you continue, Sprint on Sprint, Story on Story you can incorporate your client’s feedback (both bugs and suggestions) to deliver the best possible product at the end of the project. It’s only by presenting your build to the customer early that you can get this feedback. Forcing your clients to wait until your project is feature complete and signed off internally will create these heavy bug fixing phases we’re all working so hard to avoid.

Creating a Valuable Signoff Process

We all have them, a series of tests which need to be run before a build is considered good enough to be released to a customer. You may have different scripts for different stages of release (Alpha/UAT/Live), you may have different processes depending on the severity of any bugs found. Whatever your process I’m sure we can all agree that your signoff process is one of the most important pieces of your development puzzle and one of the most vital to get right in order to avoid issues further down the line.

But what makes a signoff process good? These are the things I believe a good signoff process needs to achieve:

  • Detect all unknown bugs in all critical features
  • Quick and easy to complete

Clearly these are at odds, if you want to find every bug then you’ll need to invest significant time! Let’s look at why each of these are so important before we try to find a solution.

The first one is fairly obvious, the most critical parts of your system are the ones which will generate the most urgent Support Tickets. In order to minimise those stressful late nights we need to validate that those areas of the system are as robust as possible.

But why the speed? Why not simply run through every test and permutation before each release (assuming you don’t mind driving your QA team mad)? In this world of Agile Development and quick turnarounds it’s becoming more important than ever to test and release quickly. After all, I mentioned in my previous post about Sprint Planning I suggested that you should aim to both develop and test your new features in each Sprint. In order to maximise your development potential you need to make your signoff process as efficient as possible.

So how do we do this? The key is to target your testing effectively. You need to work with your Product Owner to identify the Critical Functionality which must not fail, these form the basis of your signoff scripts. Other areas of the system can be tested on a rotational basis or when changes are made.

This prioritisation of the most critical functionality guarantees that the vital happy paths are always tested. This leaves more time for the QA team to expand their efforts into other areas while the developers are coding new features. By targeting your signoff scripts you can guarantee a high quality build without the lengthy delays which come from a bloated signoff process.