Measuring your Support Queue

Last week I wrote about how we use a dedicated “SWAT Team” to handle the inevitable unplanned work which threatens to creep in and disrupt our sprints. This week I want to talk about how we measure our SWAT Team, what KPIs we use and how we know whether we are doing a good job.

There are hundreds of posts out there which discuss the merits of KPIs for developers and how the wrong ones encourage behaviour such as cherry picking or incorrect prioritisation. I agree with them entirely, that’s why it’s important that the measurements you do take should reflect the customer experience, rather than the team’s performance.

For example, counting the number of tickets solved would be a poor metric because only quick wins would be looked at. Equally timing how long was spent on each ticket would also encourage people to rush in order to get better stats. However, recording how long a customer waited (from raising a ticket to resolution) measures the customer experience – which is, after all what we should be more concerned about!

In my team we use two main statistics to measure how well we’re performing. The first is one I’ve mentioned before. Average Support Ticket Age is crucial for us because it places a numeric value on how long customers are currently waiting for our help.

The second metric we use is Cycle Time, this is the time between a Ticket being passed to the team and it being resolved. In other words, how long is it taking to solve tickets passed to us This is taken as an average of tickets closed over the last six weeks.

The reason we like these two values is not only that they give a customer’s perspective on our work, but because they balance each other nicely. If we measured Cycle Time alone then we’d get fantastic results by simply solving tickets as they come in but longer running and challenging tickets would be left behind. Equally by focusing only on the older tickets it’s likely we’re missing urgent tickets and quick wins which could be resolved quickly. It’s only by continuously improving both values do we provide a good service.

You’ll notice that we don’t worry too much about the volumes of tickets. I find this doesn’t actually matter, the number of tickets being raised varies from month to month, from customer to customer, and will change as customers leave and (much more ideally) join us. If the team is becoming overloaded with tickets then this will be highlighted in the metrics we already have (as we won’t solve the tickets as quickly). A measure of tasks in the queue is less import than whether the team is keeping up with the required workload.

A final point to make us where we start and finish timing. If you’ve ever read The Goal you’ll know that one of Eli Goldratt’s key points is whether you are measuring the right thing (he points out that efficiency increases do not necessarily produce an increase in profit). The decision you have to make with your KPIs (particularly when a ticket was opened) is whether to start your timer when the customer raises the ticket, or when it’s passed to your team. There is no perfect answer here, if you want to understand the entire customer journey then you need to look at Customer to Customer timings, however – if your team only plays a small part in that journey (as in our development team’s case as we have several support teams before us in the process) then your metrics will be less valuable if they include areas outside your control. Consider what you’re measuring, but never forget that you may only play a small part of the customer’s overall journey.

Hopefully I’ve given you a few ideas? Do you agree with my views? How do you measure the performance of your support teams?

The Goal Book Review

I recently finished reading The Goal by Eliyahu M. Goldratt. This is the book which is referenced in The Phoenix Project and Rolling Rocks Downhill so I was determined to get my hands on a copy and read it!

Alex Rogo is a Plant Manager, work isn’t doing so well… in fact he’s been given an ultimatum and is about to get shut down. In order to save his job he enlists the help of an old teacher who persuades him to question the standard measures and processes he’s been using his whole career.

What follows is a very interesting and in depth look at the Theory of Constraints (although I don’t actually believe the term is used in the book) as the team try to identify the bottlenecks in their plant and work on ways to exploit them.

Like the best books Eli doesn’t try to throw too many concepts at you, his message is clear – find the bottleneck and organise work around it. There are some great analogies (such as Herpie leading his fellow boy scouts through the woods) and you genuinely feel like you’re working these problems out alongside Rogo.

The last few chapters felt a little disorganised to me but they they carried an important message – build up the process of how to examine your system, don’t just rely on a defined step by step guide. Continuously review, understand, and adapt.

Would I recommend The Goal to other IT Managers? Absolutely! You’ll gain a great understanding of how to observe and measure your team’s throughput.However, I’d say it’s absolutely essential for anyone in software development to read The Phoenix Project first so you understand why we’re looking at manufacturing plants to help us run IT departments!

Measuring Sprint Velocity is Useless Unless You Ship!

I’ve recently been reading The Goal by Eli Goldratt, in it Alex Rogo (the Plant Manager) boasts to an old teacher of his that the new robots they’ve installed have greatly increased their efficiency. The Yodaesque Jonah then promptly proves his data worthless and sets Alex on the path to enlightenment.

What struck me however was how clearly this mirrors the software industry. The Goal has been the go-to book for managers for years but only with the relatively recent release of books like The Phoenix Projext have the applications to software industries been recognised.

It’s a well established idea to model a software development team like a factory. Time and money goes in, features and fixes come out. In the story Alex was delighted that his robots had given him an increased efficiency for making a particular part of the process, it was only when Jonah pointed out that the robots did not result in any increase in sales that he saw the problem in his logic.

So, let’s imagine that our software business is Alex’s factory. We’ve brought in a robot to do the work of the Development Team and Sprint Velocity has gone up from 100 Story Points to 300. As the Development Manager you get to pat yourself on the back, celebrate your success, and go home at the end of the day.

But what happens to your release at the end of your Sprint? Is your product stacked up on your factory floor awaiting the next machine (perhaps your deployment or infrastructure team)? Or have you sold it and added value to your business?

This may seem like false measurement, your numbers are telling you that you’re delivering but the reality is very different. The truth however is even worse, unshipped features are the software equivalent of Work in Progress, they’re the half finished products sat on your factory floor taking up time and space. Until your business can deliver them they’ll continue to come back and haunt you, injecting unplanned work into your Sprints and sucking time out of your Development Team.

So, If your Sprint Velocity measurements only take into account the work pushed through your Development Team then you’re not measuring the value you’re adding to the business, you’re making the same mistake as Rogo and only considering one part of the system. You may be getting great results, but are you helping achieve the goal!?