Is Velocity a Vanity Metric?

A few weeks ago I went to a talk at Agile Yorkshire by Tom Hoyland where he discussed the how he formed an agile team. In it he claimed that Velocity is a Vanity Metric.

If you haven’t read it then I highly recommend you pick up a copy of The Lean Startup by Eric Ries, this was my main introduction to the term. In the book Eric talks about how a Vanity Metric is any figure which is skewed to inflate the success of a product. One of the example he gives is Number of Registered Users, a figure which will clearly go up and up over time. A better metric, Ries argues, would be Number of Active Users. Obviously if you’re looking for a successful product you’re more interested in how many users are currently using it than how many people completed the signup process.

He claims that by basing metrics and KPIs on these Vanity Metrics is akin to giving yourself a pat on the back for building a successful product while sticking your head in the sand about why your business was losing money (or the startup running out of runway as he describes it in his book).

photo of woman looking at the mirror
Beware the Vanity metric! Photo by bruce mars on Pexels.com

So, to return to Tom’s question – is Velocity a vanity metric?

Most Scrum Masters calculate a teams’ velocity by taking an average over the previous sprints. The number of sprints varies but six (around 12 weeks of work) is the norm. Using this figure the the team can then calculate how far a particular story is away from delivery or epic from completion.

person typing on laptop
Photo by rawpixel.com on Pexels.com

There’s no doubt that Velocity is a useful forecasting tool (although, as one of my colleagues pointed out recently that if you plan a sprint on your average velocity then you will, by definition, fail 50% of the time). However, is it deluding us into thinking we’re a successful team and detracting us from more accurate measurement?

I think this comes down to how you measure success. Most people would agree that judging a development team on how many features they can deliver is fairly narrow minded. A development team includes a Product Owner, their remit should be to develop a product not to simply crunch features. If that was their goal then they could simply create huge numbers of easy, yet useless features just to score points.

In this new age of DevOps a Development Team should use their PO’s expertise and take ownership of the success of their product. For them to be judged on how much work they produce, rather than how much success they’ve had is a limited measure. It reminds me of the from the book The Goal where they measured and optimised the workstations rather than asking whether the system was effective.

Therefore, although shocked when I heard it I agree with Tom. Velocity is a useful tool for the team to forecast. But if it’s chased as a KPI or used to measure the team then it is indeed a vanity metric and will distract the team from trying to improve their product. If we want to measure our teams’ success then we should look at the metrics of our products, not try to calculate some kind of Hours to Story Point ratio or chase an ever increasing Velocity. We should focus on Number of Active Users or Number of Requests via the API, this will measure the teams’ success, rather than it’s productivity. As a friend of mine is fond of saying, effective rather than efficient.

Growing Agile Teams at Agile Yorkshire

I attended Agile Yorkshire last week and saw two great talks by Tom Hoyland and Jon Fulton. I really enjoyed both but a few points in Tom’s talk really interested me and I’d like to take a few minutes to share them.

Tom is a Scrum Master at Sky Betting and Gaming, I’ve heard good things about the company in the past so I was interested to hear one of their success stories. It turns out that Tom was part of a team of twelve who really stripped Agile “back to basics” and conducted a series of experiments on the road to continuous delivery. Working in a regulated industry myself I was intrigued how they’d got on.

One of the first things Tom talked about was how many different people in the team came to the table with ideas of what was agile best practice. We all laughed at his “my guru is better than your guru” but it makes a lot of sense! I am heavily influenced by Jez Humble, Gene Kim, and Clarke Ching but many of my colleagues may watch talks and read blogs from very different thought leaders. Tom explained that one of the first thing they had to do in the team’s formation was break many of the concepts down to their fundamental concepts and understand what worked for them.

Something else Tom discussed was how the team consolidated their own backlog. This was not controversial, how else would you prioritise the work against it? It was only when he gave examples of some of the different backlogs they’re identified that I became intrigued. Risk Logs, Retro Actions, and Design Session – all of these moved onto the board and each became visible and prioritised.

It’s dangerous out there – take your buddy! I’ve heard many of the advocates or pair programming before but the idea of your buddy following you into meetings, design sessions, and CABs!? Tom explained that if your buddy went with you to these sessions not only would they learn how to design, and walk work through the CAB but they’d also know the current state of everything you were working on. If you were sick or the proverbial bus came along the team wouldn’t need to bother you because everyone would know what was going on.

group hand fist bump
Photo by rawpixel.com on Pexels.com

 

There were many other good ideas (and I intend to borrow quite a few of them myself) but the final one I’m going to mention was the idea that Velocity is in fact a vanity metric (read The Lean Startup if you have no idea what I’m talking about). Velocity is just a number, like Number of Users or Number of Page Views). What we want are actionable metrics, like team predictability and accuracy of forecasts. As a Development Manager I frequently use the team’s average velocity to forecast delivery dates, Tom recommended that there are better measures out there such as a temperature check of the team’s current mood (which would often dip before any reduction in velocity). It’s an interesting idea, and one I intend to think more about over the next few weeks!

A big thank you to Royd and the guys who put Agile Yorkshire together each month. An equally big thank you to Tom and Jon for their great talks!

The Lock Complex

I have recently coined the term Lock Concept as a symptom of what many people call Fake Agile. Allow me to explain…

Waterfall development is often described with the Design, Development, and Testing phase structure. Many teams adopting Scrum tend to fall into one of two mistakes.

waterfalls
Photo by Trace Hudson on Pexels.com

The first mistake is to split up these into sprints. So Sprint 1 is for design. Sprints 2, 3, and 4 are for development and testing and bug fixing will go into Sprints 5 and 6. This isn’t Scrum. Clarke Ching uses a phrase I like in his book Rolling Rocks Downhill, he talks about GETS software. That’s Good Enough To Ship, at the end of each sprint the software must be production ready. By falling into the sprint phase trap you’re lowering quality between releases and not realising the value of scrum.

The second mistake teams make is to try and run each Sprint as a mini waterfall. This is what I now describe as The Lock Complex. Teams falling into this trap will design in the first few days, develop for a few more, and then test their work towards the end. Yes, the software is GETS at the end… but doesn’t this look like a waterfall on a smaller scale?

neptune27s_staircase_2017_head-on
Canal Locks of Neptune’s Staircase by aeroid CC BY-SA 3.0

The main symptom with this approach is people twiddling their thumbs (testers at the start of the sprint and developers at the end). While wasted time is frustrating, the real problem is the lack of shared knowledge and by unlocking that you can quickly raise your game towards Continuous Delivery.

The way to solve this becomes quite apparent if you look at the DevOps utopia we’re all told about. In a world of Continuous Delivery and automated approvals we create automated acceptance tests to ensure that our code functions as expected. If the feature doesn’t meet these automated tests then it will not be merged in, or if it has been then the deployment pipeline will stop.

In this world, not only are we deploying faster and achieving single piece flow but we’re breaking that Lock Complex. People are busy all the time and pair and mob programming becomes the norm. Instead of having a testing phase where it’s our QA’s engineers’ time to shine we have continuous collaboration and our quality specialists advising on the best tests and mechanisms to be implemented. Testers no longer run manual tests, we get computers to do that. Testers work to ensure that the automated tests give us a coherent test strategy.

If we can help our teams to break the Lock Complex and stop working in mini-waterfall sprints then we’ll see the benefits as people collaborate more and achieve better velocities and higher qualities as a result.

Agile Development with Lego Star Wars

Recently I was lucky enough to attend a workshop run by IndigoBlue as part of the Leeds Digital Festival.

Our objective was to build a Death Star for the Emperor. The Dark Lord of the Sith had kindly refined the construction of his battle station into user stories and the two different teams had to build various sections ready to be deployed into production.

It was only at the end of the first sprint, after my team had constructed a very impressive throne room and firing dish that we realised that none of us had asked how we were supposed to deploy into production.

The Emperor it turns out had an operations team and was very particular about production access. We couldn’t just drop sections of the space station anywhere, there were rules, and if we didn’t talk to the operations guys, the other team, and on occasion the emperor himself there was little chance we’d succeed. I don’t want to share all the rules as IndigoBlue had clearly put a huge amount of time and effort into developing them, if you want to know more I’d highly recommend you to get in touch!

After three sprints I’m glad to say we started to get the hang of things and our Death Star reaches its MVP.

The Emperor was delighted that his Death Star had a trench surrounding it…

A board room for holding important meetings.

And a shuttle bay.

Not to mention all the other bits I didn’t get a great photo of!

Overall it was a brilliant evening, a great game to remind ourselves of some the basic Agile and DevOps concepts, and I had a lot of fun. If you see another of these events advertised I’d highly recommend it!

The Four Types of Work in an Agile World

If you have read The Phoenix Project (which I highly recommend you do) you’ll be more than familiar with Eric who introduces us to the four types of work.

  • Planned Work – these are typically business projects or new features
  • Internal Projects – server migrations, software updates and so on
  • Changes – usually driven by feedback on already completed work
  • Unplanned Work – support escalations and emergency outages

In the book Eric talks about how Unplanned Work is unpredictable and destructive as it can often push out planned work. I can’t even begin to count how many times have I’ve planned in projects and tasks only for a crisis to arrive and my sprint to fail!

Managing a Product Backlog is an excellent way to handle three of the four types of work. After all, Agile Principles encourage us to collaborate with stakeholders, listen to feedback, and embrace change. The value of internal projects can be evaluated against other projects and then planned correspondingly and what is the backlog at all if not a prioritised list of Planned Work?

It’s the infamous Unplanned Work which continues to disrupt our Sprints, break our commitments and lower our productivity. Over a few years I’ve tried a number of different approaches to handling the inevitable Unplanned Work.

I’ve tried introducing slack into everyone’s sprint, only planning fifteen instead of twenty Story Points. I’ve tried allocating Support Tickets as they come in, assuming that the demand will cause our average sprint velocity to drop and the balance between Planned and Unplanned Work to find a happy equilibrium. It never does!

The only strategy I’ve found to work is to completely isolate the Unplanned Work from the other three types. Create a specific team (in our company we call this the SWAT team) who will react to all incoming unplanned work and “pitch in” with other work when there isn’t enough to do (Parkinson’s Law pretty much guarantee’s that this never happens).

The SWAT team handle the urgent, unplanned, and reactive work (often using a strategy like Kanban or Scrumban) while the rest of your team focus on delivering maximum value to the business (Sprints, commitments, and all the other wonderful concepts we hear about on Agile courses). That’s not to say they don’t fix bugs or take on internal projects, it’s that they work in a structured, planned environment in an effort to build solid releases and reduce the amount of Unplanned Work long term.

Speaking of Parkinson’s Law if you do decide to create this buffer between Planned and Unplanned Work (and I strongly suggest you do) you need to consider what happens when the work expands to fill the allocated time. What happens when the urgent support demands of the business exceed the capacity of your SWAT team? Hire extra people? That would be nice! But I can tell you what you must not do – DO NOT pass the extra work over into your Scrum team. I’ve made that mistake before and I can assure you that it’s a very slippery slope which is nearly impossible to come back from. Once you start roping in extra developers you undo your good work and Sprints start to fail once again. I mention it between I’ve seen it happen!

I hope I’ve given you some thoughts, I hope that my views on throttling the amount of damage your Unplanned Work can do help you safe your own commitments but I’d be delighted to hear your views – what do you to do safeguard your team’s productivity while meeting the urgent needs of the business? Post a comment below and let me know!

April Agile Yorkshire

I managed to get a seat at Agile Yorkshire this month, I’ve missed a few of these recently partly due to other commitments but also the sheer popularity of these events.

Royd (from NewRedo) organises and coordinates these evenings, this week he’d arranged for Chris Cheadle and Sean Craig from NHS Digital and John Le Drew who runs The Agile Path to speak to us.

Chris and Sean went first, they spoke to us about an event they’d run a little before Christmas. They’d called it Firebreak, during a two week period almost the entire organisation downed tools and worked on “whatever they wanted”.

They’d started with almost a kickstarter approach, people posted ideas on postit notes and their colleagues pledged their time – once a project was fully resourced it was banked and it would go ahead.

I love the idea of this sort of thing, really opening the doors to let teams work on what they want – what they feel wild make a difference. Anything from process improvement to a proof of concept or a piece of server maintenance. It certainly seemed to be a positive experience for the NHS team, some of the projects saved thousands of pounds on licence fees!

The second talk of the night was about Safety and how important it is to effective teams.  John explained that he considered safety to be “free to make suggestions, give feedback, and make mistakes without fear of punishment or humiliation.”

For me this feeds into the fundamental requirement of trust which Patrick Lenconi described in his book Five Dysfunctions of a Team. He postulated that without trust (or in John’s words safety) teams would not challenge each other and discuss ideas.

John had a few examples of this, my favourite was a role play exercise where three characters were so determined to avoid taking the blame for pressing The Big Red button which would overload the nuclear reactor they refused to cooperate to press the three buttons which would save them. Contrived? Perhaps… but it makes the point that if you are scared to look foolish then you’ll naturally be less confident to make suggestions.

Something which did niggle me was the idea of accountability, as managers we need to hold our directs responsible for their performance but this is challenging without undermining that feeling of security. In Lenconi’s book he encourages the team to hold each other accountable, finding the balance between a blame culture and safe, self motivating team is a difficult balance to find!

I’d certainly recommend listening to John if he’s speaking in your area. At the very least I’d say every manager should hear his views about engagement and workplace stress! I for one will be listening to his podcast.

It was a great night, as I’ve said before I’d always suggest Agile Yorkshire if you’re a Leeds based  geek!

Defining Value

I was recently at a talk where one of the key topics was “Defining Value”. I actually felt rather prepared for it – after recently reading The Goal I was fairly confident that the goal of any business was to make money. So “Value” would be anything which either earns us revenue or aids in our ability to make money.

It sounds boring, but it’s true.

My point actually kicked off a lot of discussion, in a room of experienced developers the proposal that unless we’re earning money we’re not adding value was rather controversial “what about paying technical debt!?” was one of the initial reactions (and one we came back to very quickly).

Delivering maximum value is of course what Scrum is all about, the idea that the team is constantly working on the most valuable piece of work is key to the sprint framework. So to be challenged to define value was an interesting idea, especially when Pete (our facilitator) threw out the following quote things became interesting:

“As a general rule of thumb, when benefits are not quantified at all, assume there aren’t any” – Tom DeMarco and Timothy Lister

With paid for work the value is often clear, the work is worth whatever the customer is willing to pay for it. Again, work to generate sales the Product Owner can list the contracts which will be opened up by a particular feature if it was to be implemented.

Support work becomes much harder. We all know that it’s easier to retain an existing customer than to earn a new one and anyone in sales will tell you the value of upsell and good customer service. So how do we justify spending hard pressed development resource fixing bugs, answering questions and supporting our existing customers when it does not earn us any revenue?

We all agreed that the only way to quantify the value of technical debt was to look at how much time and effort was being spent in supporting it. If an area of your software is full of bugs and you spend four days of each month assisting customers using it then over the course of a year it’s worth at least forty eight days to improve it!

Support work is valuable because by not doing it you’re putting your customer’s loyalty to you at risk. Reducing technical debt is valuable because it makes support work easier.

The next time you’re working in an unloved area of code, don’t moan about it – measure it. Present the business case to your Product Owner and help them quantify the value of refactoring and improving the code. It will make it a lot easier to justify doing the work!