Is Velocity a Vanity Metric?

July 8, 2019July 5, 2019 ~ Adam ~ Leave a comment

A few weeks ago I went to a talk at Agile Yorkshire by Tom Hoyland where he discussed the how he formed an agile team. In it he claimed that Velocity is a Vanity Metric.

If you haven’t read it then I highly recommend you pick up a copy of The Lean Startup by Eric Ries, this was my main introduction to the term. In the book Eric talks about how a Vanity Metric is any figure which is skewed to inflate the success of a product. One of the example he gives is Number of Registered Users, a figure which will clearly go up and up over time. A better metric, Ries argues, would be Number of Active Users. Obviously if you’re looking for a successful product you’re more interested in how many users are currently using it than how many people completed the signup process.

He claims that by basing metrics and KPIs on these Vanity Metrics is akin to giving yourself a pat on the back for building a successful product while sticking your head in the sand about why your business was losing money (or the startup running out of runway as he describes it in his book).

photo of woman looking at the mirror — Beware the Vanity metric! Photo by bruce mars on Pexels.com

So, to return to Tom’s question – is Velocity a vanity metric?

Most Scrum Masters calculate a teams’ velocity by taking an average over the previous sprints. The number of sprints varies but six (around 12 weeks of work) is the norm. Using this figure the the team can then calculate how far a particular story is away from delivery or epic from completion.

person typing on laptop — Photo by rawpixel.com on Pexels.com

There’s no doubt that Velocity is a useful forecasting tool (although, as one of my colleagues pointed out recently that if you plan a sprint on your average velocity then you will, by definition, fail 50% of the time). However, is it deluding us into thinking we’re a successful team and detracting us from more accurate measurement?

I think this comes down to how you measure success. Most people would agree that judging a development team on how many features they can deliver is fairly narrow minded. A development team includes a Product Owner, their remit should be to develop a product not to simply crunch features. If that was their goal then they could simply create huge numbers of easy, yet useless features just to score points.

In this new age of DevOps a Development Team should use their PO’s expertise and take ownership of the success of their product. For them to be judged on how much work they produce, rather than how much success they’ve had is a limited measure. It reminds me of the from the book The Goal where they measured and optimised the workstations rather than asking whether the system was effective.

Therefore, although shocked when I heard it I agree with Tom. Velocity is a useful tool for the team to forecast. But if it’s chased as a KPI or used to measure the team then it is indeed a vanity metric and will distract the team from trying to improve their product. If we want to measure our teams’ success then we should look at the metrics of our products, not try to calculate some kind of Hours to Story Point ratio or chase an ever increasing Velocity. We should focus on Number of Active Users or Number of Requests via the API, this will measure the teams’ success, rather than it’s productivity. As a friend of mine is fond of saying, effective rather than efficient.

Growing Agile Teams at Agile Yorkshire

June 17, 2019June 14, 2019 ~ Adam ~ 1 Comment

I attended Agile Yorkshire last week and saw two great talks by Tom Hoyland and Jon Fulton. I really enjoyed both but a few points in Tom’s talk really interested me and I’d like to take a few minutes to share them.

Tom is a Scrum Master at Sky Betting and Gaming, I’ve heard good things about the company in the past so I was interested to hear one of their success stories. It turns out that Tom was part of a team of twelve who really stripped Agile “back to basics” and conducted a series of experiments on the road to continuous delivery. Working in a regulated industry myself I was intrigued how they’d got on.

One of the first things Tom talked about was how many different people in the team came to the table with ideas of what was agile best practice. We all laughed at his “my guru is better than your guru” but it makes a lot of sense! I am heavily influenced by Jez Humble, Gene Kim, and Clarke Ching but many of my colleagues may watch talks and read blogs from very different thought leaders. Tom explained that one of the first thing they had to do in the team’s formation was break many of the concepts down to their fundamental concepts and understand what worked for them.

Something else Tom discussed was how the team consolidated their own backlog. This was not controversial, how else would you prioritise the work against it? It was only when he gave examples of some of the different backlogs they’re identified that I became intrigued. Risk Logs, Retro Actions, and Design Session – all of these moved onto the board and each became visible and prioritised.

It’s dangerous out there – take your buddy! I’ve heard many of the advocates or pair programming before but the idea of your buddy following you into meetings, design sessions, and CABs!? Tom explained that if your buddy went with you to these sessions not only would they learn how to design, and walk work through the CAB but they’d also know the current state of everything you were working on. If you were sick or the proverbial bus came along the team wouldn’t need to bother you because everyone would know what was going on.

group hand fist bump — Photo by rawpixel.com on Pexels.com

There were many other good ideas (and I intend to borrow quite a few of them myself) but the final one I’m going to mention was the idea that Velocity is in fact a vanity metric (read The Lean Startup if you have no idea what I’m talking about). Velocity is just a number, like Number of Users or Number of Page Views). What we want are actionable metrics, like team predictability and accuracy of forecasts. As a Development Manager I frequently use the team’s average velocity to forecast delivery dates, Tom recommended that there are better measures out there such as a temperature check of the team’s current mood (which would often dip before any reduction in velocity). It’s an interesting idea, and one I intend to think more about over the next few weeks!

A big thank you to Royd and the guys who put Agile Yorkshire together each month. An equally big thank you to Tom and Jon for their great talks!

April Agile Yorkshire

April 17, 2017April 14, 2017 ~ Adam ~ Leave a comment

I managed to get a seat at Agile Yorkshire this month, I’ve missed a few of these recently partly due to other commitments but also the sheer popularity of these events.

Royd (from NewRedo) organises and coordinates these evenings, this week he’d arranged for Chris Cheadle and Sean Craig from NHS Digital and John Le Drew who runs The Agile Path to speak to us.

Chris and Sean went first, they spoke to us about an event they’d run a little before Christmas. They’d called it Firebreak, during a two week period almost the entire organisation downed tools and worked on “whatever they wanted”.

They’d started with almost a kickstarter approach, people posted ideas on postit notes and their colleagues pledged their time – once a project was fully resourced it was banked and it would go ahead.

I love the idea of this sort of thing, really opening the doors to let teams work on what they want – what they feel wild make a difference. Anything from process improvement to a proof of concept or a piece of server maintenance. It certainly seemed to be a positive experience for the NHS team, some of the projects saved thousands of pounds on licence fees!

The second talk of the night was about Safety and how important it is to effective teams. John explained that he considered safety to be “free to make suggestions, give feedback, and make mistakes without fear of punishment or humiliation.”

For me this feeds into the fundamental requirement of trust which Patrick Lenconi described in his book Five Dysfunctions of a Team. He postulated that without trust (or in John’s words safety) teams would not challenge each other and discuss ideas.

John had a few examples of this, my favourite was a role play exercise where three characters were so determined to avoid taking the blame for pressing The Big Red button which would overload the nuclear reactor they refused to cooperate to press the three buttons which would save them. Contrived? Perhaps… but it makes the point that if you are scared to look foolish then you’ll naturally be less confident to make suggestions.

Something which did niggle me was the idea of accountability, as managers we need to hold our directs responsible for their performance but this is challenging without undermining that feeling of security. In Lenconi’s book he encourages the team to hold each other accountable, finding the balance between a blame culture and safe, self motivating team is a difficult balance to find!

I’d certainly recommend listening to John if he’s speaking in your area. At the very least I’d say every manager should hear his views about engagement and workplace stress! I for one will be listening to his podcast.

It was a great night, as I’ve said before I’d always suggest Agile Yorkshire if you’re a Leeds based geek!