The Three Ways of DevOps

If you’ve ever read The Phoenix Project (or pretty much any other DevOps book) then you’ll most have heard about the three ways.

The first time I heard the name I imagined some deep levels of understanding that the DevOps Sensai was imparting mystical knowledge to young grasshopper. I still like the mental image but having read a little deeper I believe they are three steps to building a successful collaborative team.

Let’s look at the three ways.

The First Way – Systems Thinking

The idea of the first way is to appreciate that each series of tasks is interconnected into a complex system. Recently I described how a delay at one stage in the system knocks on to each subsequent task, having a good understanding of the overall system allows you to identify where your bottlenecks are, exploit them and avoid work queuing up in front of any individual person or process.

The Second Way – Amplifying Feedback

The second way is all about looking at what happens at the end of the system and using that knowledge to improve the process. If a particular result is of a poor quality then the team need to know so they can focus more effort in the testing phases. Equally if a release is of a particularly high quality or gets good customer feedback then the team need to know that what they are doing works!

The Third Way – Experimentation

The third way is about constant experimentation and improvement. The team is willing to take risks (because they know their feedback loops are good and they will find out about any issues quickly). These teams adapt to changes quickly and are not afraid to challenge processes just because “that’s the way we’ve always done it”.
I believe that the fact these ‘ways’ are numbered is in fact very important. Without an appreciation of the end to end system it’s impossible to develop strong feedback, after all where does the system end and where should your feedback be coming from!? Equally, without this strong feedback experimentation is extremely dangerous – you may in fact be making things worse!

I also believe that these concepts can be introduced to a scrum team to lift them to the next level. By introducing ideas such as systems thinking and feedback (already a strong Agile theme) you’re asking the question of what in fact is the definition of done? DevOps encourages collaboration across Development and Operations, if a scrum team includes both roles then it’s logical that work is only “Done” when it is running properly in a customer’s environment. That surely, is the ultimate goal of any software development team?

Have you come across the The Ways? What are your thoughts? Can you use them in a scrum team?

But I’m Not In Customer Service…

I’m going to start with a confession before hopping on my soap box. I was recently copied in on a couple of emails going back and forth with a customer. I wasn’t overwhelmed by our responses, in fact there were quite a few things I would have changed, the information was accurate but it wasn’t conveyed particularly well. I was about to get involved when I had the thought “But I’m not in customer service…”

All I can say is that I’m extremely glad I didn’t say those words out loud, as soon as my internal monologue had paused for breath I realised just how little sense it was making. We are ALL in the business of customer service!

In most corporations we’re split into various teams, often with complimentary skills. We are then given goals and objectives to complete. The majority of my time is spent supporting my team and working with upcoming releases. On a day to day basis I don’t speak to our customers that frequently (unless we’re collaborating on a new feature or UAT obviously).

Does that mean I’m not in the business of customer service?

Does that mean that I don’t need to consider the customer?

Absolutely not!

Aside from the fact that it’s our customers who pay my salary as a developer the entire goal of my working day is to provide a high quality product to our customers. There are many parts of that, a good design, a high quality implementation, but also the marketing and support we provide to our customers.

If you were searching the app store and you found two equivalent 99p applications one with beautiful screenshots and another with bad grammar, typos, and a vague feature list which would you invest your money in?

If you were making the decision for a multi-million pound software investment would you rather work with the team who fob off your questions or the one one who can provide high quality answers?

Now, I’m not saying that we all need to move into marketing and join the OU’s customer service courses. But I do believe that we all need to remember that the reason our business is operating is because we have customers – providing a good service to them is in ALL of our remits from the developers to the accountants and everyone in between. Build your features with the customers’ frame of mind, write your email with the phrasing you’d like to receive and let’s not any of us mutter the phrase “but I’m not in customer service!”

Why Do Most Projects Finish Late?

I’m currently reading Agile Estimating and Planning by Mike Cohn, one of the things he discusses in the early chapters is why so many of projects fall behind. Many of his ideas fall in line with Eli Goldratt’s thinking in The Goal and what is described as The First Way in The Phoenix Project.

Mike discusses Parkinson’s Law which postulates that work will always take the time allocated to it. In other words if you’ve got a project and a deadline you won’t finish early because you’ll use the remaining time to refine, improve, and polish the work.

He also discusses an idea raised in The Goal where tasks are dependent on each other. In Eli Goldratt’s book Alex realises the importance of interdependencies when he takes the boy scouts walking through the woods. Cohn uses the example of developers and testers and how the person doing the QA cannot begin until the functionality has been created.

If you combine these two theories you realise that if tasks only run late, never early and the subsequent tasks can never begin until the previous one has finished you end up with an ever slipping schedule. If each task misses it’s deadline 10% of the time then once you’ve multiplied up by the number of tasks the probability that your delivery will run late climbs very quickly towards a statistical certainty!

So what can we do about this? Build in contingency? This is a risky strategy as if the team (or even you) know that there’s flexibility built into the schedule then the work will continue to consume all available time.

One approach I’ve heard a lot more recently is to allow projects to be constrained by either time or feature lists but never both. In a time based approach functionality is ranked in order of importance and when the clock runs out the project is delivered (regardless of whether or not all features are complete). In a feature driven release (for example in a lean MVP project) then the team will continue to work until all features have been completed – regardless of how long that takes.

Personally I’m much more of a fan of the first approach. By keeping this prioritised list transparent with the clients and stakeholders you can work on exactly the right work. Your work is cut when the time runs out (rather than adding low value features and running over) and one of my favourite reasons for adopting it – if a project does take less time than you expect your client gets more functionality for their money instead of feeling ripped off by inflated estimates, that’s something I’d certainly appreciate as a customer!

What do you think? How do you agree on project deadlines and commitments?

Testing Atelier

On Tuesday I was lucky enough to get tickets for one of my colleagues and I to go to the Leeds Testing Atelier. I’ve never been to one of these before but wow, the guys had worked extremely hard and created an amazing day!

There were two tracks (hipsters and nerds) throughout the day and it was action packed with different talks, topics, and workshops.

Before we got going however Clem led a group of us in a Lean Coffee session. I’ve never done one before (most likely because of my intense dislike of coffee!) but it’s definitely an idea I’ll be be trying out in team meetings at work!

I attended a couple of talks in the morning. The first was on Unit Testing best practices which I enjoyed, I got the chance to as a question on custom assertions which test multiple things (something I’ve been debating in my own head for a while). The answer by the way was “it’s ok, as long as your test continues to only test one thing” – a view I agree with!

Next up we’re a couple of short talks, one one using agile techniques to plan family life and other on website performance profiling. Both interesting and certainly talking points!

After a break Alex Carter spoke to us about the roles QAs can play in building the three ways of DevOps.

The three ways (in case you’ve not come across them are)

  1. Systems thinking
  2. Amplify feedback
  3. Continuous experimentation and improvement

img_7514

It turns out that a QA is key in making this work. They’re the quality gatekeepers, they challenge processes to build quality in at all stages and act as the team’s safety net when risky changes are made. If you’ve never run through this in your head (or even better your team) then I highly recommend you do!
Lunch was pizza, in fact huge amounts of pizza! Then we headed upstairs for some QA based fun and games (some seriously difficult interviewing and spot the differences).

img_7515

My final session of the day was a panel session on continuous delivery. The guys answered questions on everything from getting started to business challenges. There was a chance to ask questions at the end.

In summary the Leeds Testing Atelier was great. It was informal, informative, and had a great atmosphere with people willing to share experiences and ask questions. I’d like to thank the sponsors and organisers for all their hard work. If you’ve not been to one of these before then I’d highly recommend going to 2018 – I know I will be!

The Four Types of Work in an Agile World

If you have read The Phoenix Project (which I highly recommend you do) you’ll be more than familiar with Eric who introduces us to the four types of work.

  • Planned Work – these are typically business projects or new features
  • Internal Projects – server migrations, software updates and so on
  • Changes – usually driven by feedback on already completed work
  • Unplanned Work – support escalations and emergency outages

In the book Eric talks about how Unplanned Work is unpredictable and destructive as it can often push out planned work. I can’t even begin to count how many times have I’ve planned in projects and tasks only for a crisis to arrive and my sprint to fail!

Managing a Product Backlog is an excellent way to handle three of the four types of work. After all, Agile Principles encourage us to collaborate with stakeholders, listen to feedback, and embrace change. The value of internal projects can be evaluated against other projects and then planned correspondingly and what is the backlog at all if not a prioritised list of Planned Work?

It’s the infamous Unplanned Work which continues to disrupt our Sprints, break our commitments and lower our productivity. Over a few years I’ve tried a number of different approaches to handling the inevitable Unplanned Work.

I’ve tried introducing slack into everyone’s sprint, only planning fifteen instead of twenty Story Points. I’ve tried allocating Support Tickets as they come in, assuming that the demand will cause our average sprint velocity to drop and the balance between Planned and Unplanned Work to find a happy equilibrium. It never does!

The only strategy I’ve found to work is to completely isolate the Unplanned Work from the other three types. Create a specific team (in our company we call this the SWAT team) who will react to all incoming unplanned work and “pitch in” with other work when there isn’t enough to do (Parkinson’s Law pretty much guarantee’s that this never happens).

The SWAT team handle the urgent, unplanned, and reactive work (often using a strategy like Kanban or Scrumban) while the rest of your team focus on delivering maximum value to the business (Sprints, commitments, and all the other wonderful concepts we hear about on Agile courses). That’s not to say they don’t fix bugs or take on internal projects, it’s that they work in a structured, planned environment in an effort to build solid releases and reduce the amount of Unplanned Work long term.

Speaking of Parkinson’s Law if you do decide to create this buffer between Planned and Unplanned Work (and I strongly suggest you do) you need to consider what happens when the work expands to fill the allocated time. What happens when the urgent support demands of the business exceed the capacity of your SWAT team? Hire extra people? That would be nice! But I can tell you what you must not do – DO NOT pass the extra work over into your Scrum team. I’ve made that mistake before and I can assure you that it’s a very slippery slope which is nearly impossible to come back from. Once you start roping in extra developers you undo your good work and Sprints start to fail once again. I mention it between I’ve seen it happen!

I hope I’ve given you some thoughts, I hope that my views on throttling the amount of damage your Unplanned Work can do help you safe your own commitments but I’d be delighted to hear your views – what do you to do safeguard your team’s productivity while meeting the urgent needs of the business? Post a comment below and let me know!

April Agile Yorkshire

I managed to get a seat at Agile Yorkshire this month, I’ve missed a few of these recently partly due to other commitments but also the sheer popularity of these events.

Royd (from NewRedo) organises and coordinates these evenings, this week he’d arranged for Chris Cheadle and Sean Craig from NHS Digital and John Le Drew who runs The Agile Path to speak to us.

Chris and Sean went first, they spoke to us about an event they’d run a little before Christmas. They’d called it Firebreak, during a two week period almost the entire organisation downed tools and worked on “whatever they wanted”.

They’d started with almost a kickstarter approach, people posted ideas on postit notes and their colleagues pledged their time – once a project was fully resourced it was banked and it would go ahead.

I love the idea of this sort of thing, really opening the doors to let teams work on what they want – what they feel wild make a difference. Anything from process improvement to a proof of concept or a piece of server maintenance. It certainly seemed to be a positive experience for the NHS team, some of the projects saved thousands of pounds on licence fees!

The second talk of the night was about Safety and how important it is to effective teams.  John explained that he considered safety to be “free to make suggestions, give feedback, and make mistakes without fear of punishment or humiliation.”

For me this feeds into the fundamental requirement of trust which Patrick Lenconi described in his book Five Dysfunctions of a Team. He postulated that without trust (or in John’s words safety) teams would not challenge each other and discuss ideas.

John had a few examples of this, my favourite was a role play exercise where three characters were so determined to avoid taking the blame for pressing The Big Red button which would overload the nuclear reactor they refused to cooperate to press the three buttons which would save them. Contrived? Perhaps… but it makes the point that if you are scared to look foolish then you’ll naturally be less confident to make suggestions.

Something which did niggle me was the idea of accountability, as managers we need to hold our directs responsible for their performance but this is challenging without undermining that feeling of security. In Lenconi’s book he encourages the team to hold each other accountable, finding the balance between a blame culture and safe, self motivating team is a difficult balance to find!

I’d certainly recommend listening to John if he’s speaking in your area. At the very least I’d say every manager should hear his views about engagement and workplace stress! I for one will be listening to his podcast.

It was a great night, as I’ve said before I’d always suggest Agile Yorkshire if you’re a Leeds based  geek!

Who Should you Share your Backlog With?

I mentioned last week that I’d been to an interesting meeting discussing (and challenging) a few of the Scrum ideas. One of the questions we asked was about Backlog Visibility.

We all agreed that by making the backlog visible to our customers would invite feedback and reduce the likelihood that we’d spend time developing features which gave little value to our clients (see last week’s post on exactly what “value” is). But we couldn’t shake a distinct unease at making our plans visible, it took quite some discussion to articulate why this was.

Many customers and businesses work to project plans (ours included), they have long term visions of where they want to be and what they want to achieve. We realised with some trepidation that by sharing a fluid and ever changing backlog list of ideas with these customers it would create an expectation, which upon the next meeting would lead to disappointment as we had to admit the features we discussed were never in fact developed.

In an Agile environment we want to encourage this change, we want to update our backlog to remove unnecessary items and reorder the list to put the maximum value items at the top. However when presenting this to a client it could undermine us, particularly if the customer liked several of the items we ultimately decided to prune.

We decided that this is where clear communication is key. We talked about making it extremely clear that these were items we were considering working on and not a definitive product plan. A Development Forecast we suggested, after all – no one blames the weatherman if the reality turns out to be slightly different to our expectations!

What are your thoughts, do you share your Product Backlog with your clients? How do you handle their expectations that some work they are hoping for may still be cut?