Tag: The Phoenix Project

Five Books to Revisit

~ Adam ~ Leave a comment

I’ve probably mentioned before that I try to read or listen to The Phoenix Project each year. Each time I pick up different things and, although rather dated now, I believe it’s still one of the most important books in our industry.

This year I’ve decided to take things a little further and run over five key revision books and listen to them back to back. My greatest hits if you like of what I feel are the cornerstones for current software development without digging into the actual code. In no particular order my books are:

The Phoenix Project

As I mentioned above TPP is one of the most important books of recent years for our industry, it’s a great introduction into devops and the idea of system thinking and should be required reading for anyone in a software development role. Lets skip over the bit where they consipire to conceal a huge customer data breach from the auditors.

The Unicorn Project

The Unicorn Project came out much more recently and tells The Phoenix Project from the perspective of Maxine, the developer who caused the payroll failure which kicked off the story. The Unicorn Project talks about the value of paying down technical debt, decoupling systems, and architecting for sustainability. It evangelises functional programming a little too much for my liking and Eric calls everyone “sensei” but it’s a very valuable and enjoyable read.

Accelerate

Ok, The Phoenix Project and The Unicorn Project are fun stories about DevOps. This is data and proof. Nicole Forsgren is a PhD, research is her day job and she was the driving force proof between The State of DevOps reports for years. In this book they delve into industry best practices and categorically prove that they lead to not only better development team satisfaction and performance but better business performance.

Accelerate: The Science of Lean Software and Devops: Building and Scaling High Performing Technology Organizations: Amazon.co.uk: Nicole Forsgren, Jez Humble: 9781942788331: Books

Rolling Rocks Downhill

You’re going to be surpised by this one but I REALLY like Rolling Rocks Downhill by Clarke Ching. It talks about many of the similar ideas of the previous three books on my list but goes into much more detail around the financial benefits and priortising options of working in an agile manner. It’s also actually really funny, I find myself chucking all the way through – something which is very rare in a technical book!

Drive

I wanted to go for something different for book number five. There were some VERY strong contenders including Team Topologies, Radical Candor, and The Lean Startup. I also can’t really list my own books… so I finally settled on Drive.

If you’ve not read any of Dan Pink’s books before they’re worth a look. He typically looks at a particular psychology idea (in this case motivation) and discusses it in nice accessible language. He’s very good at translating scientific research into business and layman’s terms.

In Drive he discusses many of the key elements which are required to keep employees motivated and happy and, shockingly many of those same aspects line up with the research conducted by Nicole Forsgren and advocated by Erik in Gene Kim’s books.

Drive: The Surprising Truth About What Motivates Us eBook : Pink, Daniel H.: Amazon.co.uk: Books

What do you think to my five revision books? What have I missed which I really must read next? I’d love to hear in the comments below or on Twitter.

Four Types of Work in An Agile World – Revistied

~ Adam ~ 3 Comments

It’s been over three years since I wrote my post Four Types of Work in an Agile World and a lot has changed since then but it is still, by far the most popular post on my blog. I wanted to take a minute to revise the post as I believe things have changed, in the industry in general – not just my head!

When I wrote about this topic originally I described the four types of work introduced by The Phoenix Project as:

  • Planned Work
  • Internal Projects
  • Changes
  • Unplanned Work

I discussed how we used the product backlog to mange the first three and introduced slack into everyone’s sprint and a SWAT team to mop up as much of the Unplanned Work as possible to avoid it jeopardising the planned work.

Over the subsequent years I’ve considered this a little futher. Partially as my knowledge of DevOps and Scrum have grown, but also because – let’s face it, no one wants to be in that SWAT team!

The first change I’d make to The Phoenix Project’s famous 4 types of work is to recognise that in reality all work is either Planned or Unplanned. Internal Projects are a type of Planned Work and Changes are an implementation phase of both. Lets say instead we have:

  • Planned Work
    • Business Projects
    • Internal Projects
  • Unplanned Work
Unplanned Work will always threaten to destroy Planned Work. Photo by Christina Morillo on Pexels.com

Next I’d argue that my previous approach of having a distinct team to shelter the project team is incorrect. It’s another phase of waterfall, splitting different phases of the Develop, Verify, Run into different teams creating silos which don’t talk to each other or share learning. As someone once said – if it’s you getting paged at 3am because the website has crashed you’ll quickly make sure you resolve it in office hours!

Teams must be either orientated around projects or focused on delivering platforms as described in Team Topologies. Those teams must ensure the entire lifecycle from design to operation. To do anything else is to isolate that team from the user and reduce feedback see DevOps 2nd Way).

These teams must also ruthlessly hunt down and eliminate the technical debt which leads to this unplanned work. Creating a buffer team to try to contain the work will only last until lack of feedback and continuously declining quality consume that buffer and overflow back to the development team.

Tech Debt can only be paid down by:

  • Increasing the frequency of deployments (thereby decreasing the delta).
  • Allocating between twenty and thirty percent of the team’s capactity to preventative maintenance and tech debt work.
  • Improving operability and monitoring in production.

Teams have to be utterly ruthless when they resolve issues in production to take action so that they can’t happen again. Or at least that they will be much easier to detect and resolve the next time. Unfortunately, many businesses do not invest the time necessary to maintain their own products. This leads to conflict between the teams and the business and ultimately fails both sides.

This is where it is so important to have your Product Owner on side. I won’t attempt to duplicate many excellent articles on the value of paying down tech debt and having that conversation. I’d only be repeating the words of others. What I would say is that providing meaningful metrics on code quality using tools like StyleCop, Fortify, and SonarQube help quanitify the risk to the business of continuing to operate in this way.

Negotiate the composition of your backlog with your Product Owner. Photo by Tirachard Kumtanom on Pexels.com

If you want to get your Planned Work back on track you need to hunt down and destroy the sources of Unplanned Work (this is not to say you shouldn’t respond to change when it comes along). It means that Unplanned Work is a symptom of technical debt, either in the form of bugs, lack of clarity, or poor operability.

If you want your team to be able to get on with improving your product, you need to ensure that they aren’t getting distracted by people struggling to use it – otherwise you won’t have anyone wanting to use it at all!

The Unicorn Project Book Review

~ Adam ~ 2 Comments

When I first heard about The Unicorn Project I have to admit I was disappointed, I’ve long been a evangelist for Gene Kim’s book The Phoenix Project but I’d just spent months working on my own development DevOps book, Code Black.

I shouldn’t have worried, I really enjoyed The Unicorn Project and we’d gone down different angles. Where I’d focused more on the Continuous Deployment journey Kim’s book focuses much more on developer empowerment and continuous experimentation.

The Unicorn Project

The story follows Maxine, the developer who caused the now legendary payroll outage at Parts Unlimited towards the start of The Phoenix Project. Exiled to the documentation team as punishment she’s instructed to support the Phoenix rollout but quickly realised how woefully under supported the engineering teams are. As the business piles on more and more pressure, expects more features, and has less and less appreciation for the technical debt they’re wracking up they continue. Until, as we know the entire project explodes.

Working with some familiar characters such as Bill, Brent, Erik, and Maggie and a few new ones including Cranky Dave and Kurt our heroine works to make life better for the entire company. These are the engineers, the red-shirts, not the bridge crew. They’re the ones who actually do the work and they’ve got a lot of it to do!

What did I like best? Kim put lots of emphasis on testing and improving the entire system not just a small part of it, he focuses on collaboration and the importance of making it easy to onboard developers and share knowledge, and really drives the need to innovate and out experiment the competition. He also emphasis the importance of treating engineering tools as important systems and draws distinctions between the IT products we build, and the miscellaneous ones which just keep the lights on.

What wasn’t so good? Within a few chapters I was absolutely sick of Erik’s use of the word “sensei”… seriously can’t some of the people he quotes simply be experts, evangelists, or even gurus!?

On a more practical point the book spends a lot of time evangelising functional programming and scalability technologies. Which is great, they’re very powerful tools. But one of the things I liked so much about The Phoenix Project was how it was clear the team were struggling the same tech debt we all are. It made it more relatable and I worry in this book Kim’s “rip it out and use the latest and greatest” will overpower his more generic messages of continuous incremental improvement. Perhaps it’s personal preference but I like my DevOps books technology agnostic.

So, would I recommend this book? Absolutely without question! I believe that The Unicorn Project will take its place alongside its elder sister on the bookshelves of developers, testers, managers, product owners, and operational engineers for the next decade. If you haven’t already go and buy it, any while you’re at it not get a copy of Code Black too!? 🤣

The Three Ways of DevOps

~ Adam ~ 1 Comment

If you’ve ever read The Phoenix Project (or pretty much any other DevOps book) then you’ll most have heard about the three ways.

The first time I heard the name I imagined some deep levels of understanding that the DevOps Sensai was imparting mystical knowledge to young grasshopper. I still like the mental image but having read a little deeper I believe they are three steps to building a successful collaborative team.

Let’s look at the three ways.

The First Way – Systems Thinking

The idea of the first way is to appreciate that each series of tasks is interconnected into a complex system. Recently I described how a delay at one stage in the system knocks on to each subsequent task, having a good understanding of the overall system allows you to identify where your bottlenecks are, exploit them and avoid work queuing up in front of any individual person or process.

The Second Way – Amplifying Feedback

The second way is all about looking at what happens at the end of the system and using that knowledge to improve the process. If a particular result is of a poor quality then the team need to know so they can focus more effort in the testing phases. Equally if a release is of a particularly high quality or gets good customer feedback then the team need to know that what they are doing works!

The Third Way – Experimentation

The third way is about constant experimentation and improvement. The team is willing to take risks (because they know their feedback loops are good and they will find out about any issues quickly). These teams adapt to changes quickly and are not afraid to challenge processes just because “that’s the way we’ve always done it”.
I believe that the fact these ‘ways’ are numbered is in fact very important. Without an appreciation of the end to end system it’s impossible to develop strong feedback, after all where does the system end and where should your feedback be coming from!? Equally, without this strong feedback experimentation is extremely dangerous – you may in fact be making things worse!

I also believe that these concepts can be introduced to a scrum team to lift them to the next level. By introducing ideas such as systems thinking and feedback (already a strong Agile theme) you’re asking the question of what in fact is the definition of done? DevOps encourages collaboration across Development and Operations, if a scrum team includes both roles then it’s logical that work is only “Done” when it is running properly in a customer’s environment. That surely, is the ultimate goal of any software development team?

Have you come across the The Ways? What are your thoughts? Can you use them in a scrum team?

Why Do Most Projects Finish Late?

~ Adam ~ Leave a comment

I’m currently reading Agile Estimating and Planning by Mike Cohn, one of the things he discusses in the early chapters is why so many of projects fall behind. Many of his ideas fall in line with Eli Goldratt’s thinking in The Goal and what is described as The First Way in The Phoenix Project.

Mike discusses Parkinson’s Law which postulates that work will always take the time allocated to it. In other words if you’ve got a project and a deadline you won’t finish early because you’ll use the remaining time to refine, improve, and polish the work.

He also discusses an idea raised in The Goal where tasks are dependent on each other. In Eli Goldratt’s book Alex realises the importance of interdependencies when he takes the boy scouts walking through the woods. Cohn uses the example of developers and testers and how the person doing the QA cannot begin until the functionality has been created.

If you combine these two theories you realise that if tasks only run late, never early and the subsequent tasks can never begin until the previous one has finished you end up with an ever slipping schedule. If each task misses it’s deadline 10% of the time then once you’ve multiplied up by the number of tasks the probability that your delivery will run late climbs very quickly towards a statistical certainty!

So what can we do about this? Build in contingency? This is a risky strategy as if the team (or even you) know that there’s flexibility built into the schedule then the work will continue to consume all available time.

One approach I’ve heard a lot more recently is to allow projects to be constrained by either time or feature lists but never both. In a time based approach functionality is ranked in order of importance and when the clock runs out the project is delivered (regardless of whether or not all features are complete). In a feature driven release (for example in a lean MVP project) then the team will continue to work until all features have been completed – regardless of how long that takes.

Personally I’m much more of a fan of the first approach. By keeping this prioritised list transparent with the clients and stakeholders you can work on exactly the right work. Your work is cut when the time runs out (rather than adding low value features and running over) and one of my favourite reasons for adopting it – if a project does take less time than you expect your client gets more functionality for their money instead of feeling ripped off by inflated estimates, that’s something I’d certainly appreciate as a customer!

What do you think? How do you agree on project deadlines and commitments?

The Four Types of Work in an Agile World

~ Adam ~ 4 Comments
NOTE: I’VE REVISTED THIS TOPIC MUCH MORE RECENTLY, IF YOU’RE INTERESTED ON HOW MY VIEWS HAVE CHANGED READ 4 TYPES OF WORK REVISTIED.

If you have read The Phoenix Project (which I highly recommend you do) you’ll be more than familiar with Eric who introduces us to the four types of work.

  • Planned Work – these are typically business projects or new features
  • Internal Projects – server migrations, software updates and so on
  • Changes – usually driven by feedback on already completed work
  • Unplanned Work – support escalations and emergency outages

In the book Eric talks about how Unplanned Work is unpredictable and destructive as it can often push out planned work. I can’t even begin to count how many times have I’ve planned in projects and tasks only for a crisis to arrive and my sprint to fail!

Managing a Product Backlog is an excellent way to handle three of the four types of work. After all, Agile Principles encourage us to collaborate with stakeholders, listen to feedback, and embrace change. The value of internal projects can be evaluated against other projects and then planned correspondingly and what is the backlog at all if not a prioritised list of Planned Work?

It’s the infamous Unplanned Work which continues to disrupt our Sprints, break our commitments and lower our productivity. Over a few years I’ve tried a number of different approaches to handling the inevitable Unplanned Work.

I’ve tried introducing slack into everyone’s sprint, only planning fifteen instead of twenty Story Points. I’ve tried allocating Support Tickets as they come in, assuming that the demand will cause our average sprint velocity to drop and the balance between Planned and Unplanned Work to find a happy equilibrium. It never does!

The only strategy I’ve found to work is to completely isolate the Unplanned Work from the other three types. Create a specific team (in our company we call this the SWAT team) who will react to all incoming unplanned work and “pitch in” with other work when there isn’t enough to do (Parkinson’s Law pretty much guarantee’s that this never happens).

The SWAT team handle the urgent, unplanned, and reactive work (often using a strategy like Kanban or Scrumban) while the rest of your team focus on delivering maximum value to the business (Sprints, commitments, and all the other wonderful concepts we hear about on Agile courses). That’s not to say they don’t fix bugs or take on internal projects, it’s that they work in a structured, planned environment in an effort to build solid releases and reduce the amount of Unplanned Work long term.

Speaking of Parkinson’s Law if you do decide to create this buffer between Planned and Unplanned Work (and I strongly suggest you do) you need to consider what happens when the work expands to fill the allocated time. What happens when the urgent support demands of the business exceed the capacity of your SWAT team? Hire extra people? That would be nice! But I can tell you what you must not do – DO NOT pass the extra work over into your Scrum team. I’ve made that mistake before and I can assure you that it’s a very slippery slope which is nearly impossible to come back from. Once you start roping in extra developers you undo your good work and Sprints start to fail once again. I mention it between I’ve seen it happen!

I hope I’ve given you some thoughts, I hope that my views on throttling the amount of damage your Unplanned Work can do help you safe your own commitments but I’d be delighted to hear your views – what do you to do safeguard your team’s productivity while meeting the urgent needs of the business? Post a comment below and let me know!