After all the positive feedback I recieved writing Code Black last year I’ve been working on something else.
For those who don’t know Code Black is a business parable novel for DevOps techniques. In other words it’s a story, about a team at a failing IT company who embrace modern software development techniques to produce higher quality software in a much more efficient way.
Writing a book is not easy, it’s never turned into a best seller (although who knows, this post may go viral!) but I enjoyed the process and I learned a lot putting it together.
Once again Donuts and Dragons is a story, this time about Megan who joins a team who are working to develop the next best selling game. Instead of DevOps I’m focusing on agile techniques.
I’m publishing the story on LeanPub, the site (in a very agile manner) encourages you to publish early and often. The word count is going up, slowly and steadily as a first draft. As the story is very much in the early stages I’m not charging for anyone who wants to read it and provide feedback.
As I mentioned in my recent goals post, I’m hoping to finish the book in 2021. To do this I’m hoping to have the first draft completed by June and then I’ll have plenty of time to proof read and listen to your feedback. Why not have a look at what I’ve got so far? I’d love to hear your feedback!
Many larger companies are following the lead of companies like Spotify’s to create engineer led communities, sometimes called Guilds. The concept is simple, and very good. Creating a cross team culture of engineering excellence to drive best practice forward among employees sounds like a brilliant idea. In this post I’m going to discuss my experience of working with Communities of Practice and my thoughts on their true value.
Companies usually start a CoP initiative when they want to engineers to take ownership of:
Defining Standards and Best Practice
Training and developing it’s members
Breaking down Silos around teams
Hiring and building a better team
It sounds great right?
As with most things the reaslity isn’t quite as straightforward as that.
As anyone who reads my blog may pick up I enjoy a little amateur psychology, I believe it helps me in my role as a team manager. One of the things I’m acutely aware is the tribe mentality of scrum teams, when people get so focused behind their products they often find it difficult to see other teams as allies – too often they’re seen as impediments to deliveries or competing products. Forming a Community of Practice asks people to shift their focus and split it away from their scrum teams to think as a wider team. It can be hard to sit with two hats on!
So, how do you get started?
There’s a very good book by Emily Webber called Building Successful Communities of Practice which outlines a very good approach. She talks about a Community Maturity Model which outlines various objectives the forming community should aim to hit as they grow. By splitting out into distinct Potential, Forming, Maturing, and Self Sustaining phases. It’s important to realise that giving someone a goal and letting them go is not enough to build a Community of Practice.
With CoPs I’ve helped develop it’s crucial to start with a small group of engaged people who set goals and achieve them. Using these successes and achievements as an example to recruit more members and take on more ambitious goals. You can force dozens of people to join a meeting but you won’t get engagement, you want to be able to hold up what the community has achieved, show it off, advertise it, and ask other people if they would help deliver the next goal.
Communities should set their own goals. They need the ability to ask their members what work is needed and the autonomy to be able to go and make it happen. Otherwise they’re simply a vehicle for management to share project work between teams.
Finally, these formative and fragile communities need real organisational support. They need people to have goals set around forming the community, these people need time (I recommend setting the expectation of a day a week out of regular team duties), they also need a discretionary budget. Breakfast, coffee, prizes, and events go a long way to getting people in the door to hear what you want to achieve and once you’ve got them a social budget is a great way to help those cross team relationships form!
If you’ve been involved in Community of Practices or “Guilds” as they’re sometimes known let me know. What advice would you give to someone who wants to form them in their organisation?
It’s been over three years since I wrote my post Four Types of Work in an Agile World and a lot has changed since then but it is still, by far the most popular post on my blog. I wanted to take a minute to revise the post as I believe things have changed, in the industry in general – not just my head!
When I wrote about this topic originally I described the four types of work introduced by The Phoenix Project as:
I discussed how we used the product backlog to mange the first three and introduced slack into everyone’s sprint and a SWAT team to mop up as much of the Unplanned Work as possible to avoid it jeopardising the planned work.
Over the subsequent years I’ve considered this a little futher. Partially as my knowledge of DevOps and Scrum have grown, but also because – let’s face it, no one wants to be in that SWAT team!
The first change I’d make to The Phoenix Project’s famous 4 types of work is to recognise that in reality all work is either Planned or Unplanned. Internal Projects are a type of Planned Work and Changes are an implementation phase of both. Lets say instead we have:
Next I’d argue that my previous approach of having a distinct team to shelter the project team is incorrect. It’s another phase of waterfall, splitting different phases of the Develop, Verify, Run into different teams creating silos which don’t talk to each other or share learning. As someone once said – if it’s you getting paged at 3am because the website has crashed you’ll quickly make sure you resolve it in office hours!
Teams must be either orientated around projects or focused on delivering platforms as described in Team Topologies. Those teams must ensure the entire lifecycle from design to operation. To do anything else is to isolate that team from the user and reduce feedback see DevOps 2nd Way).
These teams must also ruthlessly hunt down and eliminate the technical debt which leads to this unplanned work. Creating a buffer team to try to contain the work will only last until lack of feedback and continuously declining quality consume that buffer and overflow back to the development team.
Tech Debt can only be paid down by:
Increasing the frequency of deployments (thereby decreasing the delta).
Allocating between twenty and thirty percent of the team’s capactity to preventative maintenance and tech debt work.
Improving operability and monitoring in production.
Teams have to be utterly ruthless when they resolve issues in production to take action so that they can’t happen again. Or at least that they will be much easier to detect and resolve the next time. Unfortunately, many businesses do not invest the time necessary to maintain their own products. This leads to conflict between the teams and the business and ultimately fails both sides.
This is where it is so important to have your Product Owner on side. I won’t attempt to duplicate many excellent articles on the value of paying down tech debt and having that conversation. I’d only be repeating the words of others. What I would say is that providing meaningful metrics on code quality using tools like StyleCop, Fortify, and SonarQube help quanitify the risk to the business of continuing to operate in this way.
If you want to get your Planned Work back on track you need to hunt down and destroy the sources of Unplanned Work (this is not to say you shouldn’t respond to change when it comes along). It means that Unplanned Work is a symptom of technical debt, either in the form of bugs, lack of clarity, or poor operability.
If you want your team to be able to get on with improving your product, you need to ensure that they aren’t getting distracted by people struggling to use it – otherwise you won’t have anyone wanting to use it at all!
After I wrote about Getting Things Done last week I began to think how the GTD approach could work for developers working in a Scrum Team.
There’s no doubt in my mind that scrum is one of the best working processes for software development teams (yes there are others but scrum is incredibly prevalent and many teams across the world have a lot of success with it). However, having an effective scrum process does not guarantee that developers are productive or that they remember to complete the myriad of other day to day work tasks.
But let’s talk firstly about why GTD can help developers and why it’s not just another process to follow. Have you ever turned up to a meeting and had that sinking feeling when you realise you’ve forgotten to do your actions? Have you ever forgotten your timesheet? What about those times when you have tried to get your head down to write code but can’t quite forget all the other jobs miscellaneous jobs which you have to finish before the end of the day? This is because there are many tasks associated with your role which do not relate to the product or the team.
As well as writing code we ask our developers to complete an endless array of other tasks such as regulatory training, end of year reviews, interviews, timesheets, and other initiatives. These, in my humble opinion, have no place on a Sprint Backlog. They’re not for the value of the team or the but they do detract time from the sprint. Annoyingly, we we don’t provide a mechanism for developers or testers to track these if we’re not using the board – we just expect them to “be organised” and somehow find that time during a planning session. This isn’t fair and it isn’t good enough.
Our goal has to be to assist developers with the tasks their job requires which don’t fit nicely into a sprint format.
Before we go any further I’d like to point out that GTD and Scrum are not entirely unrelated ideas. The five steps of GTD are:
These match up almost perfectly with their scrum equivalents of:
Sprint Reviews and Retrospectives
David Allen, the creator of GTD, suggests reviewing your entire project list every week to ensure actions are kept up to date and nothing is missed. This process draws many parallels to a team reviewing its progress against a larger goal in a Sprint Review.
Implementing GTD for Developers
But how would you go about implementing GTD for a developer without treading on the scrum process? The first thing to realise is that the two frameworks manage work for different areas. Scrum is about product development, GTD is about managing your personal workload. There’s a crossover, where you’re working on work specifically for the product (hopefully a significant part of your day), but there are also significant areas where both work independently.
The first step, as with pure GTD, is to capture everything. This can be on paper, on your computer, or in a notetaking app. This list will form the basis of your in tray and should contain everything which is in your head or on your desk at the moment. You can limit it to work, but personally I’d recommend including your personal life too.
Your list may include items like:
One of the key tenants of GTD is to move all these items out of your head and into an inbox. Free up your brain for making decisions and having ideas not for storing stuff which would be better suited to a piece of paper. Chances are the first time you run through this you will end up with hundreds of items on this list, far more than you ever expected! If you find yourself slowing down take a look at the GTD Trigger List to see if you can tease out any more and I’m sure you’ll think of a few you’ve forgotten.
Don’t be intimidated by the sheer number of items you’re finding now. They’re already in your head, the only difference is now you’re seeing that list for real. If we can visualise it, we can throttle the number of items in progress – just like you would on a scrum board.
Going forward you’re going to capture items as soon as you think of them you don’t have the chance to forget them (remember what we said about remembering actions from meetings?). This list should never grow this large again.
Capturing Actions From Email
Email can be a particuarly tricky one so I’ll cover that one specifically. I recommend creating three folders:
I’ll explain about Waiting For later on but I want you go through your inbox and move every single email into either Action Required (you need to do something with this) or Archive. If you added it to the first folder, put it on your list.
Your goal should be to repeat this process at least daily. Remember your inbox is just that – it’s an in box. You should empty your inbox and decide what to do with it (not necessarily action it) as often as is reasonably possible.
Once you’ve constructed your list from your mind, various postit notes, your scrum board, and your inbox you’ll have a lot of work on your hands. I’m expecting you to have well over a hundred items. However, this list makes sense to you now but it won’t after a days (or even hours). We’re not going to complete these items all at once and we need to know that whenever we come back to then will still be clear to us. We want to record actions, things we are going to do – not just vague bullet points.
Implement login functionality
Reply to Billy’s email requesting feedback
Call hair dresser to confirm appointment
Reply to boss’ email and answer question
Check metrics on live site are within expected range
Note that all of these are now specific things for you to do, not vague points designed to remind you. David Allen (the creator of Getting Things Done) teaches us that one of the biggest causes of procrastination is not knowing what the next action or decision should be. Save your future self from that frustration by deciding what is actually needed right now, before you file it away to be done in the future.
GTD uses projects to organise work. The definition of a project is very broad.
“Projects are defined as outcomes that will require more than one action step to complete and that you can mark off as finished in the next 12 months.”
We’re going to use the actions list you captured to create your project list. I would recommend you keep a couple of high level projects to group various pieces of work together. I use:
However you should choose the ones which seem most appropriate to you. Within each of these I would create a project called One Off Actions.
Next you’re going to work your way down your inbox and create and organise as you go. The question you should keep asking yourself is whether your task is part of a wider project or if it’s a one off action. At this point you should also make a note of any deadlines for any of your actions.
Organising Actions Which Relate to Scrum Tasks
Many of the actions you have will relate to work you are currently undertaking on behalf of the scrum team. Code this, Code Review that, etc… There are a couple of ways to approach this. You could either create a duplicate item on your own system to track your work on the scrum board or you could ommit these from your personal system.
There are advantages to both. Having work listen on both makes it less likely you’ll lose track of something however at the expense of organisational overhead and keeping things in sync.
Personally I would recommend that you don’t track work you are doing as part of the scrum team in your GTD system as I would want to create a single source of truth. However, as long as you’re consistent I don’t believe there’s real harm either way.
As you are working through your actions you will undoubtedly encounter projects which are waiting someone else to complete an action. One of the most common reasons projects go off the rails is because a task is sat with someone and it is never followed up. This is where the additional folder we created in your email comes in useful.
Whenever you have an action which is sat with someone else tag that action with the word Waiting. I would also record the date it was delegated, the name of the person, and the date you wish to follow it up. Depending on the technology you are using you will have different mechanisms for doing this ranging from reminders and tags to simply changing the title.
Drop any corresponding emails into that new Waiting For inbox folder so those messages are always close at hand. BCC yourself into any emails you send asking for someone to pick something up and drop it in that folder!
If at all possible keep the action sat in the same project folder, this will help maintain context with the overall project.
Agendas are one of the hidden gems of GTD. Inside your work folder (although you could possibly do it for others too) I want you to create a folder for Agendas and then others underneath for:
1:1 with Boss
Along with any all hands calls or mentorship sessions you take part in. These folders represent everything you want to discuss in those upcoming meetings. This means if you have an issue you need to speak about you will never have that frustraiting feeling of not being able to quite remember what it is.
This means you can gather frustrations or highlights for the retrospective throughout the week and make a list of items you want to talk to your boss about rather than having to think of them on the spot. You can also capture any items which come from either of those two meetings quickly and easily in your inbox for clarifying and organising later.
Once you’ve raised your point and got your answer you can check them off. Remember you can use Waiting For if someone doesn’t have the solution for you right away!
While creating your actions is it often helpful to tag (or otherwise highlight depending on the technology you are using) the next action which is required to move a project along. I also tag my actions with Email, Phone, and/or 15Minutes to help me catagorise each item so I don’t have to jump from one application to another and can pick up quick tasks in between meetings.
By now you should have a list of all the actions you have committed to and their appropriate due dates. David Allen recommends you should reseve some time each week to zero your inboxes, review your projects, next actions, waiting for lists, and agendas and I would agree.
You’re looking for anything which is out of date, irrelevant, or no longer needed. You’re also looking for the next steps to progress your projects.
I would add one add additional suggestion. Ensure you schedule your personal GTD review before your sprint ceremonies. If you don’t have a clear vision of your commitments outside the sprint then you are never going to be able to give a fair view of capactity in those planning sessions. If you know you’ve got a large task to complete for a senior manager you’re now aware of that task and can take on less work in the sprint planning session.
Now we’ve got this wonderfully organised list of work it’s time to do some! There are a number of ways for deciding what to work on:
How long it will take
How urgent it is
Whether you have the right tools required (and to a degree the inclination) at that time.
Obviously the vast majority of your time will be for scrum tasks (or at least I hope it would be). However as we mentioned at the start it’s important that these other pieces are picked up and not forgotten about. I would recommend finding a certain amount of time each day to review this list and look for any actions and make sure you have plenty of time to complete them before their respective deadlines.
10 minutes at the start of the day (ideally before the daily standup) is all that’s needed to make sure you can plan effectively and you don’t miss anything which needs doing alongside your scrum deliverables.
A Note On Technology
I deliberately haven’t talked about technology in this post. The tool doesn’t make the system but it’s clear you’re going to need a good tool to keep this system working.
In his book David Allen talks a lot about Pen and Paper but also doesn’t go too much into digital tools (although personally I think he’s a little naive assuming that people won’t reach for an online tool first.
Personally I’d recommend Todoist, I find it very intuitive and reliable and they have a great GTD guide available which walks you through many of the suggestions I’ve made. However clearly there are other applications and tool available – please feel free to share any suggestions you’ve got in the comments.
In this post I’ve tried to explain why using a system like GTD is so important to developers and how it can be used to handle work which the scrum process doesn’t. I’d highly recommend having a read of the Todoist GTD Guide and giving their free version a go. If you have any other suggestions on how to integrated the two processes I’d be very keen to hear them.
In total it took about 18 months to write Code Black, my recently published technical parable story. I’d originally had the idea in the summer of 2018 but it took a little time to properly outline the story.
Instead of using a common format like The Hero’s Journey I used the various stages a team would progress through as they developed and refined their DevOps journey.
Whenever I write the first thing I do is try to outline where I want to go. This involved Mike being approached by his friend Bob (who was called Robert) at that point. Obviously he had to join the company and walk into chaos, I tried to describe a bad day we could all relate to.
As the team learns they begin to invest in more frequent releases. I wanted to explain as many of the good reasons why this was as good an idea as possible. The reduced technical risk, the reduced delivery risks, and the increased ability. I also wanted to discuss some of the common objections. Before moving onto discussing Continuous Delivery and Continuous Deployments and how using these techniques makes it less likely your sprints will fail and makes it easier to help your customer with your resorting to release branching strategies.
Once I’d outlined the story and had a basic idea of the characters it was time to sit down and write. In reality it only took a couple of months to create a first draft. Knowing where I am going always makes it a lot easier to put words in a page.
Once I’d finished writing I printed everything off and put it on a shelf for a few months. I wanted to forget as much as I could before I started proof reading so I could spot as many errors as possible.
Many of my colleagues found me over this period sat throughout lunchtime with a stack of paper and a highlighter pen. Believe me, I found a lot of things which didn’t make sense.
Once I’d corrected as much as I could it was time to publish. I’d already created my LeanPub account and in true agile style I decided it was best not to procrastinate and to start gathering feedback. The great thing about LeanPub is that it’s very easy to update your book in response to suggestions.
So that’s the story, I’ve now sold a handful of copies and so far the feedback has been very positive. I probably shouldn’t but I’m already thinking about what I should write next!
A few weeks ago I went to a talk at Agile Yorkshire by Tom Hoyland where he discussed the how he formed an agile team. In it he claimed that Velocity is a Vanity Metric.
If you haven’t read it then I highly recommend you pick up a copy of The Lean Startup by Eric Ries, this was my main introduction to the term. In the book Eric talks about how a Vanity Metric is any figure which is skewed to inflate the success of a product. One of the example he gives is Number of Registered Users, a figure which will clearly go up and up over time. A better metric, Ries argues, would be Number of Active Users. Obviously if you’re looking for a successful product you’re more interested in how many users are currently using it than how many people completed the signup process.
He claims that by basing metrics and KPIs on these Vanity Metrics is akin to giving yourself a pat on the back for building a successful product while sticking your head in the sand about why your business was losing money (or the startup running out of runway as he describes it in his book).
So, to return to Tom’s question – is Velocity a vanity metric?
Most Scrum Masters calculate a teams’ velocity by taking an average over the previous sprints. The number of sprints varies but six (around 12 weeks of work) is the norm. Using this figure the the team can then calculate how far a particular story is away from delivery or epic from completion.
There’s no doubt that Velocity is a useful forecasting tool (although, as one of my colleagues pointed out recently that if you plan a sprint on your average velocity then you will, by definition, fail 50% of the time). However, is it deluding us into thinking we’re a successful team and detracting us from more accurate measurement?
I think this comes down to how you measure success. Most people would agree that judging a development team on how many features they can deliver is fairly narrow minded. A development team includes a Product Owner, their remit should be to develop a product not to simply crunch features. If that was their goal then they could simply create huge numbers of easy, yet useless features just to score points.
In this new age of DevOps a Development Team should use their PO’s expertise and take ownership of the success of their product. For them to be judged on how much work they produce, rather than how much success they’ve had is a limited measure. It reminds me of the from the book The Goal where they measured and optimised the workstations rather than asking whether the system was effective.
Therefore, although shocked when I heard it I agree with Tom. Velocity is a useful tool for the team to forecast. But if it’s chased as a KPI or used to measure the team then it is indeed a vanity metric and will distract the team from trying to improve their product. If we want to measure our teams’ success then we should look at the metrics of our products, not try to calculate some kind of Hours to Story Point ratio or chase an ever increasing Velocity. We should focus on Number of Active Users or Number of Requests via the API, this will measure the teams’ success, rather than it’s productivity. As a friend of mine is fond of saying, effective rather than efficient.
I attended Agile Yorkshire last week and saw two great talks by Tom Hoyland and Jon Fulton. I really enjoyed both but a few points in Tom’s talk really interested me and I’d like to take a few minutes to share them.
Tom is a Scrum Master at Sky Betting and Gaming, I’ve heard good things about the company in the past so I was interested to hear one of their success stories. It turns out that Tom was part of a team of twelve who really stripped Agile “back to basics” and conducted a series of experiments on the road to continuous delivery. Working in a regulated industry myself I was intrigued how they’d got on.
One of the first things Tom talked about was how many different people in the team came to the table with ideas of what was agile best practice. We all laughed at his “my guru is better than your guru” but it makes a lot of sense! I am heavily influenced by Jez Humble, Gene Kim, and Clarke Ching but many of my colleagues may watch talks and read blogs from very different thought leaders. Tom explained that one of the first thing they had to do in the team’s formation was break many of the concepts down to their fundamental concepts and understand what worked for them.
Something else Tom discussed was how the team consolidated their own backlog. This was not controversial, how else would you prioritise the work against it? It was only when he gave examples of some of the different backlogs they’re identified that I became intrigued. Risk Logs, Retro Actions, and Design Session – all of these moved onto the board and each became visible and prioritised.
It’s dangerous out there – take your buddy! I’ve heard many of the advocates or pair programming before but the idea of your buddy following you into meetings, design sessions, and CABs!? Tom explained that if your buddy went with you to these sessions not only would they learn how to design, and walk work through the CAB but they’d also know the current state of everything you were working on. If you were sick or the proverbial bus came along the team wouldn’t need to bother you because everyone would know what was going on.
There were many other good ideas (and I intend to borrow quite a few of them myself) but the final one I’m going to mention was the idea that Velocity is in fact a vanity metric (read The Lean Startup if you have no idea what I’m talking about). Velocity is just a number, like Number of Users or Number of Page Views). What we want are actionable metrics, like team predictability and accuracy of forecasts. As a Development Manager I frequently use the team’s average velocity to forecast delivery dates, Tom recommended that there are better measures out there such as a temperature check of the team’s current mood (which would often dip before any reduction in velocity). It’s an interesting idea, and one I intend to think more about over the next few weeks!
A big thank you to Royd and the guys who put Agile Yorkshire together each month. An equally big thank you to Tom and Jon for their great talks!
I have recently coined the term Lock Concept as a symptom of what many people call Fake Agile. Allow me to explain…
Waterfall development is often described with the Design, Development, and Testing phase structure. Many teams adopting Scrum tend to fall into one of two mistakes.
The first mistake is to split up these into sprints. So Sprint 1 is for design. Sprints 2, 3, and 4 are for development and testing and bug fixing will go into Sprints 5 and 6. This isn’t Scrum. Clarke Ching uses a phrase I like in his book Rolling Rocks Downhill, he talks about GETS software. That’s Good Enough To Ship, at the end of each sprint the software must be production ready. By falling into the sprint phase trap you’re lowering quality between releases and not realising the value of scrum.
The second mistake teams make is to try and run each Sprint as a mini waterfall. This is what I now describe as The Lock Complex. Teams falling into this trap will design in the first few days, develop for a few more, and then test their work towards the end. Yes, the software is GETS at the end… but doesn’t this look like a waterfall on a smaller scale?
The main symptom with this approach is people twiddling their thumbs (testers at the start of the sprint and developers at the end). While wasted time is frustrating, the real problem is the lack of shared knowledge and by unlocking that you can quickly raise your game towards Continuous Delivery.
The way to solve this becomes quite apparent if you look at the DevOps utopia we’re all told about. In a world of Continuous Delivery and automated approvals we create automated acceptance tests to ensure that our code functions as expected. If the feature doesn’t meet these automated tests then it will not be merged in, or if it has been then the deployment pipeline will stop.
In this world, not only are we deploying faster and achieving single piece flow but we’re breaking that Lock Complex. People are busy all the time and pair and mob programming becomes the norm. Instead of having a testing phase where it’s our QA’s engineers’ time to shine we have continuous collaboration and our quality specialists advising on the best tests and mechanisms to be implemented. Testers no longer run manual tests, we get computers to do that. Testers work to ensure that the automated tests give us a coherent test strategy.
If we can help our teams to break the Lock Complex and stop working in mini-waterfall sprints then we’ll see the benefits as people collaborate more and achieve better velocities and higher qualities as a result.